edit-22   Chapter:  108  

In this script specific directories are searched for the extension *.ipynb

In this script specific directories are searched for the extension *. ipynb and post the results as blobs in an Sqlite3 database. This stores a notebook as a whole entity. Not as a line by line searchable file.

import sqlite3
#database = "FTS4_IPYNB. db"
database = "FTS4_IPYNB_indexed. db"
conn = sqlite3. connect(database)
c = conn. cursor()
conn. text_factory=str 
c. execute(""
USING FTS4(file, content, description);
conn. commit()
conn. close()
conn = sqlite3. connect(database)
c = conn. cursor()
while count<19:
    if count==1:PATH = "/home/jack/Desktop/deep-dream-generator/notebooks/"
    if count==2:PATH = "/home/jack/Desktop/text_stuff/"
    if count==3:PATH = "/home/jack/Desktop/imagebot/"
    if count==4:PATH = "/home/jack/Desktop/Snippet_Warehouse/"
    if count==5:PATH = "/home/jack/Desktop/gitjupyter/"
    if count==6:PATH = "/home/jack/Desktop/jack_watch/"
    if count==7:PATH = "/home/jack/Desktop/jack_watch/nltk/"
    if count==8:PATH = "/home/jack/Desktop/jack_watch/Python-Lectures/"
    if count==9:PATH = "/home/jack/Desktop/jack_watch/jupyter_examples-master/"
    if count==10:PATH = "/home/jack/Desktop/Books/numerical-python-book-code/"
    if count==11:PATH = "/home/jack/Desktop/Books/pydata-book/"
    if count==12:PATH = "/home/jack/Desktop/Ruby/"
    if count==13:PATH = "/home/jack/Desktop/alice/ChatterBot/"
    if count==14:PATH = "/home/jack/Desktop/deep-dream-generator/LOCAL-notebooks/"
    if count==15:PATH = "/home/jack/Desktop/numpy-array-filters/"
    if count==16:PATH = "/home/jack/Desktop/pycode/"
    if count==17:PATH = "/home/jack/Desktop/pycode/vpython2/TrigonometryBot/"
    if count==18:PATH = "/home/jack/Desktop/temp/args_csv_Twython_ImageBot/"
    if count==19:PATH = "/home/jack/python3-starter/notebooks/"
    for file in os. listdir(PATH):
        if file. endswith(". ipynb"):
            filename = PATH+file
            filein = PATH
            filein = filein. replace("/home/jack/", "")
            filein = filein. replace("/", "_")
            filein = filein+file
            description = filein
            description = description. replace("_", " ")
            description = description. replace("-", " ")
            description = description. replace("/", " ")
            description = description+"
            with open(filename, "rb") as input_file:
                    ablob = input_file. read()
                    content  = sqlite3. Binary(ablob)
                    c. execute("INSERT INTO ipynb (file, content, description) VALUES(?, ?, ?)", 
                              (filein, content, description))
                    print os. path. join(PATH, file, filein)
                    conn. commit()
            line = file
            #line ="Good-mouse-sizing-and-cropping. ipynb"
            title = "index"
            c. execute("INSERT INTO ipynb VALUES (?,?,?)", (title, file, description)) 
            conn. commit()

c. close()
conn. close() 

KEYWORDS:   search extension *.ipynb, walk directories, search directories

edit-21   Chapter:  107  

Verify a file as an SQLite database

The following script will verify a file as an SQLite database. It prints the first 18 characters of the file. The first 19 characters identity a database with this string "SQLite format 3@ "

import string
count = 0
with open("words. db", "r") as myfile:
    for line in myfile:
        printable = set(string. printable)
        line = filter(lambda x: x in printable, line)        
        line = str(line)
        lines = line. strip()
        if len(lines) > 2:
            print lines[0:18]
            count =count +1
            if count > 0:break

SQLite format 3@

KEYWORDS:   verify SQLite, verify database, verify SQLite database, verify database file

edit-20   Chapter:  106  

Using Python to get filesize

Getting the file size of the list generated in:
Chapter 104 'Recursively Finding All Files With a Specific Extension' The list is a list of databases on my computer. I wanted to get the file size of each *. db.
in my case the full path was not required, I just wanted the file name.

import os
count = 0
fn = 0
title = "database. list"
for lines in open(title,"r"). readlines():
    fn = fn + 1
    filename = os. path. basename(lines)
    lines = lines. replace("
    b = os. path. getsize(lines)
    filename = filename. replace("
    print fn," :",b,"-",filename
    count = count +1

Results Example

1  : 7005179904 - unique-End2.db
2  : 14336 - AssetVersioning.db
3  : 3072 - ShaderCache.db
4  : 4096 - city.db
5  : 1798 - iu_mien_samp.db
6  : 29121 - MDFSampl.db
7  : 706 - MDF_AltH.db
8  : 2048 - test.db

KEYWORDS:   get filesize, filesize, what is the size of a file, size of file

edit-19   Chapter:  105  

Count number of lines in a text file, print a text file line for line.

This script will open a text file, count the number of lines it contains, then read it line for line and print the results.  This particular script is used to read the text created by the script in Chapter 104 'Recursively Finding All Files With a Specific Extension'.  

fname = "database. list"
def file_len(fname):
    #open the file database. txt
    with open(fname) as f:
        #count the lines
        for i, l in enumerate(f):
    total = i + 1
    # return the total lines
    return total

def readAll(fname):
    #open the file and store the lines in memory
    #this is not good for large files because the lines are stored in memory
    f = open(fname , 'r' )
    lines = f. readlines()
    f. close()
    return lines

tl = file_len(fname)
print "Total Lines : ",tl
for line in readAll(fname):
    line = line. replace("
", "")
    print line

KEYWORDS:   count lines text file, print text file line for line, print line for line

edit-18   Chapter:  104  

Recursively Finding All Files With a Specific Extension

This script will recursively locate all the files with a defined extension. As it finds the path to the file, the path is listed in a file. This works with all extensions. For example if you want to find all mp3s or find all mp4. In this case it is searching for all files ending in *. db.

import os
import os. path
title = "database. list"
f= open(title,"w");f. close()
for dirpath, dirnames, filenames in os. walk("/home"):
    filenames = [f for f in filenames if not f[0] == '. ']
    dirnames[:] = [d for d in dirnames if not d[0] == '. ']
    for filename in [f for f in filenames if f. endswith(". db")]:
        Path = os. path. join(dirpath, filename)
        with open(title, 'a') as outfile:
            path = Path+"
            outfile. write(path)

Here is an example of the files in database. list file.

/home/jack/unique-End2. db
/home/jack/rollerball/Library/AssetVersioning. db
/home/jack/rollerball/Library/ShaderCache. db
/home/jack/nltk_data/corpora/city_database/city. db
/home/jack/nltk_data/corpora/toolbox/iu_mien_samp. db
/home/jack/nltk_data/corpora/toolbox/MDF/MDFSampl. db
/home/jack/nltk_data/corpora/toolbox/MDF/MDF_AltH. db
/home/jack/sketchbook/jack/Sqlite_FTS4/data/test. db
/home/jack/sketchbook/jack/SQLite_update_by_id/data/test. db
/home/jack/sketchbook/jack/Sqlite_Delete_by_id/data/test. db
/home/jack/sketchbook/jack/SQLite_read/data/test. db

KEYWORDS:   recursive, recursively, find, locate, recursively find extension