Programically convert a fixed width .dat file with a .fmt file into a .csv file using Python or Python/Pandas -
i'm trying learn python i'm stuck here, appreciated.
i have 2 files.
1 .dat file no column headers fixed width containing multiple rows of data 1 .fmt file contains column headers, column length, , datatype
.dat example:
10ifkdhghs34 12ifkdhghh35 53ifhdhgdf33
.fmt example:
id,2,n name,8,c code,2,n
desired output .csv:
id,name,code 10,ifkdhghs,34 12,ifkdhghh,35 53,ifhdhgdf,33
first, i'd parse format file.
with open("format_file.fmt") f: # csv.reader parses away commas # , rows nice lists reader = csv.reader(f) # give list of lists looks # [["id", "2", "n"], ...] row_definitions = list(reader) # iterate , unpack headers # gives ["id", "name", "code"] header = [name name, length, dtype in row_definitions] # [2, 8, 2] lengths = [int(length) name, length, dtype in row_definitions] # define generator (possibly closure) slices # string bit bit -- yields, example, first # characters 0 - 2, characters 2 - 10, 8 - 12 def parse_line(line): start = 0 length in lengths: yield line[start: start+length] start = start + length open(data_file) f: # iterating on file pointer gives each line separately # call list on each use of parse_line force generator list parsed_lines = [list(parse_line(line)) line in data_file] # prepend header table = [header] + parsed_lines # use csv module again output csv open("my_output_file.csv", 'w') f: writer = csv.writer(f) writer.writerows(table)