Django — How to load data from Bank statements of 1S in MySQL?

Example took from here
https://habr.com/post/245397/

Bank statement (. txt)that you want to process.
spoiler
1CClientBankExchange
Wireformat=1.02
Encoding=Windows
Sender=System Sun Internet Bank
Receiver=Accounting, revision 4.4
Datacache=27.11.2013
Datacons=27.11.2014
Rasschet=12345678901234567890
Soccerschool
Datacache=27.11.2013
Datacons=27.11.2014
Rasschet=12345678901234567890
Nachalnichek=0
Have sagapolutele=68770
Segobiano=68770
Chechnyatoday=0
Concrescent
Sentidocomun=Bank order
Room=1
Date=26.12.2013
Amount=1000
Dataspace=26.12.2013
Have decapotabila=
Platelike=12345678901234567890
Payer=INN 778899001122 Ivanov Ivan Ivanovich (SP)
Platelike= 778899001122
Плательщик1=Ivanov Ivan Ivanovich (SP)
Plattenstrasse=12345678901234567890
ПлательщикБанк1=AKB "the Sun"
ПлательщикБанк2=MOSCOW
Platelike=044525201
Plateletcrit=98765432198765432100
Plateletcrit=98765432198765432100
Procatalyse=89765456787654345678
Receiver=INN 7765434566\775001001 AKB "the Sun"
Polycatechins=7765434566
Получатель1=AKB "the Sun"
Procatalyse=89765456787654345678
ПолучательБанк1=AKB "the Sun"
ПолучательБанк2=MOSCOW
Poluchelovek=044525201
Poluchalasy=98765432198765432100
Videoplay=17
Naznacheniia=Commission for opening an account according to the tariffs of JSCB "Sun"
Connecticutt
Sentidocomun=Payment order
Number=176
Date=26.12.2013
Amount=4770
Dataspace=
Payer=INN 7725747515\772501001 LLC "WIESN"
Platelike=7725747515
Platelike=40702810822000034869
Плательщик1=LLC "WIESN"
Plattenstrasse=40702810822000034869
ПлательщикБанк1=AKB "ABSOLUT BANK" (OAO)
ПлательщикБанк2=MOSCOW
Platelike=044525976
Plateletcrit=30101810500000000976
Have decapotabila=26.12.2013
Receiver=INN 778899001122 SP Ivan Ivanov
Polycatechins=778899001122
Procatalyse=12345678901234567890
Получатель1=SP Ivan Ivanov
Procatalyse=12345678901234567890
ПолучательБанк1=AKB "the Sun"
ПолучательБанк2=MOSCOW
Poluchelovek=044525201
Poluchalasy=98765432198765432100
Videoplay=01
Crocethia=26.12.2013
Priority=5
Naznacheniia=Payment of invoice No. 1, dated 26 December (for site maintenance ) is not subject to VAT.
Connecticutt
Sentidocomun=Bank order
Room=1
Date=10.01.2014
Amount=20
Dataspace=10.01.2014
Have decapotabila=
Platelike=12345678901234567890
Payer=INN 778899001122 Ivanov Ivan Ivanovich (SP)
Platelike=778899001122
Плательщик1=Ivanov Ivan Ivanovich (SP)
Plattenstrasse=12345678901234567890
ПлательщикБанк1=AKB "the Sun"
ПлательщикБанк2=MOSCOW
Platelike=044525201
Plateletcrit=98765432198765432100
Procatalyse=70601810200021210220
Receiver=INN 7765434566\775001001 AKB "the Sun"
Polycatechins=7765434566
Получатель1=AKB "the Sun"
Procatalyse=70601810200021210220
ПолучательБанк1=AKB "the Sun"
ПолучательБанк2=MOSCOW
Poluchelovek=044525201
Poluchalasy=98765432198765432100
Videoplay=17
Naznacheniia=Charge for reception and processing of payment documents
Connecticutt
Sentidocomun=Payment order
Room=1
Date=10.01.2014
Amount=3500
Dataspace=10.01.2014
Payer=INN 778899001122\772801001 Ivanov Ivan Ivanovich (SP)
Platelike=778899001122
Platelike=12345678901234567890
Плательщик1=Ivanov Ivan Ivanovich (SP)
Plattenstrasse=12345678901234567890
ПлательщикБанк1=AKB "the Sun"
ПлательщикБанк2=MOSCOW
Platelike=044525201
Plateletcrit=98765432198765432100
Have decapotabila=
Receiver=INN 7707704692\772801001 JSC "unified electronic trading platform"
Polycatechins=7707704692
Procatalyse=40702810000760001497
Получатель1=JSC "unified electronic trading platform"
Procatalyse=40702810000760001497
ПолучательБанк1=OJSC "BANK of MOSCOW"
ПолучательБанк2=MOSCOW
Poluchelovek=044525219
Poluchalasy=30101810500000000219
Videoplay=01
Statusstatus=01
Platelike=772801001
Poluchal=772801001
Pokazatelj=00000000000000000000
OKATO=0
Pokazatelnoe=0
Pokazatelnymi=0
Pokazatelnoe=0
Pokazatelyami=10.01.2014
Pokazatelya=0
Crocethia=10.01.2014
Priority=5
Naznacheniia=Amount 3500.00 including VAT 388,99
Connecticutt
Codecfile


Python code
# Empty lists (arrays)
a = [] # Here are lines of the file
b = [] # Here the rows of the file, separated by =, where b[i][0] = "Key", b[i][1] = "Value"
main = [] # Basic information
f = open("f.txt") # Open file statements f.txt
a = f.readlines() # Read the file line by line
l = [ line.rstrip() for line in a ] # Remove the \n
for x in l:
 b.append(x.split("=")) # to Cut the string at the = symbol and add to list b
# Read the basic info (to the first line 'Sentidocomun')
for i in range(len(b)):
 if b[i][0] == 'Sentidocomun':
break
else:
 main.append(b[i]) # Add basic info to the main list

key = 0
doc = []
q = {key: []} # Make a dictionary to store documents
# Start to share the documents
for k in range(len(main), len(b)):
 if b[k][0] == 'of Connecticutt': # 1 the document is finished
 q[key] = doc # Add the document to the dictionary
 doc = [] # Clear the list doc
 key += 1 # the dictionary Key increase +1 for trail. document
 elif b[k][0] == 'of Codecfile': # If the end of file, then
 close(f) # Close the file
 break # Exit
else:
 doc.append(b[k][1]) # While the document is not over, 
 # Add the line (after the = sign) to the list doc


Then wanted to create in models.py the field and load cycle of dictionary data in those fields. But the trick what, eg., roughly speaking, in the document (Sentidocomun) 1 - lines 17, in the document 2 - 19, document 3 - 23.

Accordingly , for 1 have 17 fields, 2 - 19, etc. moreover, in the document, some rows are repeated, some have been reversed. I.e. the format is not fixed.

How to implement?
March 12th 20 at 08:03
1 answer
March 12th 20 at 08:05
Solution
1 -to find Out how many different unique fields we have to work on the basis of these data it was possible to create a table in the database and prevent a situation that in the statement this field is, and is not in the database.
2 - When parsing/unload statement to remove the duplicate fields, if we take the example of the statement that Plateletcrit=98765432198765432100 is duplicated and one line should be thrown out
3 - Upload data to database may remain empty field, but if the statements the number of fields is not the same then this can't go anywhere.

Either convert all the fields in the XML or JSON string and save as is.
Understand, in principle, and wanted. Thank you.
Was wondering if there's a more elegant solution.
I will try to implement. - ashtyn82 commented on March 12th 20 at 08:08
@ashtyn82, may have, but I figured it out, let's see what else will advise. - Monica_Fu commented on March 12th 20 at 08:11
Did. Though a bit clumsy, but works) - ashtyn82 commented on March 12th 20 at 08:14
@ashtyn82, sorry that nobody replied, curious are there any other solutions to the problem - Monica_Fu commented on March 12th 20 at 08:17

Find more questions by tags Django