How to speed up executemany in Python with INSERT IGNORE (mysql)?

Python 3, mysql 5.7, mysql.connector. Insert into database a large number of values (~5 million rows in batches of 5000 rows) using executemany. Some values are not unique. In this case you just want to skip the insert/update/replace it. If you use INSERT when at least one error in the package for 5000 rows, when a duplicate is not inserted the whole package. The insertion speed is high. To solve I tried INSERT IGNORE, the speed drops at times, the impression that the queries are executed separately and not a package. How can I solve this problem without compromising the speed?
June 5th 19 at 21:47

Find more questions by tags Database administrationDatabasesMySQLPythonSQL