En cours

Import 5GB .SQL file to MySQL Database

I have very large .sql files that contain tens of millions of "insert" queries; the queries as example:

use x; insert into table1 (xxx,zzz) values ('aaa', 'bbb');

use x; insert into table22 (xxx,zzz) values ('aaa', 'bbb');

use y; insert into table1 (xxx,zzz) values ('aaa', 'bbb');

use g; insert into table5 (xxx,zzz) values ('aaa', 'bbb');

and so on... millions of these rows have been found in the file, and it has to enter data to multiple databases and multiple tables inside each database.

So, I tried to upload these files using the following ways, but all of them required a real huge time, and all these ways was capable to insert only 1200 - 2000 rows per 1 minute. So, it is not perfect solution for me, because I'll need tens of thousands of hours to complete all the rows.

The ways:

1- I tried to use PHPMyAdmin import stage to upload these files, and I have changed all the resources in [url removed, login to view] and [url removed, login to view] to unlimited values, but the import was very slow, about 1200 - 2000 rows per 1 minute.

2- I tried to use shell commands to import these files, and the speed was same, about 1500 per minute.

3- I used [url removed, login to view], and the speed was same!!!.

4- I built a php script to enter these rows, but it takes 1 minute to insert 1200 rows. Same!!!

Note: the size of each file is about 500 MB, but it doesn't matter, because when I tried to upload only 16MB, I got the same results, 1500 or 2000 rows per 1 minute.

Anyways, If you have a perfect way to upload these files in reasonable time, please bid on this project, and I'll provide you the details using PMB.

Remember:

1- I'll provide you a test .sql file of 500 MB to upload it for me, and you should give me a feedback about the speed. This is something like DEMO!, and I won't pick you without see a successfull demo.

2- I use a localhost server not online hosting. It is Apache 2.5.8.

Compétences : Apache, MySQL, Oracle, Script Shell, SQL

Voir plus : mysql import 5gb database, 5gb sql import, import 5gb sql file mysql, import 5gb mysql database, mysql 5gb database import, sql file mysql apache, php script mysql database import sql file, import 5gb sql, mysql import 5gb sql file, shell script import multiple sql mysql, 5gb sql file, import 5gb sql file, 5gb sql database import, import 5gb database, sql file, upload sql file, mysql database import 5gb, sql to oracle, online oracle database, mysql database server, my sql databases, data stage 8.1, import 5gb mysql database mysql, mysql import 5gb sql, mysql import 5gb database time

Concernant l'employeur :
( 10 commentaires ) ankra, Iraq

N° du projet : #1080336

Décerné à :

gangabass

I can do this for you. See PM for details.

50 $ USD en 1 jour
(60 Commentaires)
4.8

5 freelance ont fait une offre moyenne de 48 $ pour ce travail

ciachn

Hello Sir. I can do this in about an hour right on your server.

40 $ USD en 1 jour
(4 Commentaires)
4.6
easycoders

Will you please provide me one file to check before going to final deal.

50 $ USD en 1 jour
(16 Commentaires)
4.2
TheCammelCoder

Ready to start when you are

50 $ USD en 1 jour
(3 Commentaires)
3.2
expertj

Solution is ready. Please check your PM

30 $ USD en 2 jours
(0 Commentaires)
0.0
fcortijo

Please check PM.

50 $ USD en 1 jour
(0 Commentaires)
0.0