I have a problem

I have to create an export of a query result, the number of rows always are more than 2Millons

My proccess starts through a JSP file
The app is running into a Apache Tomcat 5.5

I have tried to write a txt file but the procces is to slow

I don't know if you can tell me some tips ?

Have a nice day ^^

Marcos

2 millon rows is alot to export no matter what you do. One suggestion would be to split up the query into multiple threads. Basically you create a thread to do the query and some number of threads to write out separate files. This isn't easy but you'd then be limited by the I/O of the system and not anything that java is doing.

This message was edited 1 time. Last update was at 08/09/2006 12:37:13

yeap, i already use threads in range of 100,000 ,

but i have to create only one .txt file, it was easy but is still slow

I guess that an important question would be what do you do with data after it is exported? If you are reading it with a different Java process then you might get some performance gains with serialization. However, that would require Java to read the file. If this data is to be imported to something else then can you skip the export to file altogether and write it directly to the destination?

Lastly, do you know where your bottleneck really is? Is it Java or the database or the file I/O or what? Before going too far in optimizing the code you might want make sure that you are optimizing the correct thing.

This message was edited 6 times. Last update was at 08/09/2006 13:00:10

mmmmmm,

This code have to generate a file with the calls of all customer in a range of dates to other company.

Basically is to send the calls in/out of each customer to fill their DB with this information, cause they have to know when they are the destiny/origin of the call.

The only way is the file =(, cause we dont have the same db estructure

And the proccess starts ina a JSP before a search, cause my client dont have a database administrator and is more easy in this way to my client

I dont know if i can run a directly export in the database through JAVA?