I am considering using Java's TaskExecutor to fireplace off asynchronous database creates. Naturally threads don't come free of charge, but presuming I am utilizing a fixed threadpool size say 5-10, how's this an awful idea?

Our application reads from the large file utilizing a buffer and flushes these details to some database after carrying out some data manipulation. Using asynchronous creates appears ideal here to ensure that we are able to continue focusing on the file. What shall we be held missing? Why does not every application use asynchronous creates?

Why does not every application use asynchronous creates?

It's frequently necessary/helpfull/simpler to handle a write failure inside a synchronous manner.

I am unsure a threadpool is even necessary. I'd get a devoted databaseWriter thread which does all writing and error handling for you personally. Something similar to:

 public class AsyncDatabaseWriter implements Runnable {
     private LinkedBlockingQueue<Data> queue = ....
     private volatile boolean terminate = false;

     public void run() {
         while(!terminate) {
            Data data = queue.take();
            // write to database
     public void ScheduleWrite(Data data) {

Personally, i fancy design for utilizing a Proxy for threads out procedures that might have a very long time. I am not to imply this method is preferable to using executors by any means, just adding it as a substitute.

Idea is pretty good whatsoever. Really I simply attempted it yesterday because I desired to produce a copy of online database that has 5 different groups with like 60000 products each.

By moving parse/save operation of every category in to the parallel tasks and partitioning each category import into more compact batches run in parallel I reduced the entire import time from the 3 hrs (believed) to 26 minutes. On the way I discovered good bit of code for splitting the gathering: http://www.vogella.de/articles/JavaAlgorithmsPartitionCollection/article.html

I made use of ThreadPoolTaskExecutor to operate tasks. Your jobs are just simple implementation of Callable interface.

why does not every application use asynchronous creates? - erm because every application does another factor.

are you able to believe some programs don't make use of a database OMG!!!!!!!!!

all joking aside, given while you don't say what your failure methods are - seems like it may be reasonable. What goes on when the write fails? or even the db eliminates in some way

some databases - like sybase - have (or at best had) a factor where they can't stand multiple authors to some single table - all of the authors wound up obstructing one another - so perhaps it will not really make much difference...