I'm attempting to load the tomcat log of combined pattern to MySQL db,now I push each record to db when it's parsed.

First,read a line in the InputStream,

then parse it to some LogItem which is often used to wrap a log record,

then While using MySQLDbManager to place the record to db,

MySQLDbManager hold a reference of the database connection pool,once it's known as to push a log, it'll request an association to complete the sql operation, then release it。

I'm afraid if this sounds like low preformance, because the MySQLDbManager can get and release connection all of the occasions throughout the log pushing.

I've thought make a listing in order to save all of the LogItems,following the log file are review, I pass this list towards the MySQLDbManager,nevertheless the log file maybe is the owner of a lot of records, that's to express,it'll result in a log of LogItems to become put in their email list,is safe?

This is actually the two method:

MySQLDbManager dbm = MySQLDbManager.getInstance();
private void pushLogToDbByStream(BufferedReader br) {
    String line;
    int total = 0;
    int error = 0;
    try {
        while ((line = br.readLine()) != null) {
            LogItem li = LogParser.parser(line);        
            dbm.writeLogItemToDb(li);
            total++;
        }
    } catch (IOException e) {
        log.error("Error occur when read log from file");   
    } catch (IllegalStateException e) {
        error++;
        total++;
    }
    log.info(total + " items have been processed, " + error + " failed");
}

public int writeLogItemToDb(LogItem item) {
    //get a connect from the connect pool
    DBConnection dbc = pool.alloc();
    String insertSql = "insert into log values" + item.buildSQLInsertValues();
    //The buildSQLInsertValues() will return something like this--('127.0.0.1','GET','http://www.google.com'...)
    int res;
    try {
        res = dbc.updateSQL(insertSql);
        return res;
    } catch (SQLException e) {
        log.error("error occur when try to insert the item to db +\n" + insertSql + "\n");
        return 0;
    } finally {
        //release the connection
        pool.release(dbc);
    }
}

It is possible to better idea?

BTW,if men think my issue is too elementary to reply to, you are able to reject to reply to,but please not election my publish lower, I'm hardly generating reputations. :)

If you work with an association pool then there's no actual overhead of "the MySQLDbManager can get and release connection all of the occasions throughout the log pushing", since you will not really be frequent lowering and raising physical connections for every request. This is actually the entire reason for an association pool.

Additionally, when you fear so much how something will work, the very best factor you should do is to merely test drive it yourself. Write a fast testbench and find out precisely how fast this code is - the number of logs are you able to push towards the database per second? How lengthy will it decide to try push a 100MB log towards the database?

Only by really testing the code are you going to know any solutions without a doubt.