I am writing a credit card applicatoin in vb.internet 2005. The application reads a spreadsheet right into a DataSet with ADO.Internet and utilizes a column of this table to populate a ListBox. Whenever a ListBox Item is chosen, the consumer will be provided with more information around the selected record.

One thing about this information is not within the DataSet. I must compare a column in the spreadsheet with several exterior data sources to look for the character from the record under consideration. Here's where I've my problem.

This comparison needs to sort through 9.5m rows inside a SQL table at one stage. I have checked and there is not a way to "shrink" the query lower as I am already only searching essential data.

Ultimately the application never noticeably does anything. The CPU usage shoots as much as 100% no matter what it really was at in advance and also the system's performance becomes almost unbearably slow.

Can anybody advise a way I'm able to improve this case although this massive totally running?

EDIT: I had been initially likely to write the items in the 9.5m rows within the database table to some text file which I'd then read from, but after 6.5m rows, I acquired an OutOfMemoryException.

I suspect your CPU may be utilized in inhabiting the DataSet, though you would need to profile the application to verify that. Use a DataReader rather and only storing the outcomes in more compact format in memory or, if you are drained of memory, then writing these to personal files along the way. Using the DataReader approach you won't ever have to keep entire result occur memory simultaneously.

  1. A catalog within the column to look?
  2. A brand new area within the table to assist to look faster?