I am a complete novice in database/PC application sp please forgive my lack of knowledge.
Let me capture packets to some database instantly to ensure that multiple programs would be capable of monitor physical I/O data being came back via udp packets from the PLC and that i were built with a couple of questions.
Over time it will have to be mix platform but for the moment I am utilizing a C# packet capture library in Home windows. Any suggestions on database type MySQL versus SQlite?
At ~1500 200byte packets a sec, could it be achievable for an place a packet 1500 occasions a sec? I have read that SQlite has some issues with concurency, basically come with an application querying the packet data within the database ~10 occasions a sec on the 25-50ms delay -is the fact that possible?
I be prepared to "only" have to store 20MB approximately of information within the DB at anyone time. Can the database have to run in memory only? When writing the packet data, can the information packet (byte array) be designed in one statement instead of iteratively placing each byte/word? I guess I possibly could transform it into a string however i expect that will allow it to be extremely difficult to question with any speed. I do not use whatever reference to anything just like a "byte array type" most of the databases I briefly checked out. FWIW All of the information is approaching to some devoted NIC on the static IP. The packets are consecutive (I understand it isn't guaranteed with UDP but I have never witnessed one inch order yet) I possibly could stride with the data easily when the database supported an assortment type. -That's good right, no random searches?
Thank you for making the effort to see this.
What's the perceived advantage you are searching for inside a relational database with this? Because you say you are very little into databases, this is a brief of usual explanations why SQL is definitely an options, possibly it will help you clarify your needs as well as your options:
- Queryability. If you wish to expose the information for any wealthy search which includes choices to remove records, to sort results, to aggregate information then indeed SQL databases offer such facilities. They don't come free of charge though. To accelerate searches a database engine needs to duplicate areas of the information into several indexes, which increases the place/update occasions as all individuals indexes need to be maintained.
- Recoverability. Databases can make sure that information is stored inside a consistent condition just in case of the crash. Using either write-ahead log or versioned updates they write alterations in a way than guarantees the customer that after his statement came back to him the alterations it made are durable (I am omitting a lot of particulars for simplicity).
- Consistency. By separating changes between customers until they clearly commit several related procedures the database exposes always a regular condition to some viewer. To do this a database will need to deploy either securing or versioning.
- Scalability. Databases may take proper care of maintaining large teams of data, much bigger than the usual process viable address space. They normally use a buffer pool to help keep hot pages cached and manage the actual file-offset-to-memory-address mapping as well as all of the needed I/O to see from disk and write back changes. They'll also present multiple files being an u . s . storage space, thus surpassing OS quality restrictions, if any.
- Interoperability. Other processes may use standard libraries (ie. ODBC, ADO etc) and languages (SQL) to function around the data, so there's you don't need to create a custom library/access API.
Now, is these required by your scenario? Can there be another thing I overlooked? I am asking these questions because what you would like to attain isn't trivial. You are able to achieve 1500 card inserts per second with relative ease, but is a lot harder to achieve that and offer decent read performance. Plus it appears much of the items relational databases offer (consistency, recoverability, scalability) aren't an objective for you personally. You will find numerous items updated particularly for that in-memory niche which are considerably faster than you'd get from the typical disk oriented relational database.