I am trying to puzzle out what's the right selection of data storage inside a project I am setting up at this time.

I wish to store data that's the output consequence of powershell scripts. Which means that webmaster of my application will have the ability to write a powershell script which will execute on numerous hosts and they'll publish the outcomes to an information store. Then i wish to query that store inside a flexible manner.

Allow me to clarify. The information that returns in the powershell job isn't a proper object but a vitalOrworth assortment of qualities of objects. So there's no real resist serialize.

Let us say I tell 100 hosts on the WCF plan to execute the 2 powershell instructions Get-Service and obtain-Process and they'll then publish back the outcomes to my data store. I'm not sure the schema of the data in advance.

The thing is not PowerShell nor WCF, but how does one store data that during the time of storing the schema isn't known. And querys is going to be produced by hand via some GUI later on in line with the data that's been saved.

Later on I must have the ability perform query like "Get a listing of hosts which have service X running and process Y running" ?

I am considering at nosql databases instead of relational DBs although not sure what's best.

Grateful for just about any input. /Linus

I'd use ravendb with this. It features a peaceful api so that you can shove the information in to the data store through energy-spend without having the necessity of a c# class or even speak with his c# client in energy spend and get it done this way. Additionally, it has dynamic queries so that you can query the machine the way you want within the WCF system.

If you wish to store data which you don't be aware of structure throughout design time, you've got a couple of options.

Among the choices are:

Store data as xml (in DB or files).

Create schema dynamically to complement dynamic data's structure.

Produce a generic structured schema, where all classes map towards the same table, and all sorts of qualities are dynamically attached qualities.

E.g. (Generic class structure)

GenericClass
{
    GenericProperty[] SimpleProperties;
    Dictionary[string, GenericClass] ComplexProperties;
}

GenericProperty
{
    String Name;
}

StringProperty: GenericProperty
{
    String Value;
}

IntegerProperty: GenericProperty
{
    Integer Value;
}

Using table-per-type on these classes should provide you with generic tables.

If storing the information as XML for an RDBMS does not seem sensible for you (btw, why does not it?), then you will find several NoSQL DBs that will most likely be great options because they are schema-less.

Those I'm able to suggest that you appear at (according to personal expertise, you will find many more that may be relevant) are CouchDB and Riak. Both give a disk-bound key-value datastore in which you store your values as JSON, w/o pre-determining a schema. In the two cases you'll be able to query the information via a Peaceful interface using Javascript.

The option should rely on the quantity of data that you simply expect:

  • Riak is made to operate on multiple nodes, and queries are handled through MapReduce to ensure that processing is shipped between individuals nodes, enabling relatively fast data retrieval for ad-hoc queries. For those who have plenty of data - countless records that you need to run ad-hoc queries, choose this. You'll 'pay' using the added complexity of controlling a cluster, though I'm able to attest that Riak causes it to be relatively painless.
  • CouchDB is made to run on one node. Replication can be done (and simple) but queries run against just one server. It's materialized indices, so queries against existing indices run fast. Ad-hoc queries need a full "table scan" though, and may take minutes on large datasets. OTOH, it's the advantage of a pleasant browser-based interface that Riak lacks within the free version.

I'd recommend trying Couch out first - it is easy to setup and begin having fun with - and find out whether or not this solves your condition. Whether it does not, go for Riak.