I will be developing a pretty big system that must have the "characteristics" of a big system. This technique come in the financial space, so you will find some mandatory stuff that are needed (99.999%, fail over, load balancing etc.). The machine is much more of the switch a "web" application.
I've been doing allot of research... Especially current Large systems like twitter, ebay, amazon . com, facebook and so on, but regrettably they're mostly internet based. I've selected the Java root like me quiet proficient for the reason that language although I'm available to anything new.
I've spent hrs reading through up within Stack overflow and also the one factor I gather is the fact that there's no "Silver Bullet". The machine will have to be fine updated/developed particularly to me.
Making this my current mindset and ideas...
I'm pretty impressed with Mysql 7 as well as their clustering abilities. I'd produce a Mysql cluster of the couple of machines. This cluster would then be duplicated to a different off-site machine using their async mater-slave replication. The mysql cluster could be fronted with a mysql load balancer. May be the NoSQL root better maybe?
HAProxy looks great however i might be satisfied with a hardware solution...
After experimenting with numerous AS servers (Glassfish, JBoss along with a couple of others), I have started to the final outcome that j2EE limits me inside a couple of things that cause create problems down the road (see this previous question). Things I am searching for is really a framework for java that allows me to modularize my code. Essentially write "JARS"/modules that may be distribute right into a cluster of machines that load balance and also have failover.
For instance - Allows say I've Module A and Module B. I wish to be capable of deploy the 2 modules both on two machines OR module A on a single machine and B alternatively. In addition to have module A lot good balance to B (if B is on a number of machines). Is accomplished by glassfish3 and EJB? I like the thought of scaling out and never up, I'd favour 50 machines inside a cluster than X insanely spec'ed machines (type of googles mindset, I believe) I believed to do it on your own but re-inventing the wheel is my second choice.
I'm searching at hazelcast for any memory cache like me most impressed with this solution.
Any input,feedback, links, key phrases or guidlines is going to be greatly appreciated :) I additionally don't mind non-free solutions.
EDIT: I ought to complete the particulars missing as stated by Jeremy:
- By Switch I am talking about something such as PayPal
- You will see potentially large volumes (I cant give a quote)
- Abilities and budget dont may play a role when i fortunately possess the assets.
It's nearly impossible for anybody here to provide cogent advice without understanding
- the type of application you are building (a "switch"?)
- who you are building it for - the number of people? how can they will use it?
- the abilities from the team of developers
- your budget for that system
However, what it really appears like you've made the decision is one thing that cloud suppliers like Amazon . com AWS provide as they are - a loosely combined, load balanced, highly available infrastructure.
EDIT according to your edit:
Don't build the infrastructure yourself. Others, for example Amazon . com and Microsoft, curently have this solved. Particularly Amazon's option would be PCI compliant and may provide the "five nines" you'll need immediately without you requiring to complete anything.
Rather, concentrate on exactly what the application is going to do and just how it is going to do it dependably and scalably.
At this time I'd suggest employing a few consultants for a few days, minimum. Ask them to come up with 3 or 4 alternative application architectures. Budget $1000/day/consultant minimum to obtain somebody that knows what they are speaking about.
And, keep in mind you will not get sensible solutions or guidance from the free site such as this. If you want to construct a "PayPal killer" you will need to purchase advice from someone who's experienced the trenches about this type of project. That'll be either upfront to consultants or perhaps in equity to some technical co-founder.