Question: How to handle data when the app is on multiple servers? Please provide resources, videos, links.
Here's the bottom line: I am new to cloud development. Like, fully new. I always programmed on servers where there was one server, data stored on the disk, and me doing what I want on that disk data.
Cloud feels different. Every read/write operation feels like to cost a lot (compared to a simple disk read/write) because you have to query an external server to get the data. There's no "hard disk" on AppEngine. You can't write a file. Global variables are inconsistent across servers. But that's how to make a scalable app.
But how to handle & overcome all that challenges I don't understand? That's where I need your help: I need someone who explains me how to think of data storage for the cloud, and not like how traditional people does.
What I want to do on AppEngine is, basically, a one-to-one web chat over AJAX/Channel API. My question is:
- I heard about Channel API, how could it help me?
- Where chat messages should travel for? I mean, a chat server needs to handle a message travelling to other users in real time, and doesn't need to store it. However, if there's one instance on Server 1 with my sender in Europe, and one instance on Server 2 with the recipient in US, how to communicate the message through Server 1 to Server 2 to send it to the US recipient? Should I use DataStore? Memcache? Google Cloud Storage API that stores files much like Drive does?
- How Channel API plays for the storage question? I don't need to store message if it is sent right away to the person.
- 100k/ops = $0.06. The Free version includes 50k read/write/small on DataStore without cache. Considering it takes 3 writes to write a message (is that even realistic?), it means I can handle 17,000 messages/day for free on my chat. For $0.06, it means I can handle 33,000 msgs/day, or my website would cost at least $2/mo to run the website with 33,000 msgs/day. Is DataStore really appropriate for me here?
Considering a lonely user is doing 2 message each 10 second (is that realistic?), it means with 17,000 messages, it means I can sustain 24 users staying 1 hour on the website.
- In general, is there is guides to how to handle storage on AppEngine while I came from the "one-server" development world? Or to the idea people in the same room are on the same server? (that's what happens on IRC, for example)
- Could I enforce the fact one user and the other is on the same instance and should stay on the same instance? I think no, as a part of scalability rules, but still.
Thanks in advance.