IE cache for Ajax requests

A few days ago I ran into an issue that is now obvious but took me a while to figure out. I was programming a chat client and everything was working fine in Firefox and Chrome, but when I tested it on IE (6 and 8) things were not looking so good.

This chat is following the polling pattern, issuing an Ajax call every three seconds to check for any updates and receiving a JSON array with the pending messages, if any. Using prototype.js the code to call the javascript function every three seconds is this

new PeriodicalExecuter(Aspchat.chatRemotePoll, 3);

On IE the chat was initialized properly, the first call to the remote server was working fine, but the periodical poller was not issuing any further calls. At first I thought the problem was on PeriodicalExecuter, but after a bit of debugging I could see “Aspchat.chatRemotePoll” was being called, but the Ajax call inside was apparently ignored.

To make things more interesting, I could see some other Ajax requests were working fine (for example, the one to send messages or to update the user list).

Comparing the requests that were successfully sent with the ones that were ignored, I could see the difference. In the working requests I was using POST (the default when using prototype) but in the ignored calls I was using GET.

Once I saw this, it was easy to diagnose what the problem was. IE was caching the GET requests, even when using AJAX. To be honest, this time I will not even blame IE, since I understand a GET request is subject to cache. In this case, I would even say I prefer the IE behaviour over that of Firefox and Chrome.

There are basically three things you can do to prevent this kind of behaviour:

• The easy way out would be to convert my GET request to a POST one. Alas, I didn’t want to do it because in this case I was being RESTful and I was using the same URL for two different actions. When calling “/aspchat/messages” via GET I’m asking for new messages, but when calling via POST I’m sending a new message to the channel.
• Set HTTP headers to control client cache
• Make the request unique by adding a timestamp (or similar) to the URL

The solution I like the best is the one with HTTP headers, so I just went to my poller action (which by the way is managed via a Rails metal middleware) and added the Cache-Control: no-cache header. Just to be really sure, I also added the timestamp for extra security.

The prototype for the Ajax call with the timestamp looks like this

new Ajax.Request('/aspchat/messages', {
           parameters: {timestamp:new Date().getTime()}, //we need this to avoid IE caching of the AJAX get
           method: 'get',
           onSuccess: function(transport){
               Aspchat.displayChatMessages(eval(transport.responseText));  //pass the JSON array to displayChatMessages
                         }
       });

As an extra ball, you can see how I’m using the onSuccess callback to interpret the JSON I’m receiving from the server.

Now you cannot say you didn’t know your AJAX requests could be cached. Let’s be careful out there.

Multiple rubygems versions, GEM_HOME and GEM_PATH

Installing rubygems is failrly easy and it’s great to have a package manager so you can forget about manually installing and upgrading the components you use. After installing a gem, you can require it from any ruby script and use it hassle-free. Well, given your ruby interpreter can find it.

When you install rubygems, a lot of default configuration is done behind the scenes. If you must see to believe, you can run

gem environment

do you believe me now?

Unless you are on windows, you have probably experienced already that gems can get installed in different locations. If using a superuser account, the global configuration will be used, but with a regular account gems install under your home directory.

If you are not careful about how you install your gems, or if you are using rake gems:install from regular accounts, you might end up installing the same version of a gem twice. That’s not only WET (not DRY, bear with me here) but it eats up your poor HD.

Things can get a lot worse than that. Suppose you are working with both JRuby and Ruby MRI. When you use rubygems from JRuby, it will try to use a different gem location by default. So, depending on how you are installing gems, you could have up to three different copies of exactly the same version.

And if you are on ubuntu and you upgrade from an old version of rubygems to the latest one —you will have to if you install Rails 2.3.4; if you are having problems you can read right here how to update it— you might be surprised that your gems are being installed *again*. The reason is under older versions the default location was “/var/lib/gems” and the latest one defaults to “/usr/lib/ruby/gems”.

Well, four different copies of ActiveRecord 2.3.4 are three and a half more copies than I wanted, mind you.

So.. how can we stop this gem install frenzy? Easy. Don’t use the defaults. Each of your installations is using default values, but they can be easily overridden with command line parameters or much more conveniently with environment variables.

Remember the title of this post? Can you see anything there that would make a good candidate for environment variables? That’s right, all the rubygems versions honor the GEM_HOME and GEM_PATH variables, so if they are set they will be used.

Depending on your OS, you can set these variables in different places. I’m on ubuntu and a bit lazy, so I chose the easiest, which is by adding this to my .bashrc file.

export GEM_HOME=/var/lib/gems/1.8
export GEM_PATH=/var/lib/gems/1.8

And now, no matter what I’m using: Ruby MRI, JRuby, or the latest rubygems, my already installed gems will be used, and the new ones will be put in the same place.

Saving the world is a hard job, but someone has to do it.

masochism: plugin para replicación master-slave en MySQL

Recientemente me he encontrado con la necesidad de trabajar desde Rails contra una base de datos MySQL replicando en modo master-slave.

Replicación Master-Slave In a Nutshell:

La idea de montar un master-slave es tener la base de datos maestra para realizar las modificaciones a la base de datos, y usar la/las bases de datos esclavas para realizar las lecturas. De esta forma, las operaciones de lectura, que son las más habituales, se pueden balancear entre diferentes instancias de una base de datos.

Por otro lado, cada vez que se hace una modificación en la maestra, esta modificación se replica en las esclavas, de forma que los datos siempre están sincronizados.

Esta imagen del manual online de MySQL es muy explicativa

Claramente este esquema nos puede ayudar mucho de cara a la escalabilidad de una aplicación, ya que la base de datos es uno de los cuellos de botella típicos.

Lo que necesitamos en este caso, es asegurarnos de que todas las operaciones de escritura van contra la conexión de la maestra, mientras que las lecturas van por una conexión esclava.

Una de las ventajas de usar un framework- en este caso Rails, pero podría haber sido cualquier otro- es que tu aplicación tiene una arquitectura bien definida. O al menos debería. Esto significa que al usar Rails tenemos bajo control las operaciones que se hacen contra la base de datos, con lo que identificar las de lectura y las de escritura debería ser simple. Si seguimos las reglas del framework, las operaciones contra la base de datos pasarán siempre por el modelo y por los métodos que AR nos proporciona.

Al final, el número de métodos que realmente acaban accediendo a la base de datos son más bien pocos, con lo que podríamos jugar un poco con el framework, establecer dos conexiones con la base de datos (una con la maestra, y otra con la esclava) y decidir qué conexión va a usar cada método. Así todos los métodos “find” y similares irían por la esclava, y los “delete”, “insert”, etc.. irían por la maestra.

Esto mismo es lo que el plugin masochism hace automáticamente por nosotros. El plugin viene de la mano del hiperproductivo Rick Olson.

Uno de estos días tendré que contar en más detalle cómo va, pero el resumen ejecutivo es que lo instalas, le añades una conexión master_connection en tu fichero database.yml, llamas a un método en uno de tus initializers… y te olvidas. A partir de ahí todo funciona de forma transparente (siempre que hayas configurado tu master/slave de MySQL previamente, claro).

De momento le he encontrado una única pega. Entre los métodos que se envían a la base de datos maestra no está el método “execute”, que es el típico método que sólo utilizas cuando quieres modificar algo en la base de datos. Sin ir más lejos, el (imprescindible) plugin Foreign Key Migrations usa ese método para crear las FK, por lo que si no nos aseguramos de que se usa la base de datos maestra, podemos tener problemas.

Le he mandado un parche (en realidad es añadir una palabra y una coma) a Rick , a ver si le parece apropiado incuírlo como parte del plugin.

update: y el patch ya ha sido incluído :)

Y a escalar!!

searchwords: scalability, mysql replication, master-slave, javier ramírez