Martini to Gin, Back to Martini and on to Negroni and Mux

In a previous article I described how I was going to transition one of the apps I work on from Martini to the Gin Gonic framework. I was fearful that Martini wasn’t going to be able to handle the load as the application scaled, only to find out Gin had it’s own set of issues, speed increase aside.

My Expectations

I knew that negroni was simply a middleware handler, it tied together http handlers with services that can be injected before and after requests with ease. It’s a “bring your own everything type of situation. Luckily, I had just done this a few days ago, so I was well versed in knowing what needs to be transitioned over to the new services and what needed to be changed.

Trimming the Fat

This could a factor of the speed, but I did remove any type of binding middleware, as I didn’t really need it and accessing the query string parameters from the http library was easy enough that it didn’t justify needing to use a struct in this case. I didn’t do any benchmarks before I removed it and after I removed it, but I’ve got to believe it did have a factor of what I am seeing.

Speed Issues

Wow. That’s all I can really say. I was seeing an increase in traffic today, upwards of 8k req/sec, about 133 req/sec. Heroku told me that there were 5 instances today where the application timed out, which is too bad. I didn’t do any digging through the logs on exactly what happened, but in times of peak traffic I was experiencing 30ms response time, which was 8x slower than it was a few days ago, and 16x slower than it was with Gin. It’s a shame, even though the application is experiencing a 99.9995% uptime, I don’t see 133 req/sec across 3 small dynos on Heroku as ‘too intensive’, especially since data is stored in an in-memory cache for 15 minutes at a time.

The Negroni/Gorilla Mux Cocktail

Along side negroni, I decided to use Gorilla Mux, partially because I’ve used Gorilla libraries before, and also because it was included in the Negroni docs so why the hell not. I used the unrolled render library, as it was again recommended in the Negroni docs and found it it’s VERY similar to the Martini render middleware. All in all, it took again less than an hour to convert the app from Martini to Negroni. The code I wrote is about 460 lines of code, and I probably had to do small edits to about 100 lines.

After testing everything, and sure to make sure that my unique route I used before where http://example.tld**/keyname.js** was a dynamic javascript, I found out that it was even easier to make work than it was with Martini:

router.HandleFunc("/{key}.js", injectJavascript)

You can’t beat that! So I did some manual tests one more time, and pushed the code and crossed my fingers.

Heroku updates it’s stats every 10 minutes, so I patiently waited to see what the outcome was. Once I confirmed everything worked well on production and saw nothing odd for the stats, I gave it a few hours. It’s been about 5 hours now, and so far everything is great! The 30 sec response time from Martini has dropped to 8 ms on it’s peak, still higher than it was a few days ago with Martini, and a bit higher than it was with Gin, but we are getting more traffic than we were a few days ago.

The CPU load for the Martini app was averaging about 1 for the past 12 hours or so, since the push it’s averaging about .3. Memory has gone from 28 MB in memory, and 3 MB swap (still don’t know what’s up with that, these dynos should give me 512 MB of ram per application so not sure what it means when Heroku is hitting swap memory), since the Negroni/Mux push it’s peaked to 14 MB ram and no swap.

Wrap Up

It’s safe to say that there is a 100% performance increase from Martini to Negroni/Mux, as is expected. This may be my new default setup, I’m working on another app that is using Negroni/Mux and I’ll be using an oauth2 library along with Redis session handling (which I’ll probably lean on Gorilla sessions again for that), so we shall see how performance is with that app, since it will be a much more complex setup.

Edit

The negroni/mux app has been running for 18 hours now, and traffic has picked up quite a bit. Some notes:

  • Response time has stayed low, still averaging 6ms for 6k req/sec
  • CPU load will spike when traffic spikes, which is to be expected. The graphs seem to show more of spikes which is good, versus rolling hills which leads me to believe Martini was still under heavy load even when traffic died off. Not sure why that is the case.
  • Memory has gone up to 21 MB, so it’s possible that the cache just wasn’t warmed up enough yesterday. What wil be interesting to see is if it uses swap memory, and to figure out why it uses swap memory.

One thing to note is that even though the load has risen, under the same conditions it was in yesterday there have been no timeouts yet.

Last Updated: 2015-01-16 19:09:52 +0000 UTC



What are your thoughts on this?

Categories

RSS feed

Follow Doug On Social