We launched publicly last Wednesday.
Launching meant promoting the service. This accidentally went too well, Simply Testable received more attention than I expected and all hell broke.
Here's what happened, the resulting traffic numbers and the positive outcomes.
TL;DR, Give Me Stats
|Uniques||Visits||Full-site Tests||HTML Validation Tests|
Note: increasing numbers isn't always pretty. Capacity problems all over the place. Read on.
Web Resources Depot
The post to Web Resources Depot reached over 35,000 RSS subscribers, bringing in 500 direct referrals on the first day. Web Resources Depot has since brought in another 500 direct referrals.
The Web Resources Depot post spawned about 50 re-postings across other sites.
700 new full-site tests were started that day encompassing 300,000 HTML validation tests with dozens running concurrently.
Previous days had seen up to 200 full-site tests started with at most 2 or 3 running concurrently.
A quick install of Go Squared realtime analytics revealed about 20 concurrent users at the Simply Testable homepage for most of the day.
Google Analytics later revealed 918 visits for the day, up from a previous maximum of 70.
The service worked fine on previous days, last Friday not quite so.
Coping with the Web Resources Depot related traffic seemed hard enough. Then Smashing Magazine made the previous traffic boost look like an amateur effort.
At the time I was glad that the Web Resources Depot related traffic was winding down. Casual curiosity users were losing interest and a few interested parties remained. It looked like I could start getting a handle on what on earth was causing everything to break and why everything was running so terribly slowly.
No, no chance of that happening.
As a result of the Smashing Magazine tweet, unique visitors to the Simply Testable homepage reached 2158 for the day. That's up from a previous maximum of 865.
Realtime analytics revealed a pretty constant 40 concurrent users of the homepage, hitting a peak of 53 at one point.
2000 new full-site tests were started that day encompassing 680,000 HTML validation tests with hundreds running concurrently.
At this point, “running concurrently” is valid only from an absolutely technical perspective. Hundreds of full-site tests had been started and had not yet finished. Given weeks or months they might complete. From a user's perspective, all hell ground to a halt.
The queue of HTML validation tests would have taken 5 days to clear if the system was working at full capacity. The load on the system introduced by the increase in interest reduced the HTML validation test throughput significantly. It was looking like the queue would take about 2 weeks to clear.
HTML validation tests were being added to the queue faster than they were being completed. The time required to clear the queue was fast approximating 2 weeks + infinity. That's quite a long time.
The Simply Testable homepage now receives about 5 - 10 concurrent users. Many, sometimes the majority depending on the time of day, are returning visitors. Returning visitors are a very good thing. Returning visitors are gold.
The weekly updates newsletter jumped from 10 subscribers to 17. Those are by no means large numbers, but that does represent 7 people who care enough that they want to receive a weekly digest of the changes to the service and the direction it is taking.
Followers of @simplytestable jumped from 11 to 34. Again, not large numbers compared to many other services but very valuable to me.
I've now coping with the demand. This has happened partly by chance; interest has dropped.
I'm clearing out older tests, leaving newer tests in a position where they can complete. In an ideal world I'd not touch tests. In the current situation, culling old tests is better than having all tests take literally forever.
This leaves me able to see how the system copes with a given constant number of concurrent tests. I can examine the system as the number of concurrent test rises and identify the most clear performance bottlenecks.
As this process continues, the number of concurrent tests that can be supported increases. This is a great way of measuring the how well the system can cope with demand and controls how demand impacts the system to allow improvements to be implemented without widespread breakage.