Simply Testable Blog

Figuring out how to automate away the pain of routine front-end web testing; the story behind SimplyTestable.com.

211 posts covering the initial idea, growth of the service, features, advances, failures and successes.

  1. Simply Testable Updates #120: Scheduled Tests Coming Soon (Hopefully)

    Our 120th updates newsletter went out today.

    Highlights from this week:

  2. Simply Testable Updates #119: Fixed Faulty Crawling, Scheduled Tests On-Target

    Our 119th updates newsletter went out today.

    Highlights from this week:

  3. Simply Testable Updates #118: Scheduled Tests Coming Next Week

    Our 118th updates newsletter went out today.

    Highlights from this week:

  4. Simply Testable Updates #117: Scheduled Test Development Update

    Our 117th updates newsletter went out today.

    Highlights from this week:

  5. Simply Testable Updates #116: Scheduled Test Development, How To Make Broken Link Checking Slow

    Our 116th updates newsletter went out today.

    Highlights from this week:

  6. Simply Testable Updates #115: Ignoring file:// URLs For JS analysis, Scheduled Test Development

    Our 115th updates newsletter went out today.

    Highlights from this week:

  7. Simply Testable Updates #114: Ignoring Links For Broken Link Checking

    Our 114th updates newsletter went out today.

    Highlights from this week:

  8. Simply Testable Updates #113: We're Profitable(!), Pre-Beta New Test Results, Currency In Transactional Emails

    Our 113th updates newsletter went out today.

    Highlights from this week:

  9. Simply Testable Updates #112: Test Result Improvements Underway

    Our 112th updates newsletter went out today.

    Highlights from this week:

  10. Simply Testable Updates #111: Performance++ Released, URL Discovery Encoding Bug Fixed

    Our 111th updates newsletter went out today.

    Highlights from this week:

  11. Performance Boost By Turning Things Inside Out

    Last Friday I deployed some changes to how test tasks are farmed out to workers. This has resulted in a 46% increase in task throughput and has nearly halved the amount of time it takes to carry out a test.

  12. Simply Testable Updates #110: Broken Link Checking Fixes; Performance++ Nearly Ready

    Our 110th updates newsletter went out today.

    Highlights from this week:

  13. Simply Testable Updates #109: Ignoring resource:// URLs, Performance Changes Underway

    Our 109th updates newsletter went out today.

    Highlights from this week:

  14. Simply Testable Updates #108: Job Performance++, System Stability++

    Our 108th updates newsletter went out today.

    Highlights from this week:

  15. Simply Testable Updates #107: Improvements to: Broken Link Checking, URL Discovery, Background Process Management

    Our 107th updates newsletter went out today.

    Highlights from this week:

  16. Simply Testable Updates #106: Unsendable Emails, UnChimpable Emails, Broken Link Checking Improvements

    Our 106th updates newsletter went out today.

    Highlights from this week:

  17. Simply Testable Updates #105: Handling Unsendable Emails, More Podcast Ad Prep

    Our 105th updates newsletter went out today.

    Highlights from this week:

  18. Simply Testable Updates #104: On Hiding The Free Plan

    Our 104th updates newsletter went out today.

    Highlights from this week:

  19. Simply Testable Updates #103: Oh My Goodness, Ready To Run Ads!!1!

    Our 103rd updates newsletter went out today.

    Highlights from this week:

  20. Simply Testable Updates #102: Podcast Ad Preparation

    Our 102nd updates newsletter went out today.

    Highlights from this week:

  21. Simply Testable Updates #101: Better Homepage and Features Page

    Our 101st updates newsletter went out today.

    Highlights from this week:

  22. Simply Testable Updates #100: Teams, New Account Page

    Our 100th updates newsletter went out today.

    Highlights from this week:

  23. Team Collaboration Features

    We all work in teams to achieve more than we can as individuals.

  24. New Account Page

    The Simply Testable account page evolved slowly as new features were added. It is now new again, shinier and easier to use. It wasn’t always so.

  25. Simply Testable Updates #99: Team Features Coming Soon

    Our 99th updates newsletter went out today.

    Highlights from this week:

  26. Simply Testable Updates #98: Team Features, Account Page Changes

    Our 98th updates newsletter went out today.

    Highlights from this week:

  27. Simply Testable Updates #97: Guest Blog, Cold Emailing Results, Team Features

    Our 97th updates newsletter went out today.

    Highlights from this week:

  28. Simply Testable Updates #96: Cold Emailing, Guest Blogging and Advertising Plans

    Our 96th updates newsletter went out today.

    Highlights from this week:

  29. From a Service to a Product (Product Specs For Developers)

    Simply Testable from the perspective of a service offers automated frontend web testing. From the perspective of a product, Simply Testable offers the aforementioned service at a cost in a way that provides value to the user.

  30. Simply Testable Updates #95: Presenting All Test Types, New User Drip Campaign

    Our 95th updates newsletter went out today.

    Highlights from this week:

  31. Simply Testable Updates #94: Mobile-Ready Dashboard (And Everything Else Too)

    Our 94th updates newsletter went out today.

    Highlights from this week:

  32. Simply Testable Updates #93: Mobile-Ready Test Progress

    Our 93rd updates newsletter went out today.

    Highlights from this week:

  33. Simply Testable Updates #92: Mobile-Ready Test Results

    Our 92nd updates newsletter went out today.

    Highlights from this week:

  34. Simply Testable Updates #91: Mobile-Ready Test Results Underway

    Our 91st updates newsletter went out today.

    Highlights from this week:

  35. Simply Testable Updates #90: Mobile-Ready (And Improved) Task Results

    Our 90th updates newsletter went out today.

    Highlights from this week:

  36. Simply Testable Updates #89: Clearer Test Rejection/Failure Reasons, Mobile-Ready Account and Test History Pages

    Our 89th updates newsletter went out today.

    Highlights from this week:

  37. Simply Testable Updates #88: In-Account Newsletter Management Fixed

    Our 88th updates newsletter went out today.

    Highlights from this week:

  38. Simply Testable Updates #87: Mobile-Friendly, (And Cacheable!) Sign Up, Sign In, Reset Password

    Our 87th updates newsletter went out today.

    Highlights from this week:

  39. Simply Testable Updates #86: Use Malformed Content Types If You Like; Product Info Being Built

    Our 86th updates newsletter went out today.

    Highlights from this week:

  40. Simply Testable Updates #85: Filtered Test History Released, Going Mobile-First

    Our 85th updates newsletter went out today.

    Highlights from this week:

  41. Simply Testable Updates #84: Filtering Test History, Joining The Dots Between Tests

    Our 84th updates newsletter went out today.

    Highlights from this week:

  42. Guide To Reliable HTTP Resource Retrieval In Applications

    You might think that coding up the reliable retrieval of a resource at a given URL over HTTP is easy. You’d be wrong but quite forgivably so.

  43. Simply Testable Updates #83: Improved URL Resolving, Improved Retrieval of Bad SSL Content

    Our 83rd updates newsletter went out today.

    Highlights from this week:

  44. Simply Testable Updates #82: Custom Cookie Support, Better Long URL Support

    Our 82nd updates newsletter went out today.

    Highlights from this week:

  45. Custom Cookie Support (Testing User-Specific Content and Features)

    You can now specify cookies to be set on all HTTP requests made whilst running your tests.

  46. Simply Testable Updates #81: Custom Cookie Support Almost Complete

    Our 81st updates newsletter went out today.

    Highlights from this week:

  47. Simply Testable Updates #80: Stripe Integration Complete (Phew!)

    Our 80th updates newsletter went out today.

    Highlights from this week:

  48. Simply Testable Updates #79: Postmark HTTP API Integration, Stripe Integration Continuation

    Our 79th updates newsletter went out today.

    Highlights from this week:

  49. Simply Testable Updates #78: New Production Server, Stripe Integration Continuation, Cookie-Based Auth Planned

    Our 78th updates newsletter went out today.

    Highlights from this week:

  50. Simply Testable Updates #77: Testing Password-Protected Everything, Graceful Handling of Sites That Don't Respond

    Our 77th updates newsletter went out today.

    Highlights from this week:

  51. Testing Password-Protected Sites

    I’ve been working for the past few weeks on adding the ability to run tests against password-protected sites.

  52. Simply Testable Updates #76: @Passwords Fixed, URLs Resolved Before Test Starts, JS Tests For Password-Protected Sites Is Live

    Our 76th updates newsletter went out today.

    Highlights from this week:

  53. Simply Testable Updates #75: JavaScript Static Analysis For Password Protected Sites Nearly Ready

    Our 75th updates newsletter went out today.

    Highlights from this week:

  54. Much Improved Doctype Support

    I today expanded the range of doctypes we support when carrying out full-site HTML validation and significantly improved the way we check if you’re using a valid doctype.

  55. Simply Testable Updates #74: Doctype Support Madness

    Our 74th updates newsletter went out today.

    Highlights from this week:

  56. CSS Validation For Password-Protected Sites (With Bonus CSS Retrieval Reliability)

    I’ve been working for the past few weeks on adding the ability to run tests against password-protected sites.

  57. Simply Testable Updates #73: Password-Protected CSS Validation, Full Unicode Support, Many Bugs Fixed

    Our 73rd updates newsletter went out today.

    Highlights from this week:

  58. Simply Testable Updates #72: CSS Validation Of Password-Protected Sites Underway

    Our 72nd updates newsletter went out today.

    Highlights from this week:

  59. Simply Testable Updates #71: You Can Now Test Password-Protected Sites

    Our 71st updates newsletter went out today.

    Highlights from this week:

  60. HTML Validation For Password-Protected Sites

    I’ve been working for the past two or so weeks on adding the ability to run tests against password-protected sites.

  61. Simply Testable Updates #70: Password Protected Sites Progress; Bugfix For URLs Returning Internal Server Error

    Our 70th updates newsletter went out today.

    Highlights from this week:

  62. Simply Testable Updates #69: Progress On Testing Password-Protected Sites

    Our 69th updates newsletter went out today.

    Highlights from this week:

  63. Simply Testable Updates #68: HTML Validator Improvements

    Our 68th updates newsletter went out today.

    Highlights from this week:

  64. Simply Testable Updates #67: The Week Where Nothing Happened

    Our 67th updates newsletter went out today.

    Highlights from this week:

  65. Simply Testable Updates #66: New Dashboard Was Released, HTML Validator Improvements On The Way

    Our 66th updates newsletter went out today.

    Highlights from this week:

  66. New Test Dashboard and Test History

    Our New Test Dashboard and Test History Launched!

    From day one the main focus of the service was on letting you run full-site tests that always worked.

    A couple of weeks ago, around a year after the MVP launch, it did. The service works, all the time, for any website *.

  67. Simply Testable Updates #65: New Dashboard and Test History Being Released Today

    Our 65th updates newsletter went out today.

    Highlights from this week:

  68. Simply Testable Updates #64: New Test Dashboard Preview

    Our 64th updates newsletter went out today.

    Highlights from this week:

  69. Simply Testable Updates #63: Link Integrity Improvements, Interview with Web Tester Tom Batey

    Our 63rd updates newsletter went out today.

    Highlights from this week:

  70. Interview With Tom Batey of WebDepend

  71. You Can Now Test Any Website

    The goal all along of Simply Testable has been to let you easily run a set of frontend tests against any website of your choosing with the minimum possible effort.

  72. Simply Testable Updates #62: Share Private Test Results, Filter Results By Test Type, Towards Testing Absolutely Any Site

    Our 62nd updates newsletter went out today.

    Highlights from this week:

  73. Allowing Public Access To Private Test Results

    The Need To Share Test Results

  74. You Can Now Filter Results By Test Type

    See that list of error counts towards the right-hand side above?

  75. Simply Testable Updates #61: Link Integrity Tests Launched, Testing Password-Protected Sites, Sharing Private Test Results

    Our 61st updates newsletter went out today.

    Highlights from this week:

  76. Awesome Link Integrity Testing Now Available (If You Have An Account)

  77. Simply Testable Updates #60: Link Integrity Tests Really Nearly Done

    Our 60th updates newsletter went out today.

    Highlights from this week:

  78. Simply Testable Updates #59: Link Checker Mostly Done, Improved Test Start/Progress/Results Pages

    Our 59th updates newsletter went out today.

    Highlights from this week:

  79. Simply Testable Updates #58: Simplifying Test Start, Progress and Results Pages; New Test Types Planned

    Our 58th updates newsletter went out today.

    Highlights from this week:

  80. Simply Testable Updates #57: Simplifying Test Results Page, Test History Plans, Many Bug Fixes

    Our 57th updates newsletter went out today.

    Highlights from this week:

  81. Simply Testable Updates #56: JSLint Config Options Released, JSLint Bug Fixes

    Our 56th updates newsletter went out today.

    Highlights from this week:

  82. JSLint Config Options Available

    Preamble

    This wins the prize for being a long time coming.

    I revealed in newsletter #21, back in mid-December last year, that the option to run site-wide JavaScript static analysis tests with JSLint had been released to all.

  83. Simply Testable Updates #55: Crawler Improvements, JSLint Config Options Coming, Product Announcements Newsletter

    Our 55th updates newsletter went out today.

    Highlights from this week:

  84. HTML Validation - It Doesn't Matter (Except When It Does)

    Preamble

  85. Simply Testable Updates #54: Crawler Improvements Coming, Introductory Email Course

    Our 54th updates newsletter went out today.

    Highlights from this week:

  86. Site Crawler Released (And The Story Behind It)

    The Simply Testable service lets you run a set of frontend web testing tools against your entire site all in one go.

  87. Simply Testable Updates #53: Site Crawler Is Here, Announcements List, Newsletter Subscribe/Unsubscribe Features

    Our 53rd updates newsletter went out today.

    Highlights from this week:

  88. Simply Testable Updates #52: Site Crawler Progress

    Our 52nd updates newsletter went out today.

    Highlights from this week:

  89. Simply Testable Updates #51: 200 More Help Pages, Origins Of The "Variable" Element, Crawler Plans

    Our 51st updates newsletter went out today.

    Highlights from this week:

  90. Solving 'Element "Variable"' Undefined'

    When preparing a documentation page to cover the generic HTML validation error Element "X" undefined I started researching the causes for some specific instances of this type of error.

  91. Simply Testable Updates #50: Fixes For Common HTML Validation Errors

    Our 50th updates newsletter went out today.

    Highlights from this week:

  92. On Providing the Most Likely Cause For the Top HTML Validation Errors

    Introduction

  93. Simply Testable Updates #49: Hunting Down Common HTML Validation Errors

    Our 49th updates newsletter went out today.

    Highlights from this week:

  94. Finding the Top HTML Validation Errors

    Introduction

  95. Simply Testable Updates #48: Premium Plan 365 Day Trial Offer Until July 31, Infinite Free Trial Hack, Bug fixes

    Our 48th updates newsletter went out today.

    Highlights from this week:

  96. Get an Unlimited Free Trial

    Introduction

    Today I launched our premium plans. All come as standard with a 30 day trial (or 365 days if you upgrade before 31 July 2013).

  97. 365 Day Free Trial Offer

    I wrote recently about the newly-launched premium plans.

  98. Premium Plan Credits

    Premium Plans Have a Monthly Credit Quota

  99. Premium Plans Launched (and Why They Exist)

    Premium Plans Launched

  100. Hello Postmark, More Reliable Mail Delivery

    Moving to More Reliable Mail Delivery

  101. Simply Testable Updates #47: Premium Plans Coming Next Week

    Our 47th updates newsletter went out today.

    Highlights from this week:

  102. Simply Testable Updates #46: No Progress, Moving House, Cat With Broken Leg

    Our 46th updates newsletter went out today.

    Highlights from this week:

  103. Simply Testable Updates #45: Subscription Handling, Startup Engineering, Moving House

    Our 45th updates newsletter went out today.

    Highlights from this week:

  104. Simply Testable Updates #44: Payment Handling Nearly Done

    Our 44th updates newsletter went out today.

    Highlights from this week:

  105. Setting the PHP Memory Limit for Travis CI

    Introduction

    I use Travis CI as a secondary continuous integration (CI) service to help ensure that each Simply Testable application works in isolation as expected when code changes are made.

    Following some recent changes I noticed that the PHP 5.3 build failed due to it hitting a memory limit running unit tests.

  106. Simply Testable Updates #43: Premium Plans Well Underway

    Our 43rd updates newsletter went out today.

    Highlights from this week:

  107. Simply Testable Updates #42: Plans and Pricing Released Unfinished, HTTPS Enabled

    Our 42nd updates newsletter went out today.

    Highlights from this week:

  108. Using a Multiple-Domain (UCC) Secure Certificate With Nginx

    Introduction

  109. Simply Testable Updates #41: Plans Page Mostly Done, New Documentation Site, Also CSS Validator Bug Fix

    Our 41st updates newsletter went out today.

    Highlights from this week:

  110. Simply Testable Updates #40: Your Account, Faster Single-URL Tests, Free Demo Limits, CSS Validation Bug Fixed

    Our 40th updates newsletter went out today.

    Highlights from this week:

  111. Free Demo Test URL Limits

    I’ve finished putting in place limits on free demo tests. I’d like to explain what free demo tests are and the limits that apply to them.

  112. Faster Single-URL Tests

    Running a single-URL test just got faster!

  113. Limits on Free Full-Site and Single-URL Tests

    Limits are starting to be applied for totally free full-site and single-URL tests.

  114. CSS Validation Local Path Reference Bug Fixed

    When performing a single-URL or full-site CSS validation, you can choose to ignore CSS validation errors from chosen domains.

  115. Your Account Page Now Available

    For those of you who have an account, you now have an account page!

  116. Simply Testable Updates #39: Your Account Page Mostly Complete

    Our 39th updates newsletter went out today.

    Highlights from this week:

  117. Simply Testable Updates #38: Stripe Integration Investigation, Account Plans, Your Account Page

    Our 38th updates newsletter went out today.

    Highlights from this week:

  118. Simply Testable Updates #37: HTTP library switching done, Revenue Plans

    Our 37th updates newsletter went out today.

    Highlights from this week:

  119. The Switch to Guzzle is Complete

    I’ve spent the past 4 weeks (or so) switching from our own HTTP client library to a third-party HTTP client library called Guzzle.

  120. Simply Testable Updates #36: HTTP library switching progress

    Our 36th updates newsletter went out today.

    Highlights from this week:

  121. Simply Testable Updates #35: HTTP Response Caching, Robustness++/Performance++, More Switching Http Libraries

    Our 35th updates newsletter went out today.

    Highlights from this week:

  122. Simply Testable Updates #34: Estimated Time Remaining Available, Switching HTTP client libraries, HTML Validator Updated, More Worker Testing

    Our 34th updates newsletter went out today.

    Highlights from this week:

  123. Estimated Time Remaining Now Available

    Providing feedback during a long-running operation is a very good idea.

    When viewing a full-site test as it progresses you are shown the percentage of the test that is complete.

    What bothered me what that you would have no idea how long it would take for the test to finish. Staring intently at the various progress indicators that you are given can give you an impression of how long the test will take to finish but that’s hardly a good use of your time.

  124. Simply Testable Updates #33: More Robustness++, More Bug Fixes

    Our 33rd updates newsletter went out today.

    Highlights from this week:

  125. Ignoring False CSS Background Image Data Url Errors

    I learned yesterday from @johnholtripley that CSS background image data URLs were presenting CSS validation errors incorrectly.

    By this I mean that the CSS validator was saying there is an error when there was not.

    Let me quickly explain what data URLs in CSS are for this to make sense.

  126. Simply Testable Updates 32: Bugfixes, Robustness++

    Our 32nd updates newsletter went out today.

    Highlights from this week:

  127. Batch of Content-Retrieval Bugs Fixed

    Today has been focused on fixing bugs that just so happen to all relate to content retrieval.

    With the volumes of web pages retrieved and tested daily by Simply Testable, I always think I’ve seen all the ways in which people produce HTML incorrectly.

    With the volumes of code I write daily for Simply Testable I always think I’ve covered (nearly) all the ways in which content retrieval can go wrong.

    I’m still amazed by what I find.

  128. Simply Testable Updates 31: Staying Signed In, Backups In Place

    Our 31st updates newsletter went out today.

    Highlights from this week:

  129. Choosing Data To Exclude From An Application Backup

    Overview

    I’m putting in place a data recovery plan to ensure we’re never again left with nothing should we ever again lose both hard drives in our production server at the same time.

    I can’t back up everything. It takes too long and the resulting backup is too large. I’m writing an article about that to be posted soon if you’d like to know more. For now, it’s enough to know that backing up everything is not feasible.

    Choosing what data to include in, or exclude from, an application backup is not easy.

    Here’s how to make it easy.

  130. Staying Signed In

    Account creation was launched back on 9 November 2012.

    As a feature, and in line with releasing early and often, it was released with the bare minimum to work.

    What the sign in form lacked at that stage was an option to let you stay logged in.

  131. Simply Testable Updates 30: Recovery Strategy, Backups Mostly There

    Our 30th updates newsletter went out today.

    Highlights from this week:

  132. Data Recovery Strategy

    Overview

    In the middle of January we experienced major downtime and data loss due to the failure of both hard drives in our production server.

    Although this was about as bad as it gets, I’m glad it happened now whilst we’re still somewhere between the alpha and beta stages and before we introduced any paid plans.

    At the time, the backup plan was … non-existent. There was no such thing. My thinking at the time was that the risk of losing all our data before I had a chance to put a backup plan in place was minimal and that there’d be plenty of time to work on a recover strategy once the service was generating revenue.

    Clearly my thinking could not have been more wrong.

  133. Improved Viewing of Results For Large Tests

    We’ve had a problem viewing the results of large tests for a long time.

    If you started a new full-site test on a site with thousands of URLs, closed your browser, went home, returned the next day and opened up the results page for your test, you’d very likely not get anything useful.

    Most of the time you’d get a rather unglamourous 503 Service Unavailable error page resulting from an uncaught application error.

    If you were lucky, you’d get a more graceful application error page and the details of the error that happened would be emailed to us.

  134. Simply Testable Updates 29: Large Full Site Tests Working

    Our 29th updates newsletter went out today.

    Highlights from this week:

  135. Simply Testable Updates 28: Still Moving Towards a Backupable Restorable System

    Our 28th updates newsletter went out today.

    Highlights from this week:

  136. Simply Testable Updates 27: Backupable Progress - Restorable Data, Reduced Backup Size

    Our 27th updates newsletter went out today.

    Highlights from this week:

  137. Simply Testable Updates 26: Back From The Dead, Single URL Tests, Backup Strategy

    Our 26th updates newsletter went out today.

    Highlights from this week:

  138. Service Restored

    The recent emergency maintenance is now all complete.

  139. Simply Testable Updates 25: Massive Service Failure, JSLint Improvements on the Way

    Our 25th updates newsletter went out today.

    Highlights from this week:

  140. Down For Emergency Maintenance

    Update 9 January 16 22:00 GMT

  141. 3 Non-Code Production Performance Improvements For Developers

    Last night I gave a talk titled “3 Non-Code Production Improvements For Developers” at the January 2013 Unified Diff meetup.

    A big thanks to @HandyBiteSize, @gavD_UK and @rodnaph for continuing to organise the events, as well as @craigmarvelley for helping me get my laptop connected to the projector.

    Slides and my initial preparatory notes are available at https://github.com/webignition/udiff-jan-2013.

  142. Simply Testable Updates 24: Talk @UnifiedDiff Today, CSS Validator Improvements, JS Static Analysis Improvements

    Our 24th updates newsletter went out today.

    Highlights from this week:

  143. JS Static Analysis Against 404ing JS URLs No Longer Failing

    I noticed yesterday that some full-site JavaScript static analysis tests were failing quite ungracefully.

    Such tests would fail without completing correctly, leaving the test appearing to be stuck in progress and never completing. This was not good.

  144. CSS Validation For Non-Relevant Content Now Skipped

    If you have non-HTML URLs in your sitemap, such as URLs for PDFs, HTML tests for such URLs will be marked as ‘skipped’ to indicate that the test was not run due to it not being relevant.

  145. Creating and Using a Github OAuth Token With Travis And Composer

    Introduction

  146. Simply Testable Updates 23: Missing/Invalid Doctype Support Live, CSS validator Improvements, Unified Diff Talk

    Our 23rd updates newsletter went out today.

    Highlights from this week:

  147. Internationalised Domain Name (IDN) Support Improved

    Thanks to the feedback from @igll, internationalised domain name (IDN) support has been improved.

  148. Simply Testable Updates 22: Improved IDN support, Missing/Invalid Doctype Support

    Our 22nd updates newsletter went out today.

    Highlights from this week:

  149. Simply Testable Updates 21: JavaScript Linting, CSS Validation Options

    Our 21st updates newsletter went out today.

    Highlights from this week:

  150. CSS Validation Options Explained

    New options for validating CSS

  151. Improving CSS Validation: New CSS Validation Option Choices

    Discovering and deciding on CSS validation options

  152. Improving CSS Validation: The Need for Control Over The CSS Validator

    The need for CSS validation options (controlling the validator)

    I’ve recently been adding some options to CSS validation tests to make the tests more useful. Or, depending on your perspective, to make CSS validation useful where it previously was not.

    Across a short series of blog posts, I’d like to look now at why CSS validation options are needed, the range of options I considered, where I got the options from and how I decided on which of these to implement.

  153. Simply Testable Updates 20: CSS Test Option Refinement

    Our 20th updates newsletter went out today.

    Highlights from this week:

  154. Simply Testable Updates 19: JavaScript Static Analysis, Test Option Refinement

    Our 19th updates newsletter went out today.

    Highlights from this week:

  155. Installing JSLint For Command Line Use On Ubuntu

    Preamble

    In adding JavaScript static analysis in addition to HTML and CSS validation to the set of full-site tests you can run, I need to run JSLint from the command line.

    Since I need to do this on a number of Ubuntu servers, and since this may be of use to others running Ubuntu or Debian systems, here’s how to do it.

  156. November 20th Downtime Post-Mortem

    It broke for 6 days.

  157. Simply Testable Updates November 2012 #3: Mega Downtime, Test Type Choices, Improvements You Won't Notice

    Our 18th updates newsletter went out today.

    Highlights from this week:

  158. Choose HTML Validation, CSS Validation or Both

    Since the introduction of CSS validation two weeks ago, any new full-site test ran both HTML and CSS validation for all URLs.

  159. Simply Testable Updates November 2012 #2: CSS Validation Is Live

    Our 17th updates newsletter went out today.

    Highlights from this week:

  160. Account Creation Live

    Account creation is now live! You can now sign up for your free account.

  161. Simply Testable Updates November 2012 #1: Evil People Steal My Bike, Account Creation Mostly Done

    Our 16th updates newsletter went out today.

    Highlights from this week:

  162. Simply Testable Updates October 2012 #5: Hello Capacity, Testing Local Sites, Short-Term Feature Plans

    Our 15th updates newsletter went out today.

    Highlights from this week:

  163. How To Test A Local Site

    I’ve been asked a few times if it’s possible to test a local website. You can. Here’s how.

  164. Simply Testable Updates October 2012 #4: Database Performance++, Account Creation Underway

    Our fourteenth updates newsletter went out today.

    Highlights from this week:

  165. W3C HTML Validator On Ubuntu 12.10

    I recently upgraded all local development and CI systems from Ubuntu 12.04 to Ubuntu 12.10.

    This killed the local installations of the W3C HTML validator. If the same happened to you, here’s how to fix it.

  166. Simply Testable Updates October 2012 #3: Large Interest, Post Launch Problems, Performance Comeback

    Our thirteenth updates newsletter went out today.

    Highlights from this week:

  167. Switching From Apache To Nginx on Ubuntu Server

    We’re changing the web server Simply Testable uses from Apache to Nginx.

  168. Post-Launch Post-Mortem

    We launched publicly last Wednesday.

    Launching didn’t mean making the service available - it had been live on the web at simpytestable.com for two weeks and before that at alpha1.simplytestable.com for about two additional weeks.

  169. Capacity Problems All Over The Place

    Friday saw us mentioned by WebResources depot to their 35,000 RSS subscribers. Capacity problems followed. We coped, but only just.

  170. Limited Task Assignment For Better Parallel Test Performance

    A problem of which I’ve been aware for a while and which was the root of significant problems for yesterday’s MVP release relates to how test tasks are assigned out to workers.

    A process runs every few seconds and picks a small number of tasks from each incomplete test and marks them as being ready to be assigned out to workers.

    This process was introduced to ensure that many tests can run in parallel. Prior to the introduction of this process, the tasks for a new test would be assigned out to workers at all in one go. Subsequent tests would be delayed in starting until the workers had process the tasks of all previous requests.

  171. Simply Testable Updates October 2012 #2: Public Launch Day!

    Our twelfth updates newsletter went out today.

    Highlights from this week:

  172. Multiple Sitemaps, Indexed Sitemaps Supported

    The URLs to be tested when running a full-site test are gathered from a website’s sitemap.

    A sitemap can list a collection of URLs for your site. A sitemap can also act as an index listing a collection of sitemaps. Your robots.txt file can also list many sitemaps (despite this not being part of the standard).

    Up until just a few minute ago, we supported only the first type: a sitemap that listed a collection of URLs for your site.

    I rewrote our sitemap finder, sitemap retriever and sitemap model libraries to support multiple sitemaps and sitemap indexes so that all sitemap uses are supported.

    If your robots.txt lists a sitemap index URL, we’ll now read the index and read in all the referenced sitemaps and extract all URLs across all sitemaps for testing.

    If your robots.txt lists multiple sitemaps and are not using a sitemap index, we’ll just grab all those sitemaps and extract all the URLs across all sitemaps for testing.

  173. Showing Us Some Love

    People have nice things to say about Simply Testable!

    I added today to the footer of simplytestable.com some of the love that’s been received so far from @jenko and Matcha Labs.

    Have you tried Simply Testable and found it useful? Do you think it’s a great idea and something you’d find valuable?

    Let us know: send a tweet to https://twitter.com/simplytestable or send an email to jon@simplytestable.com, including, in addition to whatever you’d like to say, the link you’d like placed below your quote.

    I can’t try and buy your love and I definitely won’t consider adding you to the list of those getting free access to the subscription pro service coming in the next few months. That absolutely will not happen.

  174. Gracefully Handling Test Failure Without Blocking Test Progress

    A Significant Problem Resolved

    Sitting at the bottom of the list of improvements on the homepage at simplytestable.com since a little before the private alpha release is the item:

    Gracefully handle test failure without blocking test progress

    A little vague perhaps, but certainly a worthwhile improvement tackling a very significant problem.

    It’s the last big problem that was preventing a smooth running service. Now that it’s resolved, I’ll look a little at what the problem was and how the solution was put together.

  175. Test Timeouts Now Supported

    Timeouts. Sometimes they just happen.

    Up until very recently, a timeout when trying to perform a HTML validation test for a URL would leave the test stuck as being in progress. This prevented the full site-wide test from ever showing as 100% complete, often getting stuck at 90-something percent.

    Timeouts join a collection of three new test failure situations supported:

    • Retrieval timeouts: a timeout when retrieving the HTML of a page
    • DNS lookup timeouts: a timeout when trying to resolve the domain of a URL
    • Malformed URLs: sometimes a URL can’t work as it is just not quite made correctly

    In any of the above three cases a test will be marked as failed and when viewing the results of a test you’ll get a brief description of why the test could not happen.

  176. Simply Testable Updates October 2012 #1: MVP In One Week, Timeouts Handled, Better Sitemap Handling, Error Pages

    Our eleventh updates newsletter went out today.

    Highlights from this week:

  177. Our Technology Stack

    I’ve been asked a few times, most recently by the guys at Matcha Labs, what tech simplytestable.com uses and how it all works together.

  178. Simply Testable Updates September 2012 #4: Public Alpha, More Performance++

    Our tenth updates newsletter went out today.

    Highlights from this week:

  179. Compressed Sitemaps Now Supported

    The URLs against which to run a full-site test are ideally collected from a website’s sitemap.xml. This is usually a plain XML file containing a list of URLs and some metadata regarding those URLs.

  180. Redirect Loops and Long Redirect Chains Detected

    Redirect loops are no fun. You visit a URL and it 301 redirects you to another URL which 301 redirects you to another URL which then eventually redirects you back to where you started at which point the fun begins again.

    Redirect loops are no fun because you never get any content for such a URL, you just get redirected indefinitely.

    Long redirect chains are no fun. You visit a URL and it 301 redirects you to another URL which 301 redirects you to another URL which 301 redirects you to another URL … and so on.

    You don’t get redirected back to a URL you have already visited whilst riding this redirectercoaster so it’s not a redirect loop.

    You might eventually get redirected to a URL that doesn’t redirect you any longer. Or you might not. One cannot in advance tell.

  181. URLs Now Collected From News Feeds

    A full-site test that aims to carry out a set of tests against every URL for a site needs to get the URLs from somewhere.

  182. Simply Testable Updates September 2012 #3: Performance++, RSS Feed Support, Test Failure Support

    Our ninth updates newsletter went out today.

    Highlights from this week:

  183. Production Bug #002 Spotted - Redirect Loops

    Today I spotted a second case where an HTML validation test doesn’t complete properly - redirect loops.

  184. Test Failures Now Supported

    Up until yesterday we had no way to spot a failed test. I’m referring not to a failure such as an HTML validation test finding errors but a test that fails to run at all.

  185. <meta name="Content-Type" ... > is invalid

    In investigating production bug #001 I noticed some web pages were trying to specify the character encoding used but were doing so incorrectly.

  186. Production Bug #001 Fixed

    Following yesterday’s private alpha release, I was trying to figure out why some HTML validation tests were failing.

  187. Simply Testable Updates September 2012 #2: Private Alpha Released, Things to Fix, Things to Improve

    Our eighth updates newsletter went out today.

    Highlights from this week:

  188. Private alpha release available

    The private alpha release is now available: http://alpha1.simplytestable.com/.

    You can run full-site HTML validation tests on any site that has a sitemap.xml file; it’s from the sitemap that the URLs to be tested are retrieved.

  189. Simply Testable Updates September 2012 #1: Epic-scale Performance Improvements, Private Alpha One Week Away

    Our seventh updates newsletter went out today.

    Highlights from this week:

  190. Working Prototype In Action #004 (Video)

    An update to the third prototype demonstration; the results page is displayed once a test completes and the details of the errors encountered can be examined.

  191. Simply Testable Updates August 2012 #5: Working Prototype Progress

    Our sixth updates newsletter went out today.

    Highlights from this week:

  192. Working Prototype In Action #003 (Video)

    An update to the second prototype demonstration; an indication of the number of errors encountered in a test is displayed as the test progresses.

  193. Working Prototype In Action #002 (Video)

    An update to the first prototype demonstration; the list of URLs being tested against is dynamically updated as they are discovered and tests against them are performed.

  194. Working Prototype In Action (Video)

    A quick demonstration of the latest working prototype running a site-wide HTML validation test.

  195. Simply Testable Updates August 2012 #4: Internal Prototype Working

    Our fifth updates newsletter went out today.

    Highlights from this week:

  196. First Test Results

    I reached a significant milestone today: the first real test was run.

    I completed the integration of the W3C HTML validator (with HTML5 support) into the worker HTML task driver, an overly technical way of saying that HTML validation tests are now performing actual HTML validation and reporting the results back to the core application.

  197. Simply Testable Updates August 2012 #3: Progress, Integration Automation, First Real Tests

    Our fourth updates newsletter went out today.

    Highlights from this week:

  198. Installing The W3C HTML Validator With HTML5 Support (On Ubuntu)

    We use the W3C HTML validator for driving HTML validation tests.

    We don’t use the W3C public service at http://validator.w3.org/; the usage limits imposed (quite fairly) by the W3C are too restrictive for our needs, and the latency involved in sending HTML out to the Internet to be validated increases by orders of magnitude the time it takes to complete an HTML validation test.

    We install a local copy of the W3C HTML validator and a local copy of the Validator.ru HTML5 validato.

    If you want to frequently validate many HTML documents quickly, here’s how you can do the same.

  199. Simply Testable Updates August 2012 #2: Progress, HTTP Testability, Roadmap

    Our third updates newsletter went out today.

    Highlights from this week:

  200. Integrating Resque Into A Symfony Application

    We use a job system to handle the queuing and running of tasks asychronously with respect to the application flow. Actions that could get in the way of things running smoothly and which don’t mind when they happen are put into a queue to be run later.

    This covers tasks such as the core application verifying a worker’s activation request, expanding a new test job into a set of test tasks or farming out test tasks to eager hungry workers.

  201. Opensource Libraries We've Created and How We Use Them (Part Two Point Two: HTTP Testability)

    I created some libraries for dealing with certain matters that would be common across the Simply Testable service.

    Such libraries address generic matters that should be useful to others. They have been open-sourced under an MIT license from day one.

    In /opensource-libraries-weve-created-and-how-we-use-them-part-one-url-handling/, I talked about what we use for handling URLs. In part two point one of this series, I talked about how we deal with many HTTP requests reliably.

  202. HTTP Failures And What To Test For In HTTP Applications

    In part two point two in the series of opensource libraries we’ve created I’ll talk about how we test HTTP-based applications.

    In preparation for that, let’s look at what can go wrong such that your HTTP conversation fails and what you need to test for to ensure that your HTTP application copes with such failure.

    This goes beyond HTTP-specified 4xx and 5xx errors and examines failures that result in no HTTP response being received.

  203. Simply Testable Updates August 2012 #1: Progress, Opensource HTTP Client Library, Unit Test Improvements

    Our second updates newsletter went out today.

    Highlights from this week:

  204. Opensource Libraries We've Created and How We Use Them (Part Two Point One: HTTP Reliability)

    I created some libraries for dealing with certain matters that would be common across the Simply Testable service.

    Such libraries address generic matters that should be useful to others. They have been opensource under an MIT license from day one.

    In part one of this series, I talked about what we use for handling URLs.

    In part two point one, I look at what we’ve created and use for handling HTTP requests reliably. Part two point two will expand on this and look at how we test HTTP-based applications.

  205. Simply Testable Updates July 2012: Architecture Overview, Opensource Libraries, Task Sequences and Progress

    Our first updates newsletter went out today.

  206. Tasks and How They Combine Into Series and Sequences

    Tasks are where all the fun is. Tasks (ultimately) give you the insight into what is right or wrong about your site. Tasks give us a fun place to play around with testing against specific quality factors.

  207. Opensource Libraries We've Created and How We Use Them (Part One: URL Handling)

    I created some libraries for dealing with certain matters that would be common across the Simply Testable service.

  208. Architecture Overview

    A client such as the web interface, an iOS app or an Android app, sends a site test job request to the core application. The client reports back to the user on the status and progress of the job and presents the results of the test once complete.

  209. Allowing Slashes in the URL Path With Apache

    Ever found yourself needing to allow slashes in a URL path? http://app.simplytestable.com/job/https://simplytestable.com/start

  210. High Level Plan

    Simply Testable is launching publicly October 2012, about 12 weeks from now. It will allow users to perform a range of quality-related technical and non-technical tests across a given web site.

  211. The Beginning

    This is a story of how I came to be sitting here writing about developing a brand new professional automated web front-end testing service.