Things in Jars

Simon Madine (thingsinjars)

@thingsinjarsFollow me on Twitter

Hi, I’m Simon Madine and I make digital toys and write guides on web development.

I'm a senior web dev and evangelist for HERE in Berlin.

  • Twitter
  • Flickr
  • GitHub
  • Forrst
  • Shelvist
  • Last.FM
  • RSS
  • How App.net became useful

    Written 2 Oct 2012

    Opinion,Geek,Ideas

    After Twitter started announcing changes to its API usage and people started to get worried about the future of the developer eco-system, App.net appeared. It provides a streaming data-platform in much the same way Amazon provides a web hosting platform. The reason for the Twitter comparison is that one of the things you can make with it is a Twitter-like public short message system. This was, in fact, the first thing the App.net developers made to showcase the platform: Alpha although that example seems to have convinced many people that the whole point was to build a Twitter-clone instead of a service whose purpose is to stream small, discrete blocks of meta-tagged data. The community-developed API spec is an important aspect of App.net as well although feature discussions can devolve into musings on Computer Science theory a bit too easily.

    For the first couple of weeks, it was fun to just hack around with the APIs, post a bit, build a little test app (disabled now that more App.net clients have push notifications). It all became much more interesting, however, when two new features were added to the platform – Machine-only posts and Well-known Annotations.

    Machine-only posts

    Pretty much exactly what they sound like. These are posts that are not intended to be viewed in any human-read conversation stream. They still get pushed around the network exactly the same as all other posts but to see them, you must explicitly say 'include machine-only posts' in your API calls. For developers who have been building silly toys on Twitter for a couple of years, this is fantastic. You don't need to create a separate account purely for data transfer. This can be part of your main account's data stream. I have quite a few Twitter accounts that I just use for outputs from various applications. I have one, in fact, that does nothing but list the tracks I listen to that I created for this side-project.

    By classifying these as 'machine-only', they can stay associated with the user and subject to whatever privacy controls they have set in general. This makes it far easier for the user to keep control of their data and easier for the service to track accurate usage. For devs, it also means you can hack away at stuff, even if it is to be public eventually, without worrying too much about polluting the global stream with nonsense.

    Well-known Annotations

    Annotations were part of the spec from the beginning but only implemented at the end of August. Annotation is just another word for per-post meta-data – each post has a small object attached which provides extra information. That's it.

    The annotations can be up to 8192 bytes of valid JSON which is quite a lot of meta-data. Even with this, however, the usefulness is limited to per application use-cases until some standards start to appear. There's nothing to stop a popular application being the one to set the standard but for the more general cases, it is most fitting to continue with the community-led development model. This is where Well-known Annotations come in.

    Well-known annotations are those attributes which are defined within the API spec. This means that the community can define a standard for 'geolocation', for example, and everyone who wants to use geolocation can use the standard to make their application's posts compatible with everybody else's.

    Obviously, I'm quite into my geolocated data. I love a bit of map-based visualisation, I do. Here's a sample of jQuery that will create a post with a standard geolocation annotation:

    $.ajax({
      contentType: 'application/json',
      data: JSON.stringify({
        "annotations": [{
          "type": "net.app.core.geolocation",
          "value": {
            "latitude": 52.5,
            "longitude": 13.3,
            "altitude": 0,
            "horizontal_accuracy": 100,
            "vertical_accuracy": 100
          }
        }],
        machine_only: true
      }),
      dataType: 'json',
      success: function(data) {
        console.log("Non-text message posted");
      },
      error: function() {
        console.log("Non-text message failed");
      },
      processData: false,
      type: 'POST',
      url: 'https://alpha-api.app.net/stream/0/posts?access_token=USERS_OAUTH_TOKEN'
    });
    

    In the same way as the machine-only posts, these annotations aren't provided on a default API request, you have to specifically ask for them to be included in the returned data. This is to make sure that the majority of use cases (public streaming, human-readable conversation) don't have to download up to 8KB of unnecessary data.

    Retrieving both

    This is an API call to retrieve posts marked machine-only and with annotations

    Potential use cases

    You might have noticed, the API call above had the machine-only attribute as well as the well-known geo annotation. If I wanted to create an app that would run on my phone and track my routes throughout the day, all I would need to do would be to run that $.ajax call periodically with my current geolocation. The data would be saved, distributed, streamed and could be rendered onto a map or into a KML at the end of the day. I could record hiking trails or museum tours, or share my location with someone I'm supposed to be meeting so they can find out where I'm coming from and why I'm late. That's just a couple of the single-user cases. Having a shared standard means that the potential for geo-tagged posts opens up to at least equal that of Twitter's. Heatmap-density diagrams showing areas of the most activity; global and local trends; location-based gaming. Add a 'news' annotation to geotagged posts and suddenly you've got a real-time local-news feed. Add 'traffic' and you've got community-created traffic reports.

    There are so many clever things you can do with location-tagged data. I hope others are just as enthused about the possibilities as I am.

    Comments

  • CSS Verification Testing

    Written 28 Dec 2011

    Opinion,Geek,CSS

    It's difficult to test CSS. In Nokia Maps, we use lots of different kinds of automated tests to ensure the build is doing what it's supposed to do – Jasmine for JS unit tests, jBehave for Java acceptance tests, a whole load of integration tests and systems to make sure all along the process that we know as soon a something goes wrong.

    CSS has typically been left out of this process – not just in our team but generally throughout the industry – because it's a difficult thing to test. CSS is a declarative language meaning that the CSS of a page doesn't describe how the page is to accomplish the task of turning text red, for example, it simply says 'the text should be red' and the implementation details are left up to the browser. This doesn't fit well into typical testing strategies as there is no sensible way to pass information in to check the outputs are as expected. With a declarative language, there is very little logic around which you can wrap a test. In essence, the bit you would normally test is the bit handled by the browser. About the closest CSS gets to having continuous-deployment style tests is to run your files against the in-house style guide using CSSLint.

    That's not to say people haven't tried testing CSS. There are lots of opinions on how or why it should be done but with no concrete answers. There has also been a lot of thinking and quite a lot of work done in the past to try and solve this requirement. This article from 2008 and this article from 2006 both propose a potential route to investigate for testing strategies.

    In summary, the two main approaches used are:

    • Generate images from the rendered HTML and compare differences (c.f. css-test by Gareth Rushgrove)
    • Specify expected values and compare actual values (cssUnit, CSSUnit)

    There must be something about the desire to test CSS and the name Simon as both Simon Perdrisat and Simon Kenyon Shepard created (separate) unit-testing frameworks called 'CSSUnit'. And there's me, obviously.

    As a related aside, there may also be something to do with working at Nokia and wanting to test CSS as Simon Shepard is the Tech Lead on Nokia Maps although I stumbled across his project completely randomly outside work. Weird.

    Another important related note: there's no point in developing a CSS testing framework for Test-Driven Development. Again, this is an aspect of being a declarative language but, by the time you've written your test, you've written your code. There's no Red-Green pattern here. It either does what you intended it to or it doesn't.

    In essence, the only way to test CSS is by verification testing – the kind of thing you do before and after refactoring your styles. This, essentially, involves working within your normal process until the job is done then creating 'snapshots' of the DOM and capturing the current cascaded styles. You can then refactor, rework and reorganise your CSS as much as you like and, as long as the combination of snapshot DOM plus CSS produces the same results as before, you can be confident that your entire system still appears the same way.

    Get to the point...

    Why the long ramble about CSS testing strategies? Well, unsurprisingly, I've had a go at it myself. My approach falls into the second category mentioned above – measure styles once finished and create secure test-beds of DOM structure which can have CSS applied to them. The test system is currently being run through its paces in Nokia Maps City Pages (my bit of maps) and, if it passes muster, I'll put it out into the big, wide world.

    Comments

  • Don’t be seduced by the drag-and-drop side

    Written 29 Aug 2011

    Geek,Opinion,Guides

    You don’t have to be a survivor of the vi and Emacs holy wars to appreciate the beauty of fully hand-crafted code. There was a bit of attention a couple of weeks ago on the soon-to-be-launched Adobe Muse which lets you “Design and publish HTML websites without writing code”. If you want to be a kick-ass developer, you must realise that tools like this aren't designed for you. They're designed for people who want to do what you can do but either don't have the time or the inclination to learn how. Although drag 'n' drop application do lower the barrier to entry for creating a website, there is still a need for web developers to know exactly what's going on in their pages.

    I'm not saying that there will never be a time when visual design can be automatically translated into as good a product as a hand-crafted site, I'm just saying it’s not yet.

    In much the same way as with JavaScript (See “You must be able to read before you get a library card”), building your HTML using that extra level of abstraction might work for almost every situation but will eventually leave you stuck somewhere you don’t want to be. By all means, pimp up your text editor with all manner of handy tools, shortcuts and code snippets but make sure you still know exactly what each bit of mark up means and does. If you structure your code well (more on that in a later post), your natural feel for the code will be as good a validator as anything automated (by which I mean prone to errors and worth double-checking).

    Learn the keyboard shortcuts. If you learn nothing else from this, learn the importance of keyboard shortcuts. You might start off thinking you'll never need a key combination to simply swap two characters around but one day, you'll find yourself in the middle of a functino reaching for ctrl-T.

    Also, there is no easy way to tell if a text editor is fit for you until you have tried it, looking at screenshots won’t work. You don't need to build an entire project to figure out whether or not you're going to get on with a new text editor, just put together a couple of web pages, maybe write a jQuery plugin. Do the key combinations stick in your head or are you constantly looking up the same ones again and again? Do you have to contort your hand into an unnatural and uncomfortable claw in order to duplicate a line?

    The final thing to cover about text editors is that it's okay to pay for them. Sure, we all love free software. “Free as in pears and free as in peaches” (or whatever that motto is) but there are times when a good, well-written piece of software will cost money. And that's okay. You're a web developer. You are going to be spending the vast proportion of your day using this piece of software. If the people that made it would like you to spend $20 on it, don't instantly balk at the idea. Think back to the idea of web developers as artisan craftsmen. You're going to be using this chisel every day to carve out patterns in stone. Every now and then, you might need to buy your own chisel.

    This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

    Comments

  • You must be able to read before you get a library card

    Written 4 Aug 2011

    Geek,Opinion,Javascript,Guides

    I like JavaScript. JS. ECMAScript. Ol' Jay Scrizzle as nobody calls it.

    I also like jQuery. jQ. jQuizzie. jamiroQuery. Whatever.

    Ignoring the stoopid nicknames I just made up, did you notice how I referred to JavaScript and jQuery separately? I didn't say "I like JavaScript, there are some really great lightbox plugins for it" just the same as I didn't say "I wish there was a way to do indexOf in jQuery".

    I'm regularly amazed at how many new (and some not-so-new) web developers either think they know JavaScript because they know jQuery or wish there was a way to do something in jQuery that they read about in an article about JavaScript. jQuery is a library written to make coding in JavaScript easier. It's made in JavaScript so you can say "jQuery is JavaScript" but only in the same way that "Simon is male". To confuse jQuery as all of JavaScript is the same as saying "Simon is all men" (don't worry, there's still only one of me).

    For most web site or web app development, I do recommend using a library. Personally, I've used jQuery and Prototype extensively and decided I prefer jQuery. Libraries are designed to make coding faster and more intuitive and they can be a great productivity aid. You can get a lot more done quicker. There is a downside, however.

    Downside

    If you're doing what the library was intended to help with, great. Slide this panel up, pop open a modal window, scroll to the bottom of the page and highlight that header. Brilliant. The difficulties come when you're either trying to do something the library wasn't intended to do or something nobody's thought of before or you're just plain doing something wrong. If you are fluent in your library of choice but don't know the JavaScript underpinnings, your usual debugging tools can only help you so far. There will come a point where there's an impenetrable black-box where data goes in and something unexpected comes out. Okay, it's probably still data but it's unexpected data.

    Don't let this point in the process be the discouragement. This is where the fun bit is.

    Learning to read

    Library authors are very clever groups of people. Often large groups. Reading through the unminified source of a library can be an awesomely educational experience as it's usually the culmination of many years best practice. If you want a nice introduction to some of the cool things in jQuery, for instance, check out these videos from Paul Irish:

    • http://paulirish.com/2010/10-things-i-learned-from-the-jquery-source/
    • http://paulirish.com/2011/11-more-things-i-learned-from-the-jquery-source/

    I've dug around in jQuery many, many times to try and figure out why something does or doesn't do what it should or shouldn't. The most detailed investigation was probably Investigating IE's innerHTML during which nothing was solved but I found out some cool stuff.

    Learning to write

    The best way to get your head around libraries is to write your own. Yes, there are literally millions of them (not literally) out there already but you don't need to aim for world dominance, that's not the point of writing your own. Start simply, map the dollar symbol to document.getElementById. Done. You've written a tiny library.

    function $(id){ 
        return document.getElementById(id);
    }

    Now you can add some more stuff. Maybe you could check to see if the thing passed to the $ is already an element or if it's a string. That way, you could be a bit more carefree about how you pass things around.

    function $(id){ 
      if(id.nodeType) {
        return id;
      } else {
        return document.getElementById(id);
      }
    }

    Add in a couple of AJAX methods, some array manipulation and before you know it, you've got a full-blown web development toolkit.

    If you're wanting a boilerplate to start your library off, I recommend Adam Sontag's Boilerplate Boilerplate.

    Here's your Library Card

    By now, you've rooted around in the jQuery undergrowth, dug through some of Moo's AJAX and pulled apart Prototype's string manipulation. You've written your own mini library, gotten a bit frustrated and wished you had a community of several thousand contributors to make it more robust. Now you're ready to start getting irked every time someone on Forrst asks if there's a jQuery plugin for charAt. Enjoy.

    This article is modified from a chapter in a book Andrew and I were writing a couple of years ago about web development practical advice. Seeing as we both got too busy to finish it, I'm publishing bits here and there. If you'd like to see these in book form, let me know.

    Comments

  • older posts

Browse articles by category:

Toys, Guides, Opinion, Geek, Non-geek, Development, Design, CSS, JS, Open-source Ideas, Cartoons, Photos

Recent side-projects

  • Open Source Snacks
  • csste.st
  • jHERE Playground
  • CoverMap.me
  • Soundscape
  • Insta-Art

Toys

Find my digital toys at thingsinjars.com/lab.

Contact

Send me a message via Twitter.

Adopt-a-Museum

@thingsinjars:

    © 2013 Simon Madine