Every time I do a user test with a beginning programmer, I remember how hard computers are, how unforgiving the tools are, and end up wanting to apologize for how annoying and strict programming is. We are making progress with teaching people how to code, but it's still really hard.

For example, if you are just getting started with Python, here's a short list of problems you might face when trying to set up Flask, which is by far the easiest Python web server to set up.

• Learning how to cd in the terminal
• How URL's requested by a user map to actual code
• HTML, CSS and Javascript, because you actually want it to be pretty.
• How to read and write things from a database
• Installing Flask, so learning how to use pip or easy_install
• Python telling you your file is no good because it mixes tabs and spaces.
• How to run Flask locally

And that's not even counting the stuff that's so obvious to us we forget to mention it. Most quickstart guides also fail to help people make incremental progress.

Game designers are great at teaching new, hard things. They have to be, or no one will play their games. You will notice that games don't start with you battling Ganon in an epic death match; they start with you learning how to use the character and perform actions like make a kick, or open a door. Through a series of incremental successes you become an expert in the game and can tackle more and more complex tasks.

It bugs me to see so many Python tutorials mention virtualenv as a requirement to get started. (virtualenv is a tool for sandboxing your Python apps, so each Python project on your computer is using its own set of packages). The biggest advantage of virtualenv is that you can have different versions of the same Python package (like Flask or requests) that are required by different projects, whereas if you install them system-wide you can't.

However, recommending virtualenv just adds another thing you have to do before you can see pretty lights on the screen, and represents another possible opportunity for people to lose interest, and start doing something else instead of learning how to get a web server set up.

It also introduces a significant opportunity for confusion; the "It was working yesterday, why isn't it working now?" problem. You need to remember to source your virtualenv file in every Terminal shell where you're running Python, or your terminal will tell you it can't find the library you literally just installed. Needless to say this is confusing, the Terminal won't tell you how to solve the problem, and Googling for the answer isn't likely to give you the solution you need, because it's such a generic error message.

I've never seen beginners run into the problem of needing conflicting versions of a Python package for two different projects. I was comfortable dumping everything into site-packages for over two years of Python development; only when I started working at Twilio did I need to start installing virtualenvs for every project.

As a community, I believe we should stop recommending that beginners install virtualenv. The faster we can get beginners to a Holy Shit, I Wrote Code That Made Something Happen moment, the better, and virtualenv is a big block for getting to that point. Instead I'd recommend installing pip using the one line curl program in the second paragraph here. virtualenv is something that's more appropriate to learn about and use once you have a few Python projects under your belt.

I love Tweetdeck. Well, I love parts of Tweetdeck. Specifically the side-by-side view makes it really easy to see my stream, my replies, people linking to my website and people talking about Twilio on one screen and I haven't found another tool that can do it. But Tweetdeck also breaks my browser in really annoying ways. Here are the most annoying examples.

Use of outline:none on buttons.

When you tab through a form, Tweetdeck doesn't show you which button is currently focused on the screen. I tend to stay on the keyboard wherever I can because switching to the mouse is so slow. It's much faster to fill out forms tabbing with the keyboard than clicking around with the mouse.

Operating systems have highlighters that show you which button is currently focused - on Mac OS X they show a blue glow around the box that's currently focused, like this:

Clearly you can see the US Dollar field is focused. Now if I tab again, the "Measurement Units" button is focused:

Tweetdeck doesn't like having outlines on their buttons. This is what you get if you tab from the "Password" field:

See how the focus disappears:

It's not that annoying there. Where it's really annoying is when you are trying to post a new tweet. In GMail when I press Tab from the message body, the focus jumps straight to the "Send" button:

This is perfect because it makes sending emails really fast - I just hit Tab, Enter and the email's sent and I'm back in my inbox.

When you press Tab + Enter from the "Tweet" field in Tweetdeck, instead of posting your Tweet, they jump you to the Camera button:

Normally this is annoying, but it's not that annoying; I can just Tab multiple times to get to the button. But Tweetdeck also hides the focus with outline:none in their CSS, so you can't actually tell which button is currently focused. This means you can't figure out how many times you need to Tab through to get to the Tweet button.

It's really not that hard to fix, and it won't make a lick of difference to the mouse-clicking hoi polloi. Just change the HTML tabindex of the form, and add a focused style for the button.

Using target=_blank for links.

When I read my stream, I read through all the Tweets first, opening up interesting links in new tabs to read later. I don't want to constantly be jumping back and forth between my stream and articles, because I lose my place in the stream.

Fortunately, browsers offer two methods for opening new tabs in the background. One is to hold Cmd while clicking on a link. The second is to right click and press "Open in new tab."

Neither of these is good enough for Tweetdeck; they force you to immediately switch out of your stream into new articles by adding a target="_blank" attribute to all links in the app. target="_blank" links override your browser's default behavior and switch your focus immediately to the new tab or window.

The reason they attach target=_blank attributes to their URL's is so that all of their links open in a different tab, instead of the same and Tweetdeck stays open in the browser. But that's also evil, because it breaks a user's expectation about what's going to happen when they click on the link. I want to open links in a different tab, I just don't want them to be focused, and Tweetdeck makes this impossible.

Completely breaking when Javascript is disabled.

Lately I've been playing with disabling Javascript in my browser. Mostly I am disappointed in how much of the Internet breaks when you disable Javascript, especially pages that only show content like news articles or blog posts.

It's nice at least when a site includes a <noscript> tag telling you that you need to enable Javascript for the site to function. Tweetdeck doesn't even do this. Here is what you get if you browse to web.tweetdeck.com with Javascript disabled:

That's a blank screen, with no notice or indication that your browser didn't crap out.

They also detect your user agent, and error out if you try to access Tweetdeck with Firefox or Opera, without even making an attempt to display the page. I can understand not supporting IE, but I don't understand why they can't even try Firefox.

Conclusion

Tweetdeck is good enough that I'll continue to use it, but there are some simple things that make my experience a little miserable every time I click on a link or try to write a new Tweet. I wish Twitter as a whole cared more about accessibility and usability for all of its users (here's another example); with an estimated 100 million active users, even a small percentage of users who lack the fine motor control to use a mouse (or even a keyboard) is still a large number of people that are affected by problems like this.

Friday one of my favorite teams, St. Mary's, dropped an extremely close game to Purdue. They were up one point with 31 seconds to go when one of their senior players tried to run the baseline, something you can only do after you have scored a basket. Purdue took the ball and went on to win the game.

Obviously it was a bad mistake for the player to make in that situation. But there's another person who is to blame: the referee. Before inbounding the ball, the referee will signal to the player that they either can or cannot run the baseline. Sure, the referee can blow the whistle every time there is a violation, but it's better to prevent the error in the first place. It would be like letting players line up for a free throw in the wrong order, letting the player shoot and then blowing the whistle for incorrect order.

Ultimately the blame belongs with the player who made the mistake; he should have known better. But it was an awful way to decide the game; on a technicality instead of through the actions on the court.

It's a known law of websites that any type of mistake that can be made by your users (entering a username instead of an email address, entering a wrong phone number, etc) will be. For those cases we write error handling code and prevent incorrect data from being written to our database. But it's better to prevent the error as quickly as you can - as soon as they make the mistake, if it's possible. The more time that elapses between the error occurring and the time you tell them about it, the more frustrated they are likely to be. Even better is to design your form in a way that prevents them from making the mistake in the first place.

Pretty much every site needs to validate form data on the server to make sure it's correct. But the best sites will also validate data on the client side, e.g. when the user is typing it into their computer. This way they can prevent users from making dumb mistakes.

Programming is hard. Especially when you are just getting started, there are a lot of things, in Donald Rumsfeld's words, that you "don't know that you don't know." Lots of quickstarts for beginners assume the reader knows things about how the command line works that my experience shows they don't. I do lots of user tests with people new-to-programming and see these errors again and again.

I thought I'd put together a short list of things quickstart writers leave out, that will still leave you totally stumped.

• Most of the time if you see an indented block of text in a fixed width font, like this:

  $ foo baz bang
  

  It usually means you're supposed to do something with the text in the box.

• If the text in the box has a dollar sign at the beginning of it, it represents a command you are supposed to enter in your Terminal. If you get an error that looks like this:

      bash: $: command not found
  

  It means you aren't supposed to copy the dollar sign. Just type foo baz bang and hit <Enter>. Unfortunately it's hard to Google for dollar signs, so you can't really figure out what you did wrong.

• If there's a dollar sign, followed by some more text on the lines below the dollar sign, the lines below represent the output of running the command correctly. For example:

      $ python run.py
       * Running on http://127.0.0.1:5000/
       * Restarting with reloader
  

  You are only supposed to type in python run.py into your Terminal. The rest of that stuff is the output of running the command correctly.

• If the block of text doesn't have a dollar sign, it's probably a snippet of code you are supposed to copy into a text editor and save into a file (On Mac, use TextWrangler; on Windows use Notepad++). Hunt around the quickstart for a filename you should use.

• If they don't tell you where to save the files, create a new folder and save all of the files in there, in the top level.

• If the command line mentions a file you've recently created, like this:

      $ python run.py
  

  You need to run the command from "inside" the same folder as the file. The terminal has a notion of being "in" a directory (Directories and folders are the same thing). Here's a short guide:

  • To figure out which directory you are in, type pwd and press Enter. (pwd stands for "print working directory").

  • To list all files and folders in the current directory, type ls -al and press Enter.

  • To go into a folder below the current one, type cd Documents (or whichever folder you are trying to navigate to). To do more than one type cd Documents/code.

  • To go up a level type cd ..

  Once you're "in" the right place you should be able to run the command properly.

• If you feel like you are spinning your wheels, ask for help! It's important to know how to ask. Make sure to tell people a) what you are trying to do, b) what you expected to happen, and c) what actually happened. Bonus points if you can talk about things you tried previously and why they failed to do what you wanted.

As an author of a quickstart myself, I feel like I owe an apology to users who are just getting started, for not including this information along with our guide. Sadly the terminal is just about the least beginner-friendly piece of software I can think of, and when you are just getting started, so-called "simple" errors can totally derail you and make you want to go outside and play Frisbee or roller blade.

Hopefully these tips will help you get started doing that cool tutorial you've always wanted to do.

Last summer I tore into CMC's website redesign, saying that the new design emphasized looks over function and did a poor job of explaining what made CMC special. I recently visited the site and they've made a bunch of usability improvements.

Here are some of the highlights. Again, I offer these up with the caveat that, I haven't done any testing or looked at any data, but I do have a lot of experience in this area.

• The old homepage, with only 14 links, call to action button that looked like an ad, and incredibly-difficult-to-change photo content, is gone. Instead users are redirected straight to http://cmc.edu/discovercmc, which is a much better page.

• The Discover CMC page has lots of dynamic content that promises to be much easier to update; photos of speakers, links to events, a Twitter widget and a sliding bar. It also has an updated meta description, so a Google search for "Claremont McKenna" returns more contextual information about CMC.

• Static assets (images, CSS, Javascript) are being cached with a Last-Modified and an E-Tag header, so that browsers will not re-request the same images and CSS every time a user requests the homepage. This will help with page load times.

• There's a call to action button on the homepage: "Plan your Visit to Claremont McKenna today."

• The "Student Gateway" replaced all of the stock photos with links to useful content, like the login form for your email account, which used to take around four clicks. That is outstanding. (For the record, http://bit.ly/cmcmail will take you right to the old form - I set up that link junior year :)) It looks like a page I would actually use to find things I was looking for - the maintenance request page, the Collins menu, etc.

• The calendar on the Student Gateway page uses Google Calendar, instead of the old ASPX event calendar that no one used.

• The "Prospective Students" page has an explanation of why you should apply to CMC.

• Skip links for disabled users!! These will help people skip to the main page content and help CMC, a nonprofit, meet government standards for website accessibility.

• The professor home pages, which I singled out for SEO improvements, have gotten about halfway there; it's clear someone is thinking about improvements in that area. Pages now contain an h1 tag and some keywords describing what the professor does; it's a lot of work, but the pages would be best with a unique <title> attribute and a meta description, however.

These add up to amazing improvements in usability and discoverability; they've addressed most of the problems with the old site. It also represents a tremendous amount of effort on the website and whoever is responsible should be proud.

That said, it's still not perfect, and there are some more quick usability/SEO wins to accomplish. Here's a shorter, less urgent, list of areas they could still improve upon:

• You can access the Discover CMC page from eight distinct URL's:

  • www.cmc.edu/discovercmc
  • cmc.edu/discovercmc
  • claremontmckenna.edu/discovercmc
  • www.claremontmckenna.edu/discovercmc
  • www.cmc.edu/discovercmc/index.php
  • cmc.edu/discovercmc/index.php
  • claremontmckenna.edu/discovercmc/index.php
  • www.claremontmckenna.edu/discovercmc/index.php

  Google will sometimes interpret duplicate content as a sign that you're trying to farm for content by placing the same text at different URL's. It also means Google is unsure which version of the page to point people to. It's better for Google rankings to redirect all duplicate content to one canonical domain/URL with a 301 (and better still to serve it at the root - shorter URL's rank more highly).

• It feels odd to have distinct pages for Admission and for Prospective Students; those two have a ton of overlap and it might be best to merge them. Prospective students are the only group interested in Admissions information, and the Admission pages may get more love (see the outdated Twitter feed on the Prospective students page).

• The dropdown menu is great and includes a ton of links to useful content. However, I expected that when I hovered over the menu item, the menu would appear automatically. Instead I had to click to make the menu appear. Normally I expect when I click on something that looks like a link, I will be taken to a different page, so I was hesitant to click on the link.

  Dropdown menus have a usability problem where users scrolling the mouse from above the menu to below the menu trigger the flyout, even though they don't mean to. The best practice here is to have the menu only appear after the user has hovered over the item for around half a second, so that transient mousers don't trigger the flyout.

One of the reasons I set up Good Morning CMC was to provide students with an actually useful calendar and a more accessible view of the information that we needed on a daily basis. With these changes Good Morning CMC is becoming close to redundant.

The number of CMC students interested in different tech fields - web design, marketing, entrepreneurship - has been on the rise recently. I wouldn't be surprised if the web team and some smart interns couldn't continue to improve the site, boosting application rates, prospective student contact rates, and alumni giving rates, through iterative improvements to the current site.

PS Sorry I didn't include images or links - I am trying to blog more often and cut down on the amount of time it takes to do so.

I lived for three months in India on a wi-fi network that would go on for two minutes, then off for three, and would randomly stop working for days at a time. So I got really, really good at getting work done on a terrible Internet connection. Here are some best practices.

1. Avoid HTTP wherever possible. Your goal is to get the information you need and nothing more. Loading a full website, with ten Javascript files each requiring their own connection, ten CSS files, and fifteen images, will take forever. Meanwhile, the page render is blocked until the CSS finished downloading, so you can't even see the damn thing.

   Instead of loading websites in your browser, use different protocols. So instead of checking your mail at gmail.com (over HTTP), check it in Mail.app (over SMTP) and save yourself a ton of bandwidth. Instead of loading a chat client in the browser, load a chat client (over XMPP) and save bandwidth. Instead of loading your favorite website in the browser, save the RSS feed and read that instead (one HTTP request instead of thirty).

   I also set up tons of stuff to work directly via email - I could post to Twitter from email, update my Facebook from email. If I could help it I tried to push every interaction I had toward email or RSS.

2. Cut down on HTTP requests, if you can't help it. Currently Firefox is the best bad-connection browser. Open up the settings and disable Javascript and images. Download the FlashBlock and AdBlock plugins. These will save you 40-50% of the number of HTTP connections you have to make, and lots of the throughput - images often use the most bandwidth and most of them add little. Furthermore any ad that streams video will kill your speed and crush your download time.

   Boost the size of your cache as high as it can go - I would say half of available RAM - so that images and other assets will be loaded from your machine, instead of downloaded from the Web. Make sure to kill tabs when you are done with them, as well - many modern AJAXy applications will continue making requests to the server even when they are not being used.

3. Use a mobile user agent. Most websites will send less data to mobile phones than to regular browsers. On Firefox you can download the User Agent Switcher add-on and pretend like you are an iPhone. You should also use the mobile versions of popular sites like Facebook (m.facebook.com) and Twitter (mobile.twitter.com)

4. Do most of your work offline. If you are developing websites, make sure you can replicate your production environment on your local machine, so you can experiment without making HTTP connections. Use a fast, compressible protocol like Git or rsync (instead of FTP) to compress the amount of data you send over the wire to your server.

   Use a good desktop mail client that can queue up outbound messages for you, then do all of your email offline. When you get back online, sending and retrieving messages over SMTP should only take a minute or two.

5. Kill background data hogs. Lots of desktop programs will automatically connect to their home bases like you have a great connection. Some you should kill: Dropbox, the Adobe AIR Updater, Microsoft Office Updater, Software Update, any programs that load on startup, the Google Chrome auto-updater, as well as any program that has an "Automatically check for updates on startup" box.

   You should also kill any programs that are set up to load on startup in the "Login Items" preference pane. You can get a list of open HTTP connections by running netstat -a in the Terminal, although note that a lot of connections to popular sites may not be to the web domain you know (eg Google serves some content from 1e100.net and other domains). Also check your /Library/LaunchAgents and /Library/LaunchDaemons, as well as your ~/Library/LaunchAgents/ folder for apps that start running themselves in the background on system startup.

Conclusion

So that's about it; I can also go on about the terrible performance of websites on degraded browsers, or how most mobile apps break unnecessarily when they don't have a data connection (looking at you, Words with Friends). A bad internet connection is really frustrating and some of these tips will hopefully help you consume the Internet like you're used to.

I have a new blog post up on the Twilio Engineering Blog about best practices for your company's documentation. The three main points:

1. Users are busy/lazy - you can't expect that they'll actually read anything, and you can't expect them to have any context, so make sure your code samples contain everything necessary to run the code.

2. Users are coming from Google - so you need to SEO all of your documentation content.

3. You are busy - so write the documentation first, when you're still excited about it.

The entire post is here - please take a look!

On a related note, I feel like I've been writing as much as I do usually, but sadly not all of it is visible to the public.

I entered my credit card the right way on the first try this morning. I think this new diagram had a lot to do with it:

I don’t understand why the white diagram works so well. From the perspective of the diagram, you are looking up at the credit card slot, and the keypad had a completely different perspective. But it does work. I guess this is why testing is so important.

The most usable credit card slot holder would be one that could accept your credit card in any orientation, or a credit card with four magnetic strips. I don’t think that’ll happen any time soon.

Last weekend I redesigned my website. The new site has a homepage featuring a bunch of different things I’ve been working on, and a blog post view with no header, just a sidebar and the post itself. As a result the site is way more usable.

With any redesign you have to start by looking at what your audience needs and then tailor the site for those needs. My user base divides into three groups:

• Friends and family, who will read more or less everything I write
• Drive by Google traffic, who are likely to visit one page and then leave
• Potential clients/employers/partners, who were referred from Hacker News, Twitter, an email I sent out or a job application.

I am guaranteed to get traffic from the first two groups, as long as I don’t break the site. The third group of clients, employers and business partners are people who do not yet know me, but may interact with me soon, and the site plays a big role in helping them make that decision. So the redesign is targeted toward that group.

To win that third group of users, my site has to answer one question: Who is this person and why should I listen to him? My old site didn’t do a very good job of answering that question. Here’s what it looked like:

The standard blog template, where posts are displayed in reverse chronological order, is good for only one use case – where the writer updates frequently and the readers want to see what’s changed since the last time they visited. The majority of blogs on the Internet don’t fit this model because the authors update infrequently and don’t have a consistent audience. Therefore, they should consider a different navigation scheme for their sites; the main problems are outlined in this Jakob Nielsen article on blog usability. My old site used a reverse chronological order for navigation, and here are some of the problems with that design:

• No way to find good content that may be a year or more older.
• Posts are on a variety of different topics, appealing to different groups of users, with no organization.
• No photo, or description of me, on any page.
• All of the action items (send email, read more about me, etc) require at least one full page refresh; this is too difficult, as most users are lazy and won’t hunt around.
• Articles are posted in full text, meaning that user attention span might run out quickly (and also that 10 posts worth of images are requested and downloaded)
• Finally, the large number of different pages made it more difficult to maintain, and each attempt to tackle a part would leave me overwhelmed thinking about fixing the whole thing.

So I had the following goals in mind for the redesign:

• Tell a better story about me on every page.
• Put the best content on my site in front of users.
• Make the action items (email, Twitter, etc) easier to click.
• Also, improve page load time, which would boost user satisfaction and my Google rankings. Google found that a half second delay in page load time dropped traffic by 20%.

So here’s what I came up with for the homepage:

And the single blog post view:

I’m pretty pleased with the way they turned out. The homepage essentially consolidates the information that was spread across five pages in the menu of the old design. Furthermore, it displays ten of my best posts, organized by category, and a sample of things I’ve tweeted, so there will be at least some dynamic content on the page for users who visit.

I also took some steps to improve the page load speed. I installed two caching plugins – WP Minify, which minifies and combines all of your Javascript and CSS files into one file of each type, and WP Super Cache, which generates a static HTML file and then serves that file to users, instead of hitting the database every time a user requests a page. These help minimize the response time, the number of requests, and the size of each request.

I also use a sprite to display the icons on the blog page view, turning five requests into just one. As a result, the blog post view gets a score of 83/100 and the home page gets a score of 78/100, about a 20 point improvement over the old design.

So that’s the analysis of my site redesign. Now if I could just to find a posting schedule and stick to it!

I just spent two hours finding and fixing all of the broken links on my site. The Broken Link Checker plugin is invaluable for this task (as well as extremely usable!) Here's what I learned:

• Don't hotlink images. Hotlinking is where you embed an image into your page that is hosted on someone else's page. Not only does it cost the other person bandwidth to serve the request, but it is extremely vulnerable to breaking whenever the other person decides to change their site, or (worse) change the image hosted at that URL, so where you thought you were showing an image of a cat, you are actually showing a porn image meant to embarrass you. The majority of broken links were hotlinks.

• The alternative to hotlinking is downloading the image, uploading it to your own site and then displaying it from there. This is pretty tedious in Wordpress; there should be a plugin that lets you do this.

• Think very, very hard before deleting or moving resources on your site. Once you create a resource, you should serve the content at that URL forever, or use a 301 redirect to indicate the new location of the resource. If you don't, you are breaking the Internet. A related problem is that it's very easy to break links: change your page title, move subdomains, or move a small section of content, and you've caused a problem for your site's users.

• If a link is broken on your site, you've created a problem for every single person that links to that resource, because now they are pointing their visitors to a broken link. The best way to handle this is with a 301 redirect. However many webmasters don't know they broke links, or don't care, so I had to update many links by hand to point to the new resource.

  It would be nice if there were some kind of link wiki service that you could update to point to the correct resource; then everyone could just point their links to the wiki and the Internet would not be broken when someone updates the location of their content.

• I had a much better chance of finding what I was looking for when the URL described the post content than when the URL was opaque (e.g. http://example.com?p=519). It's also good SEO to have the title of your posts displayed in the post URL's, so you should definitely try to do this for your posts.

• As a webmaster, you should try to make your 404 (File Not Found) page helpful for users. Try parsing the URL and outputting the results of a site search onto the page, in case you can find the content under a different name. At the very least, you should set up your CMS to send an email whenever your site serves a 404. That's an easy way to find broken links on your site that are hurting users.

All in all, there were 2300 total links on my site, and about 230 of them were broken, for about 10%. This means that, roughly speaking, a user has a 1 in 10 chance of being disappointed when they click on a link on my site; I cut that down to about 20 broken links by unlinking content that 404's or updating it to point to the correct resource. This was a good use of my time; it's important to keep a good user experience for all of my users, even the ones that will only visit once.