I'm going to try not to scream. Because I spent over an hour figuring out how to force my web browser to remember my username (not my password, just my username) for a website that I need to log into regularly.
Poor design and inaccessible design are everywhere. And they are extremely prevalent when it comes to computing, application design, and web design. I know that. I have been fighting it for years. Like, more than five years.
Generally, I find that poor design and inaccessible design come from organizations that just don't care or individual programmers and designers who simply don't know better. But it would seem that everyone gets a pass when it comes to anything vaguely resembling security. And I'm fed up with it. It's not okay. It is not ethical for you to bar me from your system because you designed security which is not accessible to me. Period.
And before someone comments to tell me that I don't understand computer security: I do. I've taken graduate level number theory courses, and I understand encryption. I've done RSA encryption by hand. I've read many articles from organizations such as the Electronic Frontiers Foundation which emphasize privacy and security in the digital world. I use free software extensively. I know how to program. I'm the kind of geek who reads the source code of webpages. I'm not an expert, but I'm significantly more knowledgeable than 99% of the population.
To be clear, I'm not advocating for lack of security. What I am advocating for are two things:
1. Allow users to choose which security risks are worth taking with respect to their own data.
2. Design security systems which are flexible enough to be accessible to people with a variety of abilities.
That's it. Neither of those things sound unreasonable to me. But yet I see extremely frustrating violations of those two principles all the time.
Today it was a website that decided it wasn't okay for me to have my web browser remember my username. First of all, it's just my username! And the website also lets me log in with my email address, which is basically just public knowledge. I mean, seriously. Not hard to find my email address. Second, it's my account and if someone else was able to access it, they would be accessing only my private data. In other words, that risk only affects me, so it should be my decision to make. Third, my web browser is my software. (And I use Mozilla Firefox, so it is free and open source.) It runs on my computer. It should take instructions from me first and foremost. My computer, my software, my choices. Period.
Here's another example: Ctrl-Alt-Del. That keystroke. I hate it. It's a historical relic which was originally a troubleshooting tool used by IBM technicians. It should have never made it into the mainstream. And yet Microsoft uses it as a required part of logging in to a Windows PC, in the name of security. (The actual security provided by that is questionable.I am not going to go there right now. A web search will point you in the right direction if you're interested.) To clarify, I can turn that off on my own computer. And I have. But, my university uses it and requires that I do the same. I can manage it, but it's extra difficulty and extra pain that's really not necessary. And there are plenty of people who can't. (For example, upper limb amputees, people with significant fine motor or coordination difficulties, people without fingers, people who can't physically get their hands to the keyboard.)
Shall we have another? Windows User Account Control. Words that strike rage into my heart. In addition to having some pretty significant technical design issues, Windows User Account Control has the absolutely charming feature of not accepting any software generated input. The claim is that this can prevent a virus or other malicious code from accepting the prompt for the user. That's at least partially true. But it also prevents a huge number of assistive technology users from being able to interact with the system. Personally, I've turned it off on my own computer. Which is fine for me. But that design decision effectively prevents anyone who relies on assistive technology (including me) from ever working in the computer technology field (using Windows, at least). Those people can't be Windows system administrators, they can't be tech-support consultants, they can't do certain types of development work. (I have a hunch that's actually a violation of the ADA, but Microsoft can hire an awful lot of lawyers....)
Looking at these and many other examples, it seems to me that we give developers a free pass on accessibility when it comes to security. Perhaps it's because our culture struggles so much to evaluate real risks (as opposed to perceived risks). Perhaps it's because we really love the way security theater makes us feel safer. Perhaps it's just because the people who tend to do disability advocacy work don't understand security and don't know how to make a solid argument when these topics come up. But whatever the cause, it's not okay.
I've said it before, and I'll say it again: Security is not an excuse for inaccessibility. End of story.
Nope, not kidding. I have actually been writing computer programs with my eyes (and a few voice commands thrown in here and there).
More specifically, I'm using an eye tracking device called the Tobii EyeX. It's a consumer-level eye tracker which mounts to the bottom of my computer monitor and connects to my computer via USB 3.0. (I've written about this type of eye tracking device before, and you can also look at Tobii's website.)
Okay, so how?...
Good question! And I'm going to answer it with the rest of this post. I'm going to write this in the general style of a how-to guide, in case any of my readers actually want to copy my set up. (To my knowledge, no one else has documented how to do this.) However, even if you have no desire to write Python with your eyes, you might still want to skim the rest of this. Several of the tools that I'm using have potentially much broader applications.
Hardware requirements:
-Tobii EyeX sensor bar (link, < $150)
-external monitor, preferably at least 20 inches but no more than 24 inches (You probably could make this work with a laptop, but having a larger screen area makes the eye tracker much easier to use. 24 inches is the size limit for the EyeX sensor.)
-Modern computer with USB 3.0 support (I have a quad core processor at 3.6 GHz, 8 GB of RAM. I wouldn't recommend much less than that, unless you want to eliminate the speech commands from the project and use keystrokes instead.)
Software requirements:
-Windows 7 64-bit (should work on Windows 8 or 10, as well. But you may have to make some adjustments.)
-EyeX drivers and associated software (included with the device)
-Dasher (free and open source)
-FreePIE (free and open source)
-Compiler and editor for the language you wish to program in (for this example, I'll be using Sage and its browser-based interface)
You will also need some sample code from the language you want to program in (so that Dasher can build an appropriate language model). You can ignore this if you only want to write in English (or another natural language).
Got all that? Good, let's get started.
1. Install the EyeX drivers and mount the sensor according to the manufacturer's instructions. Run the configuration to get EyeX to work properly with your monitor, and calibrate it to recognize your eyes. All of this is standard for any user of the EyeX, and you should be able to follow instructions from the manufacturer. When you're finished, you can test your calibration using the "test calibration" option in the EyeX software.
2. Download and install Dasher. We will make some configuration changes to Dasher in the next step, but for now just take a minute to familiarize yourself with Dasher if you haven't used it before. With the default settings, you can click the mouse inside of Dasher's window to start writing, and drag the mouse towards the letters you want in order to write. Your text will appear in the box at the top. For more details, see Dasher's website. (Obviously, skip this experimentation/exploration step if you aren't physically able to use the mouse. You can experiment with Dasher once we get it working with the eye tracker. I just included this step so that, as much as possible, new users can get used to one new technology at a time.)
3. (Skip this step if you just want to write in English, not computer code.) Close Dasher, and locate its program files on your hard drive. On Windows 7, I have a Dasher folder in Program Files (x86). Inside of that is another folder for my specific version, in this case Dasher 4.11. In your Dasher folder, find the folder labeled system.rc. We are going to add two new plaintext files to this folder. First, an alphabet file. This defines the all of the valid symbols for your language. I would recommend starting with an existing file, so make a copy of alphabet.englishC.xml and put it in the same folder. Name your copy alphabet.<yourlanguage>.xml, and open the file in a plain text editor. Make the following changes :
-Change line 5 so that the text in double quotes is the name of your language (this will be shown in that the user interface).
-Change line 9 by replacing 'training_english_GB.txt' with the filename for your training text. (I used 'training_sage.txt')
-Remove the section labeled "Combining accents" (lines 13 through 27)
-(optional and language-dependent) I additionally removed line 127 (the tab character) because my editor does auto-formatting with tab characters, and unexpected results were generated if Dasher inserted a tab character and then attempted to delete that character with the backspace key (which it regularly does if you take a "shortcut" to the character you want through another box). I decided that it was simpler to leave the autoformatting on in my editor (I really didn't want to manually insert six or seven tab characters before a line of code, as can happen in Python!) and insert my tabs manually with voice commands. You will have to decide how to handle these sorts of formatting and autoformatting issues yourself, based on the language and editor that you choose.
It is also at this point that you can reorder or regroup any of the characters if you want, in a way that makes sense for the language you are writing.
Save your changes, and now you have an alphabet file. Now, let's specify the training text. Create a plain text file titled training_<yourlanguage>.txt (or whatever you called it in the alphabet file). Paste all of your sample code into that file. (More is better! But Dasher will also learn from what you write over time.) Save your training text file in the system.rc folder, just like the alphabet file.
Now, open Dasher and make sure that you can access your new language. Click on the Prefs button, and look in the alphabet selection box. You should be able to locate the language that you just specified. Click on it, click okay, and try writing in your programming language. If everything is working, proceed to the next step. Otherwise, go back and check that you got all of the syntax and file names/locations correct. (A few troubleshooting details: If it works, but Dasher warns you that you have no training text, then your alphabet file is correct but you need to check the training text file (or file specified for training in the alphabet file). If it works, but the predictions seem bad, then you probably need more training text. If it works, but Dasher takes too long to load, try deleting some of your training text.)
4. Okay, now let's configure the Dasher user interface. Click on the Prefs button. Under alphabet selection, make sure your desired language is selected. On the control tab, select 'Eyetracker Mode' for 'Control Style'. Then, select 'Mouse Input' for the 'Input Device'. You can select the speed and choose whether or not you want Dasher to automatically adjust its speed. (I like this feature, but it's up to you.) Finally, choose the method(s) for starting and stopping. Personally, I like the "Start with mouse position – Centre circle" option for with the eye tracker. You might also want to enable the mouse button control and/or spacebar, especially until you get comfortable with the eye tracker (obviously only useful if you have a way to push the spacebar or click the mouse button). I would not recommend using the "stop outside of canvas" option, because it makes the motion feel extremely jerky when the eye tracker briefly leaves the active area. All of that is personal preference, though. There's no harm in experimenting to see what you like best.
5. More Dasher configuration. In the preferences window, click on the Application tab. Under "Application style:" select "Direct entry". This means that Dasher will automatically send keystrokes to the currently active window. This lets you write directly into your code editor of choice.
Let's test again. With Dasher open, open Microsoft Notepad. Click in Notepad so that your cursor is in the text entry field. Then go to Dasher and (without clicking on anything besides the canvas – you don't want to change focus) start writing with the mouse. Your text should appear in Notepad. Once that's working, give it a try with your desired code editor.
6. Download and install FreePIE.
7. Let’s actually use this eyetracker! At the end of this post, I have pasted the code for a simple mouse control script. Take this script and paste it into a new script in FreePIE. Click on script menu and then run script (or press F5). (Tumblr doesn’t support any sort of code environment, so, if you have trouble getting the script to work, let me know and I can email you the actual file.) Then press the Z key to start the eye-controlled mouse movements. If everything is working correctly the mouse cursor should now follow your gaze around the screen. I've also set up this script so that you can start or stop the eye mouse using speech. If you have a decent microphone (preferably headset), make sure that that device is selected as your default recording device in Windows. Then, with the script running, you can say "start eye mouse" or "stop eye mouse" to start or stop the mouse movement. (If the computer has a hard time recognizing your voice, you can improve this by doing some of the training in Windows speech recognition. You can find it in the control panel.)
8. Now all that's left to do is chain all the pieces together. Get FreePIE open with your mouse script running. Make sure that Dasher is running, and also open your code editor. Make sure that the focus is in the text entry field of the code editor, press the or use speech to start the eye mouse, and finally start Dasher (by whatever you selected in Dasher's configuration). You should now be able to enter code with only your eyes.
9. (Optional) Depending on your needs, you may want to add some additional commands to the FreePIE script. For example, I have written some code that lets me enter tab characters by voice, and also navigate with in text by voice (e.g. "go to end of line", "go to beginning of line", "go up line", "go down line", etc.). Each of these is just a couple of lines in FreePIE. Feel free to reach out to me if you need some help with adding these sorts of additional commands.
That's a lot of text, so let's finish off with a screenshot. Here's what the whole thing looks like when it's up and running. On the left-hand side of the screen you see Dasher. On the right-hand side is Firefox showing Sage. FreePIE is running but minimized.
[Image description: the image shows a screenshot of a Windows 7 desktop with two visible applications. On the left-hand side of the screen is Dasher. The majority of that window is filled with a collection of different colored squares and typable characters. There is a cross in the center of the window, and a red line shows where the user is currently pointing. On the right-hand side of the screen is a Firefox web browser window. The active tab is labeled Palindromes, and most of the window is occupied by text fields showing Sage code and output.]
The script for FreePIE:
#Use Z or speech to toggle on/off
import ctypes
import math
def update():
global prevX
global prevY
#Settings
smoothingConstant = 0.2 #between 0 and 1; 0 means no cursor movement, 1 means use raw data
rawxcord = tobiiEyeX.gazePointInPixelsX
rawycord = tobiiEyeX.gazePointInPixelsY
diagnostics.watch(rawxcord)
diagnostics.watch(rawycord)
xcord = prevX + (rawxcord - prevX) * smoothingConstant
ycord = prevY + (rawycord - prevY) * smoothingConstant
if enabled:
ctypes.windll.user32.SetCursorPos(int(xcord),int(ycord))
global enabled
#enabled = False
prevX = xcord
prevY = ycord
if starting:
global enabled
global prevX
global prevY
enabled = False
prevX = 0
prevY = 0
tobiiEyeX.update += update
toggle = keyboard.getPressed(Key.Z)
start = speech.said("start eye mouse")
stop = speech.said("stop eye mouse")
if toggle:
global enabled
enabled = not enabled
if start:
global enabled
enabled = True
if stop:
global enabled
enabled = False
Within the past year, multiple companies have released eyetrackers costing hundreds, instead of (tens of) thousands, of dollars. These new devices are aimed at gaming and multimedia applications, but they are still amazingly useful as assistive technology. The two main competitors at this point seem to be Eye Tribe and Tobii EyeX.
I purchased an EyeX unit this March, and it has significantly expanded what I can do with my computer. I will write later about the specifics of my setup, but I want to today just survey some of the development happening on the EyeX and other low-cost eye tracking devices (much software works across multiple devices) which is relevant to disability. (There is also significant gaming development on the platform, but it is largely using multiple input devices and would not be accessible to many physically disabled people. Plus,Tobii will tell you all about that.)
To be clear, I’m saying that these devices have great accessibility potential, not that they are currently ready made computer access solutions. At present, using these devices as an accessibility tool requires at least moderate technical skills.
Okay, without further ado, here are some software projects I find interesting:
OptiKey is an on-screen keyboard and mouse emulator specifically designed for eye tracking input. Free and open source. Windows only.
Project Iris uses eye tracking to control the mouse. It also allows you to define active screen regions and triggers actions when you look at them. Many potential uses for accessible gaming. Paid. Windows only.
Gaze Speaker is a self contained environment for speech generation by eye movements. Also includes other tools like an email client, a simple web browser, and a file browser, all designed to be accessed by eye alone. Free and open source.
FreePIE lets you write Python scripts that take input from many, many different devices, including the EyeX. You can use that input to trigger just about any action you can imagine, provided you can figure out how to write the code. Probably best for geeks, but extremely powerful for meeting custom needs. Free and open source.
I have been browsing the Internet in search of accessible mouse solutions (for my own use), and, as a result, I have been looking at able of websites that sell assistive technology. And I have seen something that I think is, quite frankly, a predatory practice: companies selling software that does very little from a programming perspective (and could easily be replaced with free software) for hundreds (sometimes thousands) of dollars. And they're selling these products to disabled people who have often been severely disadvantaged in getting a technology education because of their disabilities. So, I present, a list of software functions that you should never pay for. Ever.
(To be clear, not all assistive software is predatory. Some of it is very useful software that does complex tasks using innovative algorithms and large datasets. I want to help you learn to tell the difference.)
You should never buy software to perform the following tasks:
1) hover clickers or dwell clickers (software that automatically clicks the mouse when you hold the cursor still for a set length of time)
2) software to alter your mouse speed, acceleration, or sensitivity
3) software to remap inputs (for example, to switch the right and left buttons on your mouse, or to type in a phrase whenever you press a certain key)
4) on-screen keyboards (unless you have very specific needs, such as gaze interaction or interaction with another specialized input device. Or may be if the keyboard is particularly innovative and useful.)
5) software to "filter" your keyboard input or ignore repeated keypresses
6) macro writing software (such as software that lets you "record" the sequence of actions and then play them back with a single keypress)
7) break timers (software that reminds you to take a break from using the computer)
8) software to use a gaming device, such as a joystick or videogame controller, as a mouse
Please, please, please don't buy these products! Use free software (or features built into your operating system) to do these tasks, and spend your money on something that will actually help you!
If you need help figuring out how to use free tools to do any of these tasks, feel free to send me a message. As a quick reference, is here is how I would recommend approaching the tasks that I have listed above.
1) I use an AutoHotkey script. There are also several free dwell clickers available for download; a Google search should point you in the right direction. (Linux users: both the GNOME and KDE have a hover click tool available under the standard accessibility options.)
2) Start with the controls in your operating system. If that's not enough, try the drivers provided by the manufacturer of your mouse/trackball/joystick/etc. or a tool like AutoHotkey.
3) AutoHotkey!
4) Most modern operating systems have one included. There are also lots of free options online.
5) This is a standard accessibility option in modern operating systems.
6) There are lots of free macro packages available. I like AutoHotkey on Windows and simple shell scripts on Linux.
7) Workrave (hey, look, I have a whole post about it!)
8) Again, AutoHotkey!
I should probably write a post about AutoHotkey, hmmm?
Start/stop GIF animations through a keyboard shortcut or by clicking them. You can also restart animations from the beginning, or disable animations by default.
Gifs sometimes make me dizzy and/or give me headaches. This add-on for Firefox gives you the option to pause all gifs by default and only play them if you want to. Simple, easy install and works beautifully.
Okay, computer security has me all worked into a tizzy. Because it is so, so readily apparent that no one gives a shit about accessibility. I'm just so frustrated, because it is so apparent that even imagined security concerns that have no basis in fact trump the needs of disabled people. And that's not okay.
It seems to be okay, not just here, but on a broader societal level to justify inaccessibility in the name of security. I recently spoke to a woman who actually worked for an assistive technology agency. She has muscular dystrophy, and she can't do the control-alt-delete key stroke to log into Windows, either. And yet her organization is unwilling (or unable) to make an exception so that she can use her computer. (She has her husband do it, which is fine for her, but not by any means a general solution.) It's ridiculous! And here's the crazy thing about that control-alt-delete key stroke: essentially nobody in computer security agrees that it's beneficial or necessary. (Some people claim it should always have been done with a single button dedicated to security; others say that no button can be effective in today's world of complex operating systems and proliferating input devices.) There are many instances of people noting that the keystroke is an accessibility issue. (For one thing, it basically requires two hands. For another, it can't be activated from on-screen keyboards or other emulation devices. It's also more cognitively demanding and physically challenging than other alternatives.) And yet we still have it. In spite of years of hardware and software design, that horribly inaccessible key stroke is still with us. Still haunting thousands of disabled people every day. And my University, despite being populated with intelligent people who should be able to understand this, still insists that I must press those three magic keys.
It just baffles me. This is such an easy problem to understand, and such an easy problem to solve. (Here is a perfectly generic and nearly universally accessible solution: Place an extra button on the computer, which serves as the equivalent of control-alt-delete. Additionally, place an input port next to the button which can be used to attach any sort of a switch that the user desires and have such a switch perform the same function. Now any person can use whatever device they need to to activate that keystroke. Easy. Alternate solution: just ditch the whole thing. Many security experts say it doesn't matter anyway.)
Now, in the past few years, we are starting to see some solutions, in the form of a single "security button". And you know why that happened? No, it's not because hardware and software designers read Section 508. No, it's not because they decided to actually listen to disabled people. It's because of tablets. Tablets! Which, shockingly enough, don't have physical keyboards. Now that all users must use an on-screen keyboard, it seems that Microsoft et al. give a shit. I am so pissed.
Dictation is a free online speech recognition software powered by Google Chrome. You can use Dictation to write emails and documents in the browser with your voice
This is a cool website which allows anyone with a microphone and internet access to use dictation software developed by Google. It’s not perfect, but the recognition accuracy is pretty impressive. I was seriously surprised at how well it worked!
Unfortunately, there are no tools for correcting mistakes or editing your writing.
It only works with the Google Chrome browser (or the chromium browser).
I’ll be sticking with Dragon for now, but this may come in handy when I’m on the go and don’t have access to my desktop computer.
