Introducing the Smile Project
Todd M. Gureckis · · 3 minutes to readToday we are sharing a new open-source project for developing behavioral experiments online. It's called Smile.
Smile began in 2022 when I was on sabbatical. I noticed that PhD students in my lab were spending a substantial portion of their time developing our behavioral tasks. One principle I try to follow as a PI is to spend my coding time in ways that benefit the entire lab's abilities. I determined that our previous workflow, based largely on psiTurk and custom Javascript/JsPsych, had become outdated and was holding us back.
Smile has totally changed the landscape for us internally. In the last two years, we've found that development time for experiments has dropped dramatically, and we are able to tackle more ambitious designs and interfaces. This includes simple things you can do with other libraries (e.g., a form, a simple perception or memory experiment), as well as complex ones (e.g., ARC challenge, complex 3D interactive experiments, and multi-player video games). This has accelerated in the last few months because Smile leverages popular open web standards that are fully supported by contemporary AI coding tools. Now, instead of worrying about data saving, condition assignment, or implementing comprehension quizes, almost all of our grad students’ time is spent where it matters: running and improving the experiment logic.
Perhaps the most distinguishing feature of Smile is that it provides an entirely new API for developing tasks which is very flexible and powerful, while having a concise syntax (see here for a walkthrough). To get a flavor, check out this example task showing all the code required to run a simple task including saving data, randomization, etc.
In addition, Smile provides "developer mode" tools which are a set of UI elements that appear only while you are developing your project. These UI elements allow you to perform "brain surgery" on your running tasks, helping to reduce and isolate bugs. You can use the developer mode to jump between sections of your experiment (e.g., bypass instructions and jump to the task), "pin" certain parts of the task to aid in debugging, autofill forms with fake responses in just one click, and inspect/modify the running state of the experiment without needing to use a code debugger. In addition, Smile provides inline tools to help you see how your task will appear on different sized devices (phones, tablets, computers) and can help you target visibility and participant experience in multiple formats.
Here's a video demonstration of the idea:
Smile tasks are written entirely in Javascript or Typescript using the popular Vue framework. Smile doesn't leave you hanging on the styling and design front either, providing high- quality, accessible, and consistently styled components (e.g., buttons, popups, forms, layouts, etc.) built using Tailwind and Shadcn. The final application is built using automated code processing tools which ensure your code loads quickly and properly on multiple browsers (using Vite). Smile projects use Google Firebase to save data continuously during the task. Smile tracks many features of the task interaction that can be used to identify bots, while linking each data record to the Github commit that created it. Smile includes a sophisticated setup for automatically deploying your code to a webserver any time you make changes using Github Actions and integrates into your lab workflow with Slack notifications.
We put a lot of extra work into making this ready for people outside our lab group. Smile is fully documented at http://smile.gureckislab.org and has over 300 tests. The codebase is already mature, being used in multiple lab projects, and has over 1,600 commits. There is a roadmap of future features we want to build going forward, but we feel the current base API is solid enough to share with others.
Oh, and it's called Smile because it is written to make you smile while developing your experiments. Check it out, and let us know what you think, and maybe help us make it even better!
Smile development was supported by National Science Foundation Grant BCS-2121102 to TM Gureckis for a project to democratize research in the cognitive science of intuitive physics.