Wearables as a Multi-Model dataset

Today I learned about that people strap GoPro cameras to people in order to help annotate sensor readings. In particular, there's this dataset called WEAR that you can play with, which is presented in this paper.

They use a type of smartwatch known as a bangle.js watch. It seems to be designed to be very hackable, which allowed the researchers to construct their dataset. They strapped a bangle to each limb and started recording people.

When I first heard about this, the idea sounded somewhat original. But the paper points out that there are a whole bunch of datasets in this domain. !

There's even specialized transformer models, like ActionFormer, for these kinds of tasks

Part of what makes this so interesting to me is that it also makes a lot of sense to strap a camera to somebody just to ensure that you're annotating data properly. It will allow you to re-label the dataset if you ever decide that you want to measure another part of the activity. Things like "are you running on pavement" vs, "running on grass" may no longer require an eager space cadet to go for a run with hardware.