How a motion-tracking musical software is breaking down barriers for people with disabilities
She wasn’t going to be able to make it, but he went anyway. Even if it was a small step to make his business partner, Julie Unruh, 38, feel that independence — a rightful aspiration — he was still going to be there. Her improvement since the accident almost 19 years ago was proof enough.
During an evening at the Lawrence Public Library on Tuesday, Feb. 28, Oliver Hall, 46, sat with an intimate group of Lawrence residents to listen to the music they just created. It was music created during a jam session without traditional instruments — music that Hall helped make in his group, to help her.
The band members sitting around Hall that day, some anxious first timers and others poised like seasoned artists, came from various walks of life. Some were there representing the local center for independent living Independence Inc, others were there with ties to the University, like assistant professor and musician Brad Osborn, and then there were members like Hall; interested individuals from the area both with and without a physical or mental disability that come together for one purpose — creating art in a comfortable and conducive environment using an inclusive piece of technology: the Adaptive Use Musical Instrument (AUMI).
“There are a very few communities like this. Where you can come here and there’s so much creativity going on,” Hall said as the monthly jam session began to settle down after listening to each other’s recordings. “No matter what your ability or disability is, you can just grow. You can just do things. It’s a mindset. You have to have people who think forward.”
What was sitting in front of Hall — an iPad placed on a stand that can be played after it hooks onto a firm base — was a major reason why he thought so. The AUMI interface, available on desktop, iOS and Android, is a one-of-a-kind motion tracking program that promotes musical improvisation. It works by detecting high-contrast areas of the video subject on screen. With each movement it was set to track, it begins to play notes from the hundreds of instruments and soundboard bites available in its software. Henry Lowengard, one of the app’s developers who is based in Kingston, New York, said the AUMI also takes in data every 15 seconds of a user’s productivity, thus giving it the ability to be used for research and clinical studies.
There are other similar types of software that look to help people with disabilities, such as the music-to-music system (MTM) and the Virtual Music Instrument (VMI), but they work by prompting users to play recognized melodies, whereas AUMI encourages collaboration and creativity by giving people the freedom to make music of their own.
AUMI is also able to cater to a person’s physical and mental abilities. It can track a subject’s nose twitch, blinking, even differing elevations of the chest cavity while breathing — a tool that helps people with little voluntary or involuntary movement, according to a 2011 article by renowned composer and musician Pauline Oliveros published in research journal “Music and Medicine.”
Earlier this year, a University group that aims to bring together people of varying abilities, AUMI-KU InterArts — also one of the key groups behind the monthly jam sessions — received a $35,000 grant from the National Endowment of the Arts to hold a symposium in August on the technology. The week-long line of events will include opportunities for people from across the globe to learn more about AUMI’s capabilities with workshops and an international research consortium on how to move forward with the instrument technology.
Hall hopes to bring Unruh to the symposium. Unruh, a Montezuma native, has only been to one of the Lawrence’s AUMI Jam Sessions so far. The first time Hall and Unruh had gone years ago — Unruh said at a coffee shop in downtown Lawrence on a Saturday afternoon — she was discouraged that the AUMI wasn’t able to provide for her a musical score for one of her published works.
“Vegetable Garden” is an illustration book by Unruh. The story details her life and struggles during and after her accident. It was published in 2013 in cooperation with Mammoth Publications.
Despite her lack of confidence in the technology, Hall saw something different in Unruh when she played and listened to the AUMI. It’s what keeps him coming back every chance he gets.
“When she’s playing music, she’s able to focus, write better and feel better. I think that’s a big part of helping her do even better in the future with her writing,” he said.
Sherrie Tucker, a professor of American Studies at the University, along with a group of University colleagues started the jam sessions and also helped bring the grant and symposium to Lawrence.
Tucker had worked with Pauline Oliveros, who was based in New York and was a key figure in the creation of the AUMI, during international meet-ups called “think tanks.” It was a relationship that started over 10 years ago. Oliveros passed away in November of 2016 at the age of 84.
“[Oliveros] always had this philosophy, that she didn’t want to only make music that virtuoso trained musicians could play — that what she thought was really important about music is expanding it so that we could all learn from each other and hear from each other. She didn’t really make a distinction between those who can and those who can’t. One of the things she said is ‘a true virtuoso is somebody who can play with anybody and have a meaningful experience,’” Tucker said.
Oliveros’ studies with the AUMI began in 2007 at a school in Poughskeepsie, New York called Abilities First, along with occupation therapist Leaf Miller. She found three principal takeaways that can build toward a better future with people of disabilities: giving them a rightful sense of independence to fulfill one’s artistic dreams, an avenue for therapeutic and clinical benefits from using the instrument, and something that Tucker has also found during her work using the AUMI: the ability to break down the barriers in society between those who don’t have the same abilities as others.
“I think that really is a barrier that is pretty naturalized in society. It’s almost like you have to be a person with disabilities, or be in a family with a member with disabilities, or you just don’t see them,” she said.
“Vegetable Garden”
Sitting in the booth of La Prima Tazza coffee shop in downtown Lawrence, Unruh looked over at Hall. She then remembered what got her through the aftermath of her accident: Classical music.
Specifically, Beethoven.
“That’s one thing I remember my mind thinking of when i was in that coma. That and a Julius Caesar speech I had memorized when i was 16,” she said.
In July of 1998, Unruh was a year into her English degree at Kansas State University. As a highschool graduation gift for her younger sister Valerie and a celebration of Unruh’s early college achievements, her grandparents took them on a cruise along the West Coast.
As described in her book, the family was coming back from the trip when their camper broke down near Rock Springs, Wyoming. There, they were forced to find a rental car to get home. Her grandfather at the time was facing issues with alcoholism and was on heart medication, Unruh said.
When coming to an intersection on that early morning, Unruh noticed he wasn’t stopping.
“I bit my tongue,” she said. “And I went right into a coma for three months.”
After failing to stop at a stop sign, the car holding Unruh and her family collided with an incoming semi-truck. Her grandfather passed away shortly after the accident. Everyone else survived, but not without sustaining major injuries.
Unruh may have received the worst of the blow, receiving multiple broken ribs, broken pelvic and collar bones, a broken leg, a punctured lung, and more.
The accident also caused permanent brain damage.
Unruh had a long road ahead. The next year she would face a grueling schedule of intense rehabilitation, where she learned to walk and talk again. This was the state she said made her feel like a vegetable — the inspiration for her book.
The feeling didn’t stop there.
“When leaving the hospital, I felt as if i had entered another prison,” Unruh writes in “Vegetable Garden.””I yearned to escape. When I caught a glimpse of life outside the cell, I’d see people hurrying around, not taking any notice of me, ignoring the cell I lived in.”
Whether it’s from before birth or from an accident similar to Unruh’s, people with mental and physical disabilities have had a history of being wrongfully institutionalized and ostracized. Treated as people who can solely be identified with that disability, not by future goals and aspirations.
“When things get really naturalized in society, nobody sees it,” Tucker said.
In a 2016 entry of the peer-review music therapy journal “Voices,” authors Mark Finch, Susan LeMessurier Quinn, and Ellen Waterman were researchers in a study that incorporated the AUMI in a community music therapy program in Newfoundland, Canada. During the six-month research period, three subjects — two children and one young adult — with cerebral palsy and who required wheelchairs were studied for their interaction and improvement with the AUMI technology.
What they found was a goal Oliveros had set out for the AUMI at the start.
“We want to increase their possibilities for choice with improvisation as an empowerment for them,” Oliveros said during a 2010 interview by Andrew Pask that was featured in the journal, “Sounding the margins.”
While AUMI is still an ever-evolving technology, the subjects in the study were observed as being able to make music independently. They found at a grander scope — specifically with Sam’s interactions with observer Mark Finch — the potential for the instrument to add a sense of community and potentially give those with disabilities the power to express their own cultural and social aspirations in society.
“For individuals like Sam, AUMI’s adaptability has the potential to enrich these opportunities by enabling him to indulge and experiment with his own cultural/aesthetic tastes,” the journal reads.
“(Un)Rolling the Boulder”
In 2009, Hall and Unruh founded Global Green, a creative entity that brought together “Vegetable Garden” and other of Unruh and Hall’s work. Not only is Global Green looking to push artistic work, but it’s also looking to advocate for marginalized groups in society.
“You know how a society can treat people of different colors and religions, all this stuff,” Hall said. “It’s all about ostracizing people and kicking them out. But those people who have been ostracized and kept out who feel like they have power, they can help change the whole mindset of society like that.”
Hall points to Unruh next to him: “I think she’s a part of that. She has the potential with her writing. She has the potential to go beyond that and go to levels she’s never dreamt of or I’ve ever dreamt of.”
Ranita Wilks, the independent living skills/peer counseling specialist for Independence Inc. agrees that barriers are being broken for people of both abilities and disabilities to have equal opportunities.
“We are now more than 25 years past the Americans with Disabilities Act, so education is still ongoing. People are beginning to understand,” she said.
Wilks, a 1998 University alumna, sees the AUMI technology as a key piece to that puzzle. Herself a paraplegic after a childhood accident, she hopes an artistic hearth like Lawrence can be the beginning of something even bigger.
“Independence Inc., KU, and the library are bringing these ways to engage people with disabilities in an integrative setting,” she said. “We want our communities to be fully accessible and integrative, with everyone coming together and not being segregated and seeming as if they’re different from one another.”
Lawrence isn’t new on the road to inclusion. In 2013, AUMI KU-InterArts brought twenty people together for a dance performance featuring the AUMI technology.
“Somebody in the group said ‘having a disability is kind of like pushing a pushing a boulder all the time’ and somebody else said ‘sometimes people are the boulder and people won’t move,’ so we ended up working with the concept of the boulder,” Tucker said.
Just last year, other Lawrence organizations like the Lawrence Arts Center and the University of Kansas Center for Research also received NEA grants to bring those with disabilities from the community and across the nation together to celebrate inclusivity and the arts.
Now with the upcoming AUMI symposium, more people like Hall and Unruh can come together to see the possibilities still on the table to bring the community together.
As the group during the February AUMI Jam Session slowly dispersed, Hall stayed around. Just because Unruh couldn’t go, Hall was still going to come back to her with music.
As Hall collected his things, he tidied up a stack of papers and books in front of him. While organizing, he pulled out a thin book, one of his recently published books, and gave it a look.
“It’s merging Jimi Hendrix’s lyrics with Miles Davis’ artwork. Some people don’t know Miles Davis was a painter,” he said as he began flipping through the pages.
Some people also didn’t know Unruh would be where she is today. Hall said he intended to bring the track he worked on that day, “Disfunctional Noise,” back to Unruh for it to play on her headphones while she wrote.
It’s a small step, but an important one.
“I’m still helping her because, after a brain injury, your body is still healing. Your body is still trying to gain,” Hall said. “Musically, it opens up her mind. She feels better. The music is an incredible healer.”