rep-air
Our product RepAir helps make any electronic device functional again with an augmented reality-based community system that reduces the need to dispose of them when they stop working.
In today’s consumer society, it is common practice to dispose of electronic devices that have lost their functionality.
However, this results in an enormous amount of electronic waste, with 57.4 million tons of e-waste being generated worldwide in 2021.
To combat this problem, the RepAir project was initiated, which aimed to develop a repair assistance system that would enable end-users to make any electronic device functional again. The project leverages the technical capabilities of Augmented Reality (AR) to provide users with expert repair instructions and guidance.
__________
Early Prototyping and User-Testing
Our project team created a low-fi prototype of the system that would allow users to receive repair instructions to evaluate its effectiveness. Using a self-made AR glasses prototype, the test participant could see the paper to be folded into an origami ship via zoom transmission (slightly delayed).
With the help of drawn-in folding lines, arrows and verbal instructions, it was possible to determine that, despite the time delay, the task was completed faster and more efficiently. Based on this feedback, the team developed a detailed concept for the system.
__________
Concept
The RepAir system includes a community platform where users with repair skills can provide assistance to those in need. The onboarding process involves users indicating their repair skills for specific devices, and the AI analyzes this information to determine their suitability as a repair helper.
The users can follow the step-by-step instructions projected onto their field of vision, which guides them through the repair process. If the repair process stagnates, the user can call a community member or an expert who can then draw the problem or the next step into the field of view live. Afterwards users can pay for the assistance they receive from other community members using points.
__________
Research
Our team conducted research to identify the field of application for AR technology. We also evaluated the technological possibilities for AR in the near future to develop a futuristic concept.
Afterwards we concluded that the technical capabilities of AR would receive more attention in Industry 4.0, and AR applications are already being used to clarify complex scenarios, such as training new employees in the automotive industry.
__________
Interface Design
When designing the AR experience, it was also important to consider that holograms are visualized in additive color representation, which means that black appears transparent and white appears bright. If the colors are too bright, it can blind the user or they may not be able to see through the field, making interaction difficult. We designed the system’s interface to adapt to the user’s surroundings. For example, if the room is dark, the interface elements switch to light mode.
For an interaction distance of 45-100cm, font sizes between 12-48px were chosen. A legible sans-serif font was used that has similarities to the Holo Lens font. Choosing the volume at appropriate distances and directions makes the experience more immersive and sound feedback helps guide the user through the process.
__________
Interaction Design
Freehand mode during the repair process , as hands are needed for the repair. This is controlled via eyetracking (eyes pointing at a button, staying on the target selects it), or via voice commands such as „select battery damage“. Once the repair is completed, gestures such as thumbs up can be used.
The RepAir system can be operated using eye-tracking, gestures, and voice commands.
The AI can track the user’s progress, and the next instruction appears automatically when the current step is completed. The progress bar lets the user know which step they are on. We made use of the billboarding effect: the modals are always within sight of the user. The 3D model should be viewable from all sides and could be rotated in the room so that certain areas can be highlighted. It needed to be presented clearly and precisely against the background but also spatially integrated into the physical space to create an immersive experience.
__________
Outcome
At the semester exhibition stand, visitors were able to interactively test how machine learning error analysis could work. Visitors could hold a toothbrush and a remote control in front of the camera, which used YOLO machine learning to distinguish between the two objects.
The final outcome also included an AR UI, the onboarding App UI and the concept video of one use case.
____________________________________________________________________________________________________________________________________________
Conclusion
The RepAir project aims to combat the growing problem of electronic waste and empower end-users to make any electronic device functional again with the help of AR technology. The system’s community-based platform allows users to provide and receive assistance, promoting a culture of repair and sustainability. The RepAir project’s success relies on the continuous development of AI and AR technologies, making it an innovative solution for a pressing environmental issue.
__________
Learnings
My team learned that a community-based system, where users can share their knowledge and skills with each other, is an essential component for motivation. Users can share their repair knowledge with others in the community and earn points that can be used to obtain help when they need it.
Collaboration is key - we also learned that working together and leveraging the strengths of each team member is essential to creating a successful product.
