This is the website of the Augmented Reality Instruction Benchmark, which was proposed in our paper [1].
If your results are better then the current benchmark, please email me at albrecht.schmidt@acm.org and we will update the high score table.
We provide printable instructions for our proposed reference tasks.
Download:
Instuctions as PDF: Duplo Tasks (4,8,16,32 steps)
Instuctions as images in a ZIP: Duplo Tasks (4,8,16,32 steps)
Instuctions as PDF: Screws Task (4,8,16,32 steps)
Instuctions as images in a ZIP: Screws Task (4,8,16,32 steps)
Accordingly, we propose using a reference setup for the proposed assembly tasks. The arrangement of the setup and order of the parts can be seen in the pictures below.
We will provide a technical drawing which specifies the assembly areas in detail soon.
Highscore List Duplo Task:
| Paper | Instruction Method | t_locate | t_pick | t_locate_part | t_assemble |
| Funk et al.[1] | Paper Baseline | 2.139 | 0.895 | 1.258 | 1.048 |
| Funk et al.[2] | In-Situ Projection | 1.300 | 0.78 | 0.85 | 0.80 |
| Funk et al.[2] | Paper Baseline |
1.69 |
0.77 |
0.72 |
0.81 |
| Funk et al.[2] | Tablet instructions (Nexus 9) |
2.53 |
0.82 |
0.78 |
0.86 |
| Funk et al.[2] | HMD instructions (Epson Moverio BT-200) |
2.72 |
0.99 |
0.97 |
1.03 |
| Blattgerste et al.[3] | Paper Baseline |
0.92 |
0.62 |
1.12 |
0.7 |
| Blattgerste et al.[3] | HMD instructions (Epson Moverio BT-200) |
1.12 |
0.69 |
1.14 |
0.73 |
| Blattgerste et al.[3] | Smartphone |
1.29 |
0.73 |
2.19 |
0.82 |
| Blattgerste et al.[3] | HoloLens |
1.11 |
0.66 |
1.83 |
0.7 |
Highscore List Screws Task:
| Paper | Instruction Method | t_locate | t_pick | t_locate_part | t_assemble |
| Funk et al.[1] | Paper Baseline | 2.232 | 1.483 | 1.268 | 6.061 |
We will update this highscore list with AR instructions that use our proposed reference setup to enable a fast comparison between all AR instructions that are proposed by the community.
References:
[1] Funk, M., Kosch, T., Greenwald, S. W., & Schmidt, A. (2015). “A Benchmark for Interactive Augmented Reality Instructions for Assembly Tasks.” in Proceedings of the 14th international conference on Mobile and Ubiquitous Multimedia, 2015.
[2] Funk, M., Kosch, T., & Schmidt, A. (2016). Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 934-939). ACM.
[3] Blattgerste, J., Strenge, B., Renner, P., Pfeiffer, T., & Essig, K. (2017) “Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks.” In Proceedings of the 10th ACM international conference on pervasive technologies related to assistive environments, 2017