Jiaxing's Portfolio
My Exploration of AR
I have conducted long-term and in-depth exploration of AR games in the graduate level. At first, I just tried to put 3D mind maps in AR in the course project. In the subsequent research courses, I designed a series of game prototypes to explore interesting AR gameplay. During my exploration, I learned a lot about C#, LAN connection, server, Android, mobile phone sensor, LeapMotion, cloud anchor, blueprint communication. This chapter will briefly describe my exploration and final idea.
Software & Technology Used
Unity, Unreal 4, Google Arcore, Microsoft Azure, Procreate, Leap Motion, P5.js, Maya
Exploration in the first year
The beginning of exploration
In the P5.js programming class, I tried to use the ipad sensor to remotely control my mini games on the computer. This project inspired me to find innovative ways to play video games.
My second project is to test an innovative game method. I tried to combine AR technology, ARG game and LAN technology to enhance the immersion of puzzle games.
To explain my idea more easily, I drew the following comics.
My second AR project
I used Unity to make a Demo to test new ways of playing. This demo needs to connect the computer and the mobile phone. The computer represents the real world, and mobile AR is a virtual space. I try to encourage players to combine clues from reality and virtual space to solve puzzles to blur the boundary between virtual and reality. For example, in the video, the player needs to combine the password obtained in the virtual space through the camera and the password displayed on the computer in the real space to solve the mystery. The content displayed on the computer is also controlled by the mobile phone in the virtual space. This gameplay also has the potential to be connected with the Internet of Things technology to further blur the boundary between virtual and reality.
Leap Motion and AR
In the second year, I continued to try other gameplay and made some demos. The Demo in the video is my attempt to combine Leap Motion and AR. This kind of interaction is very interesting, but due to the lack of official Leap Motion support for Android and UE4, I had to stop the research on Leap Motion.
Multiplayer cooperative fishing game
By learning UE4's LAN connection and blueprint communication, I and Heran jointly completed a multiplayer AR cooperative game. In this game, players need to use two mobile phones to fish cooperatively. But in the final test, we found that we could not unify the anchor points of the two phones, which means that each player's birth point and viewing angle are different. One solution is to use markers to unify the anchor points, but it may be subject to many restrictions. I turned to research cloud anchors.
The following comic explains the difference between ordinary AR and AR with cloud anchors
After that, I conducted a lot of research, because the AR Cloud Anchor example provided by Google only supports the old UE4, which caused a lot of bugs, compilation and crash issues. An opportunity led me to find an alternative to Cloud Anchor, Microsoft's Spatial anchor. After many attempts, I gave up using UE4 to complete this project and started using Unity. I made a lot of changes to the code to allow me to upload multiple anchors. At the end of the course, I basically completed this demo. This is a continuation of the second project mentioned earlier. Players can use two mobile phones to work together to solve puzzles in the same space.
This game play allows players to build customized game environments in their own homes. In this way, I hope to further blur the boundary between virtual and reality, increase immersion, and encourage players to cooperate. I believe this idea is also related to the meta-universe that is popular this year.
