Incorporating Augmented Reality and technology into a historic museum experience

During my senior year of undergrad, my advisor, Sharla Hanaoka, asked me if I was interested in joining an innovative project that involved the First Lady of Hawaii, Mrs. Dawn Ige, and her involvement with Washington Place, a historic home in downtown Honolulu. The property is best known for being the private, residence of Queen Liliuokalani, and it was subsequently used as the residence of all Hawaii governors. Today it also a museum where the general public can schedule tours to visit the house.

Initially, Mrs. Ige requested an experience that incorporated digital technology into the museum. We came up with the idea of creating a mobile app that uses Augmented Reality (AR) to enhance the visitor experience.

January 2018 – Present

Finish building the app for a public launch in the summer

I had no business launching an AR product—I had no previous experience with it. But if there was a will, there was a way.

I first started by conducting market analysis on the technologies used by different museums. Within the local market, I found that the popular Bishop Museum used QR codes so users could scan them for more information. With another popular museum, Iolani Palace, one thing that stood out for me was their website. They featured a lot of information on their collections and artifacts, including a 3D virtual tour. I also researched museums abroad and focused more on successful use cases of AR. I found that the apps that did feature AR created games or incentivized users to explore their museum in new ways.

I concluded from my market analysis that AR could be really successful in creating these engaging and fun experiences, especially for younger audiences. For further research into Augmented Reality, Sharla mentioned an AR program called Aurasma where users can scan 2D and 3D objects with their phones and reveal more information about the object in front of them. The program was not only easy to use, but we also had the option to install the program into our very own Washington Place app with an SDK.

 From my research I found that a successful museum app presents their content to their users in two ways:

• A more traditional way to browse via galleries and text where users can read more about something they’re interested in

• A more interactive way that implements new technology in order to encourage users to engage with artifacts when they’re at the museum in person.

Therefore, in our app, we would have 2 ways to explore artifacts—”Exhibitions” and “Scan”.

With the “Exhibitions” feature, users could scroll through the different artifacts and galleries without having to be physically near the artifact. This allowed for more of an exploratory way to view the museum.

With the “Scan” feature, users could scan the artifact with their phones and learn more about what was in front of them. This allowed for more in-depth learning.

I wanted to blend the past with the present—Washington Place’s rich history with 21st century technology. I chose purple and gold to represent Hawaiian royalty, and because purple is also the color of the Queen’s favorite flower, the crown flower, which is also used in the quilted pattern designed by Sharla.

In December 2017, we debuted a beta version of our app and the planned visitor experience to Mrs. Ige and gave her a demo of the Browsing and the Interactive features.

During our testing, we did discover that the museum did have weak WiFi signals in certain parts of the museum, which affected the performance of the AR feature. With the feedback and learnings we got from this beta test, our next step was to start the build for a public launch.

“Wait, what happened?”

For Phase 2, we brought in a dedicated programmer onto the team to help us build the app fully. However, in this phase, we experienced A LOT of technical issues.

The 1st issue was with the new program we were using to build the app, which was Xcode. We had difficulties building out the app in that system. The 2nd issue was that the AR company we used, Aurasma, actually made their programming kit unavailable to the public just a few days after we debuted the beta version to the reopening ceremony. So we had to put a pause on the AR feature….

So after all these issues, I concluded as an interim we could build a website that was accessible through an internet browser rather than a native app. This meant was that the two experiences that we had designed originally would be experienced in two different platforms, a website and the in-person experience. The website would focus on the Browsing experience where users could read more about the items in the museum when they’re not at Washington Place.

Below are mockups of this website:

At the reopening ceremony, people really loved the AR scanning feature and were impressed by its potential at Washington Place. We saw kids running around the museum scanning every artifact they could and just blissfully running to look for more artifacts to scan. People were definitely really impressed by the AR feature, which led me to realize this would be an important feature in the Washington Place experience.

However, because I was graduating, this project had to be handled with Anna the new designer who would take over the project. We went through a bunch of debriefing and I walked her the handoff to make sure she had everything she needed to see the project through.