Portfolio | User Research & Experimentation
Curious user researcher with a creative mind.
Go on, drag the background.
You just chose your own viewpoint. My experiments found that choosing their own viewpoint builds trust, but telling people where to look drops trust.
Why did immersive (VR) news stall?
I interviewed the people who build it. Then I tested their assumptions on users.
Immersive (VR) 360-degree news was funded heavily, then quietly stopped being made. I asked the people who built it how they used it and what didn't work out, then tested what they believed on 999 users.
Talk to the builders
Twenty practitioners in six countries on how projects started and stopped. They named the assumptions and pain points. Core assumption: using VR to earn trust.
Study 2 | Test the designWeigh the design choices
Sixteen versions, 999 people, 7,992 ratings. Which of four design factors earn trust, and which cost it.
Study 3 | Test the contextChange who ships it
The same product under three media brands. Does tech outweigh reputation, or inherit what people already think of the brand?
What media innovators told me
Twenty in-depth interviews with product managers and innovation leads, in six countries.
- 20 interviews
- 6 countries
- 3 rounds of coding
Media hope VR immersive news can win back trust from the audience. They stopped for reasons outside the tech itself.
Two problems stood out. Teams did not agree what the format was for. One creator gave the sharpest test anyone offered:
"That technology has to increase your understanding of what is going on."
And every usability issue they raised was about control. Professionals debated about the best use of this tech. Producers added on-screen text to steer attention, taking back the freedom the format was sold on.
That became my two experiments.
How I ran it ~30s
Purposive sampling of people publicly credited on published projects, then snowball referrals to reach harder-to-find roles. I targeted different roles on purpose, because the person who builds and the person who budgets tell different halves of the same story. Recruiting across six countries solved a supply problem and meant the findings were not tied to one market's funding structures.
What I would do differently ~30s
- The sample leans toward people who stayed with the format. Sceptics and people who struggled with the technology are under-represented, so a pro-innovation bias remains.
- Three interviews were capped at 30 minutes by participants' schedules. Adapting them beat dropping them, but next time I confirm the time commitment at recruitment.
Testing the design choices
Sixteen versions of the same story, rated by a UK-representative sample of 999 users.
- N = 999
- 2×2×2×2 factorial
- 7,992 ratings
- multilevel models
Four choices came out of the interviews: immersive or flat, a high-profile topic or a quieter one, on-screen cues or none, people in frame or none. Four A/B tests would cost four times the sample and still miss how the choices interact, so I built all sixteen combinations as a factorial test and gave each user eight random news pieces to rate.
Controlling users hurt their trust.
The immersive format did lift trust, but only on high-profile topics. On-screen cues, added so viewers would not miss the point, were the largest of the four single effects, and they pointed down. People showing up in frame also lowered trust, although the immersive view offset much of that.
Build a version
A flat image on a quieter story. This is the baseline.
Single effects: immersive view and a high-profile topic raised trust; on-screen cues and people in the image lowered it. Combinations: immersion and topic prominence only worked together; the immersive view offset much of the penalty for people in the image; cues cost trust in both formats.
Why factorial and not ordinary A/B tests ~50s
Efficiency. Four attributes tested separately need four samples. A factorial design tests all four, and how they combine, in one.
Combinations. The strongest results here were combinations, and sequential A/B tests cannot find those. The single-attribute view alone would have produced the wrong recommendation.
A realistic task. People rarely judge one version in isolation; they compare across sources. Rating several versions is closer to that, and respondents are less aware of their own biases when they rate several things than when they defend one verdict.
How it was built and analysed ~60s
- Stimuli. I built sixteen fictional news items from real news imagery. Two topics set the prominence level: the war in Ukraine, and activists opposing Arctic oil drilling. Visual and emotional intensity was held constant within each topic, and the surrounding layout never changed.
- Model. Multilevel regression with random intercepts, because each person gave eight ratings. Most of the variation in trust sat with the person rather than the version, and ignoring that nesting would have manufactured significance that was not there.
- Eight of sixteen, not all sixteen. Sixteen near-identical ratings invite careless answers, and careless answers look exactly like a null result.
One finding that surprised me ~30s
Media literacy, technology confidence and general trust in news all predicted how much people trusted the content overall, and none of them changed how the immersive format worked. Targeting this kind of feature at "curious people" would spend budget for nothing.
Testing who ships it
The same immersive story under three different brands, with the same sample.
- N = 999
- 3 conditions
- behavioural segmentation
- manipulation check
Professionals assumed the format would help close a trust gap with audiences. But a feature never arrives alone: people see it and the brand at the same instant. So I held the experience constant and changed only the branding: a public broadcaster, a partisan challenger, or nothing.
People trust new tech more from brands they already use. The challenger brand was distrusted by people who do not watch it. The public media withstand challenges better.
Segmenting on behaviour showed who moved: readers furthest from the challenger applied the biggest penalty, and the small group already relying on it became less negative.
This is the study stimulus. It shows the aftermath of conflict.
Content notice: the study stimulus shows the aftermath of conflict. On the web page it is an interactive 360-degree scene.
Swap the branding
Baseline
Nothing but the experience itself.
Result: no brand = baseline. BBC brand = same as baseline. GB News brand = lower than both.
Why a no-brand control ~30s
A two-brand test tells you the two brands differ. It cannot tell you whether the strong brand is adding trust or the weak brand is taking it away, and those lead to opposite decisions: lean on the brand, or treat the brand as a tax on this feature. The unbranded baseline is the only way to separate them, and here it turned out to be the finding.
Data quality and manipulation check ~30s
- Manipulation check. After rating, participants recalled the brand from memory with no way back, from a list including distractors, "I don't remember" and "This information was not provided". They were told the answer would not affect the participation reward, so nobody had a reason to guess. The large majority in every condition recalled correctly.
Three things I found out in this study
- 01
Assumptions around VR need to be challenged.
The overlay added so users would not miss the point was the most costly element in the study. The things nobody questions are where the untested assumptions live.
- 02
Autonomy was the active selling point.
Every result pointed the same way: the format worked when it handed people control, and lost ground when producers tried to take that control back.
- 03
A new format does not fix a trust problem.
It performs differently depending on who ships it and who is looking. Media reputations from the flat era are carried into immersive media.
Who I am
I am a user researcher with 5+ years of experience with mixed-method user research and experimentations, currently finishing a PhD at Erasmus University Rotterdam, where I have spent four years on one question: what makes people trust a digital innovative product. I also have working experience in digital marketing. Research that drives decisions and makes impact is what I am always passionate about.
Methods and tools
Methods
Tools
Where do we go from here?
If you would like to know more about me, or if we can work together: