STUART ERIC PENMAN
ABOUT ME HOW I WORK CASE STUDIES

CASE STUDIES

This work is shared privately.

Enter the password to view selected case studies. If you were invited to review this portfolio, Stuart will have sent it to you.

Incorrect password. Try again.

CASE STUDIES

NHI Strategy Skills Kit Xfinity Flex Horizon MSS

JUMP TO

01 NHI Strategy 02 Skills Kit 03 Flex 04 Horizon

CASE 01

NON-HUMAN IDENTITY STRATEGY

Ongoing strategic program · C-suite · NHI / Agentic AI

Research that defined NHI strategy for IBM — white space the business did not yet see. Now a top Security priority.

NHI / agentic identity lifecycle: Discover & Onboard, Secure & Delegate, Observe & Remediate, Lifecycle & Govern
Agent identity lifecycle — discovery through continuous governance
Context & stakes
WHERE IT STARTED

Legacy PAM vs. agile challengers. IBM needed differentiation clarity and early signal on emerging needs. What began as PAM opportunity discovery surfaced non-human identity proliferation — service accounts, bots, AI agents — as a category gap, not a feature request. That signal did not stop at Registry. It opened a strategy conversation about the full NHI problem space.

WHERE IT IS NOW

NHI has become a top Security priority. Research continues to define strategy and establish shared understanding for leadership — including C-suite stakeholders. JIT access, identity ephemerality, guardian agents, continuous governance, discovery and exposure management are on the table because research made them visible when the business did not yet know them by name.

Role & continuous program

Not a closed project. A research program that keeps answering the next strategic decision.

Owned research strategy for Verify PAM and the expanding NHI / Agent Identity thread. Designed the multi-study sequence, ran or co-ran each phase, embedded with product and design, and kept decision-makers — including senior and C-suite stakeholders — inside the learning loop. Research operates as continuous input to strategy and roadmap, not gated reports after the fact.

SENIOR AUTONOMY & C-SUITE SIGNAL

Set the research agenda against strategy. Sequenced studies. Chose methods. Translated findings into recommendations leadership used — including framing white space the business had not previously recognized. When the work moved from PAM opportunity into NHI as a top priority, research stayed in the room: defining what matters, establishing shared language, and keeping strategy grounded in customer and practitioner reality.

Research arc — five connected phases
  1. PAM Opportunity Discovery — Market, maturity, competitive gaps. Emerging NHI / Zero Trust needs surfaced. Strategic recommendations.
  2. Quantitative MVP Ranking — 30+ enterprise customers. Forced prioritization. Exposed more table stakes than product assumed. Improved IBM quant tooling.
  3. Jobs & Use Case Discovery — JTBD across identity discovery, lifecycle, JIT, compliance. Table stakes vs. unmet needs.
  4. Agent Registry Concept Research — Rapid iterative testing + concurrent JTBD. Multi-stakeholder requirements. Path toward Gartner and Private Preview.
  5. Ongoing NHI strategy (current) — Research continues to define the broader NHI agenda for leadership: JIT, ephemerality, guardian agents, continuous governance, discovery & exposure — white space the business did not previously own.
White space research made visible
JIT ACCESS

Just-in-time access patterns for non-human and agent identities — when standing privileges are the risk.

EPHEMERALITY

Short-lived, purpose-bound identities for agents and workloads — not long-lived accounts treated like users.

GUARDIAN AGENTS

Agents that govern other agents — continuous oversight as autonomy scales.

CONTINUOUS GOVERNANCE

Governance that runs with the identity lifecycle, not periodic review bolted on after the fact.

DISCOVERY & EXPOSURE

Finding non-human identities and their exposure surface before they become incidents.

BEYOND REGISTRY

Registry was necessary but not sufficient. Strategy had to span discovery, lifecycle, access, governance, and agent-to-agent control.

SCALING FAILURE MODES

Customers were already hitting volume and control problems as non-human and agent identities multiplied — governance, discovery, and exposure issues the business had not yet named as strategic risks.

Decision thread & methods

Opportunity discovery → NHI framed as strategic priority (category gap, not a feature).
Ranking + JTBD → MVP scope and needs statements locked.
Registry concept research → Multi-stakeholder requirements + path to Gartner / Private Preview.
Ongoing NHI research → Broader strategy agenda for leadership.
C-suite engagement → NHI as top Security priority.

Methods: Semi-structured interviews · quantitative ranking (30+ enterprise; 60+ customers across the program) · JTBD + use-case mapping · rapid iterative concept testing · continuous generative research + competitor/desk research so the customer-defined problem space is refined against market movement.

Insights that changed strategy
  • NHI is a category gap, not a feature — Proliferation already felt; existing PAM/IAM treated non-human identities as afterthoughts.
  • Table stakes vs. differentiation locked — Rotation, session recording, audit = non-negotiable. JIT, lifecycle automation, AI-assisted discovery = differentiation.
  • Four distinct jobs around agent identity — AI developers, IAM admins, security, auditors need different outputs from one control plane.
  • Registry is necessary but not sufficient — Strategy must span discovery, ephemerality, JIT, continuous governance, and guardian agents.
  • White space the business did not own — Named and framed from customers, refined against competitor and desk research so leadership could skate to the puck.
  • Scaling issues were invisible to the business — As agent and non-human identity volume grew, customers were already hitting governance, discovery, and control problems at scale that product strategy had not yet accounted for. Research made those scaling failure modes explicit — and urgent — before they were on the roadmap.
  • Defining an emerging space — Establishing understanding for leadership was as important as any single finding.
IMPACT · STILL UNFOLDING

Research insights uncovered a strategic blind spot that should have been — and now is — a core focus. Top Security priority. C-suite stakeholders. Still unfolding as strategy converts to capabilities. Along the way: de-risked MVP scope; Agent Registry path to Gartner readiness and Agent Identity Private Preview under Verify; improved IBM quant ranking tooling for the broader UXR org.

CASE 02

XFINITY SKILLS KIT

Enterprise mixed-methods · 0→1 platform

Stood up a scalable enterprise chatbot authoring platform under COVID self-install pressure — non-engineers authoring at scale; research practice strong enough to help fund a permanent role.

Xfinity Skills Kit authoring interface and customer-facing chatbot output
Skills Kit authoring surface → customer-facing Xfinity Assistant output
Context, problem & goal

Context: COVID drove unprecedented demand for home Wi‑Fi but also forced 100% self-install. Support volume spiked. Product needed a chatbot authoring system non-coders could run — fast. Existing process: terminal view, engineer-hard-coded, few intents. Blank slate: no established authoring model to validate.

Problem: Customers handled technical installs they never had before. Support call volume skyrocketed. Product needed to expand self-serve chatbot capability fast.

Goal: Stand up a robust chatbot by creating a back-end authoring system non-coders could use. Existing bot: terminal view only, engineer-coded content, few intents.

Challenges & collaboration
CHALLENGES

Command-and-control management vs. empowered teams. Multiple re-orgs confused ownership. Tool type unfamiliar to executives. Competing goals. No internal recruitment mechanism. Stressed timelines required creative research planning — multiple, iterative, overlapping methods balancing depth with speed.

HOW I COLLABORATED

Embedded with stakeholders — not a service desk. Core team: 2 designers + design PM · coders/prototypers · product lead. Research questions, methods, and reporting cadence designed so the team could absorb findings between sessions. Recommendations often handled offline in iteration sessions so design could act immediately.

Research questions & methods
  • What are the needs and goals for using a chatbot authoring tool? → Inform V1 before high UI investment.
  • What are users’ mental models for Skills Kit? → Goals, ecosystem, opportunities task testing would miss.
  • Could users find, create, edit, and review capabilities? → Early evaluative on critical MVP path.
  • What barriers exist? → Workflow mismatches before they hardened.
  • What should Skills Kit be used for? → Scale horizon beyond immediate MVP.

Methods: Mental modeling (Indi Young) · work shadowing (multiple rounds) · co-creation & low-fi concept testing · iterative usability (3+ rounds) · one-page iterative reporting between sessions.

Mental model — Find · Evaluate · Fix
FIND · EVALUATE · FIX

Authors oriented around three goals. Tools and pain points mapped under each — driving IA and workflow decisions.

Find: Look for utterances, intents, transcripts related to a topic. Pain: hard to find transcripts; search unclear; reports couldn’t pull utterances across systems.

Evaluate: Identify problems with existing content or opportunities for new content. Pain: evaluating transcripts time-consuming; incomplete transcripts; review one at a time; customer history outside Skills Kit.

Fix: Solve problems or summarize for someone else. Pain: unclear who may edit (CXAO, Legal, Business sign-off); updates per individual intent when global changes needed.

Key insight: Authors think in customer outcomes — not only bot “intents.” Preview is non-negotiable. Findings most valuable in the next design session the same week.

IMPACT

Non-engineers were authoring production skills at scale within weeks of research beginning. Supported self-install pivot under extreme call-volume pressure. Research practice contributed to funding a permanent research role. Established iterative, decision-ready reporting product and design absorbed without waiting for a final big report.

CASE 03

XFINITY FLEX

Course-correction · Strategic evaluative

Stopped a release built on the wrong behavioral model. IA overhauled for OTT direct-retrieval behavior. Estimated avoidance of what could have been a ~$20M miss.

Xfinity Flex home interface with content rows and app tiles
Xfinity Flex interface under research — content model and navigation patterns
Context, goals & recruitment

Flex was intended as the primary streaming service for customers who downgraded to internet-only. Design applied need-states and content-foraging frameworks developed for pay-TV customers. Product was far along.

Product goal: Validate designs prior to release.
Research goal: Evaluate how OTT users watch, and whether the design supported those behaviors.

Strategic risk was not usability bugs — it was shipping a product whose foundational behavioral model did not match the audience.

Recruitment (n=10): Former pay-TV users who cut the cord · watched Netflix via streaming box/stick · paid for OTT · >1 hour OTT/day · at least 5 sports fans. Natural tasks elicited using Spradley grand-tour / mini-tour probes, then used to evaluate whether UI / IA supported users. Rapid iteration in lockstep with design and product.

Core finding & implication
DIRECT RETRIEVAL — NOT ORIENTEERING

The team built the service using frameworks for pay-TV customers and did not recognize that OTT needs and behaviors differed. OTT users predominantly exhibit “direct retrieval” content foraging — they often know what they will watch when they sit down. Orienteering (browsing) is secondary.

Implication: Design for getting users to known content fast and anticipate needs — not for extended discovery-first experiences. IA had to be overhauled across the system.

Retrospective

Shows the value of bringing research in with authority and the right frame even late — when the cost of being wrong is highest. Would push earlier for behavioral-model validation on any product built from a different segment’s framework.

Strategic evaluative research: not “find the bugs,” but “test whether the product model matches the user.” The analytical lens (content-foraging / direct retrieval vs. orienteering) made the finding portable so design and product could act on a clear behavioral model.

IMPACT

Team postponed release to iterate. Research provided the justification under real release pressure. IA overhauled to support OTT direct-retrieval behavior. Product leadership calculated a $20M miss avoided, directly attributed to research insights. The insights changed the product fundamentally — not just a few labels.

CASE 04

HORIZON SINGLE PANE OF GLASS

Strategic turnaround · SOC platform · IBM MSS

Product shipped without research. Adoption collapsed. Churn risk. Business needed a fix fast. Research uncovered the core mismatch — and grounded the viability of the whole single-pane-of-glass approach.

Horizon MSS Investigations search — single-pane search for tickets and incidents
Horizon Investigations search — single-pane discovery across tickets and severity
The situation

Shipped blind: Horizon launched as a single pane of glass for SOC analysts — without research input. Assumptions about how analysts find and work tickets were untested.

Adoption collapse: Users could not find tickets. They abandoned Horizon and went back to VSOC and other systems. Low adoption was not a polish problem — it was a product-viability problem.

Business pressure: Churn risk. Product had no clear diagnosis. Leadership needed a fix fast — and an answer to whether the single-pane-of-glass approach itself still made sense.

Role, questions & method

Role: User Research Lead, MSS. Charge: diagnose why Horizon was failing and inform the redesign — fast enough to protect the business case for a single pane of glass for SOC work.

Questions: What reasons prompt users to look for tickets? How do they look today in VSOC and systems they actually use? What information helps at a glance? What navigational aids help — and which do they ignore? How do they think about investigations vs. requests when searching?

Method: Generative interviews + concept review (lo-fi). N=8 IBM MSS customers — SOC analysts. 60-min sessions. Depth over volume for a directional pivot under time pressure.

The finding that changed everything
KEYWORDS — NOT TICKET IDS

Users don’t search by ticket ID. They search with keywords and context — descriptive language that summarizes the work. That single mismatch explained the abandonment. Fixing search was necessary — the deeper value was proving the product model could work if grounded in real behavior.

Information scent: Users scan for relevance at a glance. Generic results with no distinguishing summary cause abandonment.
Navigate, then filter: Analysts often go to a known area first, then filter — including dimensions VSOC already offered (who worked it, team).
Investigations ≠ requests: Separate mental models. Treating them as one pile fights how people already work.
Need-states shift: Reasons for looking vary by situation and role.

The thread that followed
  1. Diagnose the abandonment — Not a usability bug: a core mismatch between design assumptions and how analysts search and orient.
  2. Pivot search design — From ticket-ID-centric to descriptive keyword search with strong information scent.
  3. Ground navigation & structure — Navigate-then-filter; separate investigations and requests; richer filters. Library analogy: direct hit · neighborhood · browse.
  4. Re-legitimize single pane of glass — Once the mismatch was visible and fixable, the approach itself was no longer the question. Research conceptually grounded the potential of Horizon as a unified SOC workspace.
IMPACT

Diagnosed why adoption failed: core mismatch between product assumptions and how SOC analysts search and orient — not a surface usability issue. Pivoted search design. Informed navigation structure, filter dimensions, and separation of investigations vs. requests. Conceptually grounded the single-pane-of-glass approach after it had been called into question. Delivered under business pressure for a fast answer — generative depth with a lean sample appropriate for a directional pivot.

EMAIL

s.eric.penman@gmail.com

LOCATION

Raleigh, NC · Remote

Stuart Penman · Staff User Researcher · Strategic Research · NHI & Agentic AI