Artificial Intelligence (AI) is reshaping industries, workflows, and decision-making. Yet amid the rush to automate, scale, and optimise, many organisations still underestimate a critical risk: deploying AI systems without ethical governance. A recent PwC global AI study reports that 85% of AI projects fail to meet expected outcomes due to insufficient governance and stakeholder alignment. Misalignment, bias, compliance gaps and stakeholder distrust often surface not because the algorithms are poorly built but because they were built in isolation from human context.
As enterprise systems grow more intelligent, human-centred AI governance is no longer optional. It is the invisible scaffolding that ensures AI delivers value while respecting fairness, accountability, and purpose. And at the heart of that governance? Not just data scientists or legal teams but Business Analysts (BAs) and Product Owners (POs) who act as ethical translators, systems thinkers, and stakeholder advocates.
What Is Human-Centred AI Governance?
Human-centred AI governance means building AI systems around people, not just data. It ensures that solutions are explainable, fair, inclusive, and aligned to organisational values and regulatory frameworks. Rather than policing AI post deployment, it embeds ethics-by-design throughout the delivery lifecycle.
Governance in this context is not about bureaucracy, it’s about intentionality. It blends strategy, empathy, technical rigour, and cross-functional accountability. The approach draws from standards like ISO 9241-210. ISO/IEC 42001, the world’s first AI Management System Standard released in 2023, further codifies organisational responsibility around fairness, explainability, and human oversight which emphasise user context, accessibility, and design inclusivity. It also aligns with hybrid
intelligence principles: combining human judgement with machine capability in an integrated, value-conscious way.
Where AI Projects Break Without Human Governance
AI governance failures rarely begin with malicious intent. They typically emerge from blind spots:
- Lack of diversity in data or decision-making
- Insufficient stakeholder consultation during discovery
- Overreliance on automation without impact review
- Disconnected workflows between technical teams and operational users
In 2020, the UK Home Office abandoned its visa algorithm after civil rights groups highlighted racial bias in its training data; a prime example of automation without human oversight (Source: The Guardian, 2020).
These breakdowns result in reputational damage, regulatory fines, or underwhelming adoption. Case studies on failed AI deployments in finance, recruitment, and social services consistently trace root causes back to inadequate human checks not bad code.
The Hidden Role of BAs and Product Owners in AI Ethics
Business Analysts (BAs) and Product Owners (POs) are often perceived as facilitators connecting stakeholders, refining backlog items, and managing delivery expectations. But in AI-driven environments, their influence goes far deeper. They are critical enablers of ethical governance, acting as the first responders to ambiguity and the last line of assurance before a model reaches production.
In the realm of AI, decisions are not always explainable, and consequences can be opaque. This is precisely where BAs and POs step in, not just as delivery agents, but as ethical translators, integrity advocates, and system-wide navigators.
Stakeholder Sense-Making
At the heart of AI governance lies the ability to map stakeholder expectations, anxieties, and values. This is not a passive task. It involves interviewing users, understanding power dynamics, and anticipating unintended impacts. For example, when designing an AI that recommends employee task assignments, a BA might detect that certain demographics could be systemically deprioritised due to historical patterns in workload data.
This level of deep listening enables the product team to spot ethical risks early — before the first model is trained.
Embedding Ethics in Requirements
BAs and POs are best positioned to translate ethical principles into concrete functional and non-functional requirements. These might include:
- “Model must offer confidence scores alongside its recommendation”
- “Users must be able to request manual review of an AI decision”
- “System must log decision rationale in audit trails for 12 months”
These are not hypothetical. They are features I’ve scoped in AI governance work — and they are essential to regulatory resilience, particularly in contexts involving GDPR, FCA Consumer Duty, or employment law.
Lifecycle Governance: From Backlog to UAT
Governance must be operational, not aspirational. That means weaving it into the tooling:
- Backlog artefacts in Jira or Azure DevOps can include governance flags (ethics-check, bias-review)
- UAT scripts can test not only for functionality, but for fairness and explainability
- Definition of Done can incorporate ethical signoffs and documentation
For example, in a predictive risk-scoring model, we required that testers run edge case personas across gender, age, and region to check for consistency and bias.
Policy Anchoring and Regulatory Alignment
AI governance is not only about algorithms, it’s about aligning digital behaviours to organisational policy. BAs and POs ensure models comply with:
- Internal data usage policies
- Industry-specific frameworks (e.g., NHS AI guidelines, FCA thresholds) Global regulations (GDPR Article 22, ISO/IEC 42001:2023, ISO 27001)
In one case, I worked with legal, IT, and operations to document how AI-triggered escalations were aligned with consumer harm thresholds: a critical point under the regulatory mandates.
Cross-Functional Collaboration and Escalation
AI products introduce non-obvious dependencies across data science, legal, compliance, UX, and engineering. BAs and POs are often the only roles with visibility across all these functions. Their responsibilities include:
- Hosting governance huddles
- Documenting “grey-zone” risks for executive review
- Facilitating decision logs where trade-offs (e.g., speed vs fairness) are discussed transparently
Their role is not to resolve all risks, but to make them visible, documented, and owned.
Why This Matters?
These governance responsibilities are not peripheral, they are central to responsible AI delivery. By identifying risks early, codifying fairness into design, and holding space for ethical reflection, BAs and POs ensure AI systems are not just intelligent, but accountable and trusted.
In every project I’ve supported, from Identity and access management systems to process orchestration and risk classification, I realise that BA or PO have had to ask the questions the model couldn’t. The ones that began with: “What if this doesn’t happen, What else? What next?” That’s the real role and in the AI age, it’s never been more vital.
Some Lessons from the Field: Automation, IAM, and Complex Governance
Across multiple transformation projects, I’ve seen how human-centred governance plays out in practice.
Identity Governance (Higher Education)
In one Identity Access Management (IAM) transformation project, over 10,000 identities were reviewed across roles, access paths, and workflows. The project avoided privilege creep and bias by involving academic, operations, and security stakeholders; all guided by a BA-led discovery framework. The result: role clarity, access transparency, and an AI-ready identity model.
Automation Orchestration (Logistics Sector)
In a complex automation initiative, AI was introduced to auto-assign tasks based on location, workload, and compliance flags. Rather than letting the AI dictate workflows unchecked, the team (led by POs and BAs) tested multiple edge-case scenarios with operational teams. This human feedback prevented routing errors and surfaced critical fairness concerns before launch.
Compliance & Data Fairness (Financial Services)
As part of a regulatory programme responding to Consumer Duty, automation was deployed to flag high-risk policy breaches. The team built an AI explainer layer into the solution allowing reviewers to understand and interrogate machine decisions. The governance strategy included versioning, audit logs, and fairness monitoring all scoped at the discovery stage.
Overall, These examples prove a simple point: when governance is embedded from the beginning, AI becomes a partner not a liability to back this up; In a 2022 Capgemini survey, 63% of organisations cited ‘lack of business-aligned governance’ as the primary blocker to successful AI adoption (Source: Capgemini Research Institute, AI and Ethics 2022).
Where Governance Lives in the Delivery Lifecycle
Governance is not a checkpoint; it is a thread that runs across the project lifecycle. Here’s a lightweight framework for BAs and POs to integrate it:
Human-Centred AI Governance Touchpoints:
Phase Governance Activities
Discovery Stakeholder analysis, ethics mapping, early risk identification Design Requirements for explainability, fairness, and data sourcing Development Traceable user stories, controls for model behaviour UAT Human-in-the-loop testing, ethical edge case simulation Deployment & Support Logging, monitoring, user feedback loops, retraining triggers
This framework can be adapted to any agile environment and scaled to both low code platforms and advanced ML systems. McKinsey’s 2022 State of AI report shows that companies with formal AI governance are 3.5x more likely to report AI success at scale.
Building a Culture of Responsible AI
While policies and frameworks are essential for ethical AI governance, they are only as effective as the culture that supports them. In forward-thinking organisations, human-centred governance is not just a checklist; it’s a mindset embedded into team behaviours, values, and daily decisions. Creating such a culture requires more than technical training. It demands psychological safety, inclusivity, leadership accountability, and shared ownership of outcomes. When these ingredients are present, ethical governance becomes a natural extension of how people think, build, and lead.
Building a culture of responsible AI begins with empowering ethical curiosity across teams. Developers, analysts, and product staff must feel psychologically safe to question AI outputs, flag concerns, and challenge embedded assumptions. In my own teams, regular “ethical risk checks” during sprints sparked key design changes that no documentation would have caught. But curiosity alone isn’t enough. Organisations must foster ethical fluency: practical confidence to handle bias, data gaps, and automation risk. Through hands-on workshops, I’ve guided cross functional teams in co-creating “AI Impact Canvases” to visualise risks across user journeys.
Inclusive AI cannot be built by monocultures. Diverse, mentored teams see edge cases others miss. As a STEM Ambassador and Mental Health First Aider, I’ve supported underrepresented professionals and nurtured empathy as a risk mitigator.
Ethical leadership starts with modelling reflection sharing trade-offs, pausing for fairness, and acting on frontline feedback. When one AI allocator revealed bias in shift assignments, we paused deployment, reworked the logic, and built trust. Ultimately, ethical governance must be embedded in a learning culture: where retrospectives, open dialogue, and communities of practice make compliance a foundation not the ceiling.
Capgemini’s Global AI Ethics Report (2023) shows that only 27% of firms have AI specific ethical guidelines. Of those, fewer than half actively train product and delivery teams. And just 18% provide governance training for product, engineering, or business delivery teams (Capgemini AI Ethics Report, 2023).
End: Governance is Strategy
AI will only accelerate. The question is not whether we use it; but how we use it.
Business Analysts and Product Owners have a unique vantage point: they sit at the intersection of people, policy, and product. They are not just delivery agents — they are ethical stewards of innovation. To deliver value in the AI era, we must go beyond functionality. We must deliver trust, transparency, and accountability by design. That’s what human-centred governance means. And that’s how we lead the future of AI responsibly.
About the Author
Promise Akwaowo is a CBAP certified UK-based Business Analyst, with a proven record in AI-led delivery transformation across logistics, education, Construction and financial services sectors. He is a STEM Ambassador, certified Mental Health First Aider, and an active mentor with IIBA and a contributor to the AI and Data Interest Network of the Association for Project Management (APM UK). He has contributed to major regulatory and automation initiatives and programmes, and he is passionate about ethical, human-centred innovation in AI and digital product management.