diff --git a/content/firmitas/01-preface.md b/content/firmitas/01-preface.md new file mode 100644 index 0000000..93d4c68 --- /dev/null +++ b/content/firmitas/01-preface.md @@ -0,0 +1,73 @@ +--- +title: Preface +--- + +# Preface + +I have spent the better part of twenty-five years building things. Embedded systems in defence programmes. IoT platforms at scale. Telecoms infrastructure serving millions of customers. Products that had to work in the field, under conditions nobody fully anticipated, built by teams who were asked to commit to dates before they understood the problem and held accountable for outcomes they were never given the authority to shape. + +In that time I have watched capable engineers fail to deliver. Not because they lacked skill. Not because they weren't trying. But because the systems surrounding them were designed - often without anyone noticing - to produce exactly the outcomes they produced. + +I have watched risk registers get filed instead of acted on. I have watched honest estimates get replaced by commercially convenient ones. I have watched the gap between what engineers knew and what decision-makers believed grow into the space where programmes fail. And I have watched the blame for those failures travel downward, landing on the people who saw it coming and said so. + +This book exists because that pattern is not inevitable. It is structural. And structural problems have structural solutions. + +--- + +Firmitas is not an argument for a new methodology. There are enough of those. It is an argument for understanding why engineering programmes fail before reaching for the tools that are supposed to prevent it. The tools are often fine. The understanding is frequently absent. + +The word comes from Vitruvius - the Roman architect who argued that any structure worth building required *firmitas*, *utilitas*, and *venustas*. Solidity, usefulness, and beauty. Firmitas came first. Not because beauty does not matter, but because without structural integrity nothing else can stand. + +That sequence is the thesis of this book. Before methodology. Before process. Before governance frameworks and delivery models and maturity assessments - you need to understand the system you are working in, who owns what within it, and whether the people making the decisions that shape delivery have the information and the authority to make them well. + +Most organisations have the sequence backwards. They install process and hope that understanding follows. It rarely does. + +--- + +This book is written for the practitioner who has been on the programme that failed predictably. The engineer who knew the risk register was being filed rather than managed. The architect who watched an honest estimate get replaced by a commercially convenient one. The delivery lead who raised the issue at month two and was overridden, and who was then held accountable for the failure at month eight. + +It is also written for the leader who made those decisions and genuinely did not understand the cost. + +Not because leaders are the villains in this story - they are not. The leaders who override engineering judgement, who rubber-stamp gate reviews, who set dates before scope is understood, are almost always doing what their system has taught them to do. They are responding rationally to the incentives around them. Changing those incentives, and changing the systems that produce them, is what this book is about. + +If you are looking for a process to install, this book will disappoint you. Firmitas is not a process. It is a philosophy that constrains a framework, which in turn constrains the process choices available within it. The distinction matters because it is precisely the inversion of that sequence - process first, philosophy never - that produces the failure patterns this book is designed to prevent. + +If you are willing to examine the system you operate in honestly - including your own role within it - this book may be useful. + +--- + +A note on what this book is not for. + +Firmitas is not for every organisation. A small team building a product quickly without regulatory obligations does not need this framework. The overhead is disproportionate and the philosophy, though sound, will be applied at a scale that produces bureaucracy rather than clarity. + +It is also not for organisations where the primary obstacle is not systemic but political - where the people with power benefit from the current dysfunction and have no incentive to change it. Firmitas can help a willing organisation build better systems. It cannot make an unwilling one want to. + +And it is not a silver bullet for any organisation. Applied sincerely by a capable leader in a receptive organisation, it will improve delivery. Applied as a process without the philosophy behind it, it will produce the same outcome as every other framework adopted without understanding - compliance theatre, improved metrics, and the same underlying failures arriving slightly later and slightly harder to diagnose. + +The honest limits of this framework are addressed directly in Part Three. A framework that cannot tell you when it is wrong is not a framework. It is a belief system. + +--- + +The structure of this book follows the logic of the argument. + +Part One diagnoses the failure. It describes the specific structural mechanisms by which capable engineering organisations consistently fail to deliver, and why those failures are misattributed in ways that make the next failure more likely. If you are sceptical that your organisation has a structural problem - if you believe the issue is execution, competence, or commitment - Part One is written for you. Read it with the last programme that failed in mind. + +Part Two presents the framework. Every element is introduced as a direct response to a specific failure mechanism identified in Part One. The framework does not exist independently of the diagnosis. It is the structural answer to a structural question. + +Part Three addresses adoption, sustainability, and honest limits. Including the limits of the framework itself, the conditions under which it fails, and the specific warning signs that it is being misused. + +--- + +One final observation before the book begins. + +The most important thing I have learned across twenty-five years of building things is that the people closest to the work almost always know what is wrong. They know which risks are real and which are filed for appearances. They know which estimates are honest and which were produced under pressure to say something more comfortable. They know when the architecture will not hold and when the plan was never achievable. + +The question is never whether that knowledge exists. It is whether the system allows it to reach the people who need to act on it. + +Building that system - designing the conditions under which what engineers know can influence what leaders decide - is the work this book is about. + +It is harder than installing a process. It requires more of leaders than compliance with a framework. But it is the only thing that has ever actually worked. + +--- + +*Mark Bacon, April 2026* diff --git a/content/firmitas/02-acknowledgements.md b/content/firmitas/02-acknowledgements.md new file mode 100644 index 0000000..756af4d --- /dev/null +++ b/content/firmitas/02-acknowledgements.md @@ -0,0 +1,59 @@ +--- +title: Acknowledgements +--- + +# Acknowledgements + +The ideas in this book are not new. + +That is not a disclaimer. It is the point. + +Frederick Brooks described the essential nature of software complexity in 1975. He showed why adding people to a late project makes it later, why there is no silver bullet, and why the optimism that drives engineering programmes is also the primary source of their failure. Fifty years later, programmes are still being staffed up when they are late, silver bullets are still being purchased, and optimism is still being mistaken for a plan. + +Tom DeMarco spent a career articulating why engineering organisations fail - not through incompetence but through the systematic elimination of the conditions that make good work possible. Across his full body of work - from his early writing on structured analysis, through *Peopleware* (with Timothy Lister), *The Deadline*, *Slack*, and *Waltzing with Bears* - he returned repeatedly to the same hard truths: that schedule pressure destroys the conditions for quality, that risk is managed by naming it rather than optimising it away, and that the human cost of permanently urgent systems is real, measurable, and routinely ignored. His work on estimation and risk alone should be required reading for every programme manager who has ever asked a team to commit to a date before the scope was understood. + +Timothy Lister, DeMarco's long-time collaborator and a formidable thinker in his own right, brought the same clarity to the human dimensions of engineering work - the teams, the environment, the trust, and the conditions under which people either do their best work or learn to protect themselves from the consequences of doing otherwise. His writing on risk management, on the sociology of engineering teams, and on what actually determines whether projects succeed deserves more recognition than it typically receives. + +Dan Pink demonstrated that the conditions for sustained motivation - purpose, autonomy, mastery - are not soft concerns. They are structural prerequisites. Remove any of them and performance degrades regardless of how hard people try. Organisations that claim to value their engineers while systematically undermining those three conditions are not cruel. They are structurally dishonest. + +Simon Sinek articulated why purpose precedes everything else - why organisations that cannot answer why they exist will always struggle to align what they do. His observation that people do not buy what you do, they buy why you do it applies as much inside engineering organisations as it does in markets. Leaders Eat Last extended that argument into the conditions of trust and psychological safety that determine whether teams operate out of confidence or fear. + +Steve McConnell gave the software industry its most rigorous treatment of estimation, planning, and the cost of shortcuts. *Software Estimation* alone dismantles the assumptions behind most programme planning - the optimism bias, the estimation by analogy, the political pressure that converts an honest range into a single committed date. *Code Complete* and *Rapid Development* remain essential references for anyone serious about the craft and management of software. His work is notable for its refusal to separate technical excellence from delivery discipline. + +Watts Humphrey brought the same rigour to process discipline and the personal and team dimensions of software practice. His insight that quality is built in through disciplined individual and collective practice - not inspected in at the end - was decades ahead of its mainstream acceptance. The Personal Software Process and Team Software Process work demonstrated empirically that estimation accuracy and quality outcomes improve dramatically when engineers are given the tools and the trust to manage their own work rigorously. + +Johanna Rothman has spent decades writing practically and honestly about the reality of managing software projects - the estimation traps, the scheduling failures, the team dynamics, and the specific decisions that distinguish programmes that deliver from those that don't. Her work is grounded in the messy reality of actual programmes in ways that more theoretical frameworks are not. She writes for practitioners who have to make decisions under uncertainty, and she does not pretend those decisions are easy. + +Joel Spolsky's writing on software management, hiring, and the conditions that make engineering teams effective remains some of the sharpest practitioner thinking available. His Joel Test - twelve questions that reveal more about a development organisation's health in five minutes than a maturity assessment reveals in a week - was a direct inspiration for the Firmitas Delivery Diagnostic in this book. His willingness to state uncomfortable truths plainly, without the hedging that characterises most management writing, set a standard this book attempts to follow. + +Gerald Weinberg spent a lifetime examining the human and organisational dimensions of software work - the psychology of programming, the politics of quality, the systems dynamics that produce the outcomes organisations claim not to want. His observation that every organisation is perfectly designed to get the results it gets is one of the most important sentences ever written about organisational behaviour. His *Quality Software Management* series, *The Psychology of Computer Programming*, and *Becoming a Technical Leader* together constitute an indispensable library for anyone serious about the human side of engineering. + +Donald Reinertsen brought the economics of product development flow into sharp focus - the cost of delay, the damage done by large batches, the mathematics of queue theory applied to the reality of engineering programmes. *The Principles of Product Development Flow* is a book that rewards careful reading and re-reading. His work on flow efficiency over resource efficiency underpins much of what this book says about utilisation, slack, and the myth of the fully loaded team. The economic argument for slack - that maximising utilisation destroys throughput - is not intuitive, but once understood it is impossible to unsee. + +Gene Kim, Kevin Behr, and George Spafford made the same arguments accessible to a generation of technology leaders through *The Phoenix Project* - demonstrating through narrative what Reinertsen demonstrated through economics, and reaching an audience that might not have engaged with the underlying theory. + +Matthew Skelton and Manuel Pais gave the industry a precise vocabulary for team topology and cognitive load - the language of stream-aligned teams, platform teams, enabling teams, and the interaction modes between them. Their work on Conway's Law as a design constraint rather than a warning is directly reflected in how this book treats team structure and organisational design. + +W. Edwards Deming understood quality as a system property before most of the software industry was born. His insistence that quality problems are management problems - not worker problems - and that the system produces the outcomes it is designed to produce is the foundational insight from which much of this book's argument flows. His fourteen points and his red bead experiment remain as relevant to engineering programme management as they are to manufacturing. + +Peter Senge made systems thinking accessible and practical. *The Fifth Discipline* gave a generation of leaders the language to describe the feedback loops, delays, and unintended consequences that make organisational change so difficult and so necessary. The concept of the learning organisation - one designed to surface and act on its own failure patterns - is central to everything Firmitas proposes about how programmes should be governed. + +Steve Tockey brought rigorous engineering economics to software decision-making - the argument in *Return on Software* that engineering decisions are always economic decisions, whether acknowledged or not, and that treating them as purely technical produces the same quality of outcome as treating financial decisions as purely aesthetic. His work on the cost and value dimensions of software engineering choices is underread and underappreciated. + +Michael Lopp wrote about the human reality of engineering management with honesty and precision - the conversations that do not happen, the signals that get missed, the specific moments where leadership either earns or loses the trust of the people it is responsible for. *Managing Humans* is the book that many engineering managers wish they had been given before they started, and his ongoing writing continues to name the things that most management literature is too polished to say plainly. + +--- + +I owe an intellectual debt to all of these writers and practitioners. Their work shaped the thinking in this book in ways that are sometimes explicit and sometimes so thoroughly absorbed that the original source is hard to separate from the argument. + +The reason this book exists despite that body of work is simple and uncomfortable: the lessons are still not being learned. + +Not because the books are wrong. Not because the principles are unsound. But because knowledge does not change systems. A manager who reads *Peopleware* and then returns to a programme governed by utilisation targets, rubber-stamped gates, and reverse-engineered plans has not been equipped with what they need to change those things. The knowledge exists. The structural response to it does not. + +Firmitas is an attempt to build that structural response - to take the lessons that have been available for decades and embed them in a framework that addresses the organisational conditions that prevent them from being applied. It does not supersede the works listed above. It stands on them and asks why, fifty years after Brooks first described the problems, we are still producing programmes that fail in exactly the ways he predicted. + +The answer is systemic. So is the solution. + +--- + +*The Further Reading section at the end of this book provides a curated guide to the works referenced here, with specific recommendations on which to read first and why.* diff --git a/content/firmitas/03-introduction.md b/content/firmitas/03-introduction.md new file mode 100644 index 0000000..e76f31d --- /dev/null +++ b/content/firmitas/03-introduction.md @@ -0,0 +1,81 @@ +--- +title: Introduction +--- + +# Introduction + +This book is built around a single observation. + +The people who hold the most accurate information about what an engineering programme requires are rarely the people who make the decisions that determine whether it succeeds. And the people who make those decisions rarely bear the full cost when they go wrong. + +That observation is not new. Its consequences - the committed dates that were never achievable, the risk registers that were reviewed and filed, the failures attributed to execution rather than governance, the lessons drawn that make the next failure more likely - have been documented by writers and practitioners going back decades. The knowledge has existed for a long time. + +What has not existed, or not existed clearly enough, is a framework that addresses the structural conditions that prevent the knowledge from being applied. That is what this book attempts to provide. + +--- + +**What this book argues** + +Engineering programmes fail for a reason that is rarely named honestly. The people who make delivery commitments are not the people who have to keep them. When programmes fail - and they fail predictably, in ways engineers foresaw and flagged - the accountability travels downward onto people who never made the original commitment. The organisation draws the wrong lesson. The next programme gets tighter control, less trust, and fails faster. + +This cycle is not inevitable. It is structural. And structural problems have structural solutions. + +Firmitas is a philosophy-led framework for engineering delivery. It is built on the conviction that sustainable engineering performance requires three things to be true simultaneously: that delivery knowledge is respected and acted on, that commitments are owned by the people who make them, and that the systems surrounding engineering teams are designed to make good outcomes possible rather than merely to measure the absence of them. + +The framework is not a process to be installed. It is a set of structural positions - about where authority should sit, how risk information should be treated, what governance is actually for, how estimates should be made and used - that together address the conditions that produce the failure pattern this book describes. + +--- + +**How the book is structured** + +The book is in three parts. + +**Part One: Why Programmes Fail** makes the diagnosis. It opens with a composite programme - drawn from patterns observed across many real programmes in defence, industrial systems, IoT, and telecoms - and follows it through the sequence that will be familiar to anyone who has been on a programme that failed predictably. An honest assessment produced at week three. A risk register with three red items. A programme board that noted the risks and moved to task status. A window that closed. An outcome that nobody wanted and that everyone involved had enough information to prevent. + +The subsequent chapters of Part One explain why that sequence is so consistent. Chapter 2 examines the sandbag assumption - the belief that engineering estimates are padded and that pressure will extract the real number - and shows why it is almost always wrong and always damaging. Chapter 3 examines the committed date - where it comes from, why it is so rarely made by the people who have to keep it, and what the commercial and contractual reality means for how the gap between commitment and honest estimate should be managed. Chapter 4 examines the six interacting systems that together produced the failure. Chapter 5 examines the learning cycle - why the lessons drawn from each failure compound the conditions that produce the next one. + +Part One ends with Chapter 6, which introduces the systems thinking foundation and the twelve principles that form the philosophy layer of the framework. These principles are not values. They are structural positions earned by what the preceding five chapters described. + +**Part Two: The Framework** presents the structural response. Every chapter opens with a connection to the specific failure mechanism it addresses. The framework is not presented as best practice - it is presented as the direct answer to a named problem. Leadership and governance, commitments and estimation, risk and issues management, requirements engineering, the delivery model, quality, teams, communication, technical debt, capability, portfolio management, AI in regulated delivery - each element is grounded in the diagnosis that preceded it. + +Part Two is the majority of the book. It is where the practical guidance lives. Readers who want to move directly to a specific element - the risk register treatment, the estimation approach, the delivery model - can do so. The connections back to Part One are there for context, not as a requirement for understanding any individual chapter. + +**Part Three: Making It Stick** addresses adoption, sustainability, and the honest limits of the framework. It includes the Firmitas Delivery Diagnostic - twenty-six questions that assess how the conditions in a real programme compare to what the framework requires. It addresses the leader who recognises their own behaviour in what the book describes, the common failure modes in adoption, what the framework cannot fix, and what sustainable delivery looks like over the long term. + +--- + +**How to read this book** + +Different readers will come to this book from different positions. The following is a guide to where to start based on where you are. + +**If you are sceptical that your organisation has a structural problem** - if you believe the issue is execution competence, team capability, or insufficient process discipline - start at Chapter 1 and read Part One in full before engaging with Part Two. Part One is written for you. It makes the structural argument from evidence rather than assertion and asks you to read it with your last failed programme in mind. + +**If you are already convinced that the problem is structural but want the practical framework** - you can start at Part Two after reading Chapter 6, which provides the principles and the layered model that frame everything that follows. Part One will still repay reading, but Part Two can stand without it. + +**If you are a senior leader who governs engineering programmes** - Chapters 3, 8, 9, 10, and 24 are the most directly relevant to the decisions you make and the conditions you create. The diagnostic in Appendix B will give you a rapid assessment of your current programme against the framework's requirements. + +**If you are an engineer or architect who has experienced the failure described in Part One** - Part One will confirm what you already know. Part Two will give you the language and the structural argument to advocate for the conditions that would have changed the outcome. Chapter 24, in particular, is addressed to the leaders around you rather than to you directly - but it is useful to understand what the framework asks of them. + +**If you want to assess your organisation before reading in full** - the Firmitas Delivery Diagnostic in Appendix B is designed to be used independently. Twenty-six yes-or-no questions with evidence checks. The score and the clustering of your answers will tell you which parts of the framework are most urgent for your context. + +--- + +**What this book is not** + +It is not a methodology. There is no prescribed sprint length, no mandatory ceremony schedule, no certification path. The process layer of Firmitas is intentionally variable - it adapts to context. What does not vary is the philosophy. + +It is not a universal solution. The honest limits of the framework are addressed directly in Chapter 28. There are organisational contexts - the very small startup, the highly compartmentalised security programme, the large multinational where decision authority is structurally separated from delivery knowledge by design - where Firmitas either does not apply or can only be applied partially. A framework that cannot tell you when it is the wrong tool is not a framework. It is a belief system. + +It is not a criticism of any individual or profession. The commercial director who committed to the unachievable date, the programme board member who noted the risk register and moved on, the project manager who built the reverse-engineered plan - none of these people are the villains of this story. They were responding rationally to the systems they were in. Changing those systems is what the book is about. + +--- + +**A note on the word Firmitas** + +Vitruvius, writing in the first century BCE on the principles of good architecture, argued that any structure worth building required three properties: *firmitas*, *utilitas*, and *venustas* - solidity, usefulness, and beauty. + +Firmitas came first. Not because the other two do not matter, but because without structural integrity nothing else can stand. A beautiful building that cannot bear its own weight is not a building. A useful process built on a system that undermines it is not a solution. + +The name reflects the thesis. Before methodology. Before process. Before governance frameworks and delivery models and maturity assessments - the structure must be sound. The conditions under which engineering work happens must be designed to make good outcomes possible, not merely to measure whether they occurred. + +Structural integrity in engineering organisations means the same thing it means in physical structures. Not rigidity. Not permanence. The capacity to bear load without failing - and to flex under pressure without losing the properties that make the whole thing stand. diff --git a/content/firmitas/appendices/a-glossary-of-terms.md b/content/firmitas/appendices/a-glossary-of-terms.md new file mode 100644 index 0000000..75a4d74 --- /dev/null +++ b/content/firmitas/appendices/a-glossary-of-terms.md @@ -0,0 +1,226 @@ +--- +title: "Appendix A: Glossary of Terms" +--- + +# Appendix A: Glossary of Terms + +This glossary defines how key terms are used **specifically within Firmitas**. +Many of these words are commonly used in engineering and management, but Firmitas applies them with **deliberate, precise meaning**. Where Firmitas’s usage differs from casual or industry-ambiguous definitions, that distinction is made explicit. + +This appendix is normative. +When a term appears elsewhere in the book, it carries the meaning defined here. + + + +## Architecture +The structural design of technical systems, including software, hardware, platforms, infrastructure, interfaces, and integration boundaries. +In Firmitas, architecture is not an abstract technical artefact but a **delivery-enabling system** that encodes assumptions about change, ownership, coordination, and risk. Architecture directly shapes flow, autonomy, and long-term system health. + + + +## Architectural Guardrails +Explicit constraints, principles, and boundaries that guide architectural decision-making without prescribing specific solutions. +Guardrails enable decentralised decision-making while preserving coherence and reducing systemic risk. They are a mechanism for **lightweight governance**, not control. + + + +## Blame +The attribution of failure to individuals rather than to system design, incentives, or constraints. +Firmitas treats blame as a **learning inhibitor** that suppresses information, delays risk discovery, and degrades system health. + + + +## Cognitive Load +The total amount of mental effort required to perform work within a system. +High cognitive load reduces decision quality, slows learning, and increases error rates. Firmitas treats cognitive load as a **design constraint** influenced by architecture, team structure, tooling, and governance. + + + +## Context +The shared understanding of purpose, constraints, priorities, risks, and boundaries within which teams operate. +Context enables autonomy by allowing teams to make good decisions without constant escalation. Lack of context produces defensive behaviour and local optimisation. + + + +## Delivery System +The socio-technical system that builds and delivers the **Product System**; the "machine that builds". +It comprises the teams, processes, and culture that turn ideas into value. Firmitas treats the Delivery System as an engineered system that must be designed as rigorously as the code. + + + +## Decision Latency +The time taken for a decision to be made and acted upon once it is needed. +Decision latency is a key signal of organisational health. Excessive latency indicates unclear authority, over-governance, or fear-based systems. + + + +## Decision Record (ADR) +A lightweight, durable record of a significant decision, including its context, rationale, trade-offs, and consequences. +ADRs exist to preserve **system memory**, not to create documentation overhead. + + + +## Empowered Autonomy +The ability for teams to make decisions independently **within clearly defined boundaries**. +Empowered autonomy is neither uncontrolled freedom nor delegated responsibility without authority. It is autonomy enabled by context and constrained by shared intent. + + + +## Engineering Function +The organisational capability responsible for creating value through engineering activity. +In Firmitas, the engineering function exists to **deliver value to customers and, in doing so, create value for the business — sustainably and repeatedly**. + + + +## Engineering Leadership +The non-delegable responsibility for designing, aligning, governing, and evolving the systems that produce engineering outcomes. +Engineering leadership is a systems discipline, not a role, title, or set of ceremonies. + + + +## Feedback Loop +A mechanism by which the outcomes of decisions or actions influence future decisions. +Fast, honest feedback is essential for learning. Delayed feedback amplifies risk and cost. + + + +## Flow +The rate at which value moves through the system from idea to customer. +Flow is distinct from activity or speed. Good flow is predictable, low-variance, and enables rapid learning. + + + +## Framework +The structured set of decision logic, governance mechanisms, and boundaries that translate philosophy into repeatable organisational behaviour. +In Firmitas, the framework sits **between philosophy and process** and constrains both. + + + +## Governance +The mechanisms by which decisions are guided, constrained, reviewed, and escalated. +Firmitas defines governance as **decision support**, not decision suppression. Its purpose is to improve decision quality while enabling autonomy. + + + +## Human Systems +Systems involving people, including motivation, trust, incentives, psychological safety, communication patterns, and decision-making dynamics. +Firmitas assumes people behave rationally within the systems they are placed in. + + + +## Learning System +A system designed to detect signals, generate feedback, and adapt behaviour over time. +Delivery is treated as a learning system, not merely an execution pipeline. + + + +## Local Optimisation +Improving performance in one part of a system at the expense of overall system health. +Firmitas treats local optimisation as a primary source of systemic failure. + + + +## Outcome +A meaningful, observable change in the world that creates value for customers or the business. +Outcomes are distinct from outputs, which are intermediate artefacts such as features or documents. + + + +## Output +An artefact produced by engineering activity, such as code, designs, or documentation. +Outputs only matter insofar as they lead to outcomes. + + + +## Philosophy +The explicit set of beliefs, values, and constraints that define what matters and why. +In Firmitas, philosophy constrains the framework, which in turn constrains process. + + + +## Process +The specific practices, tools, workflows, and ceremonies used to execute work in a given context. +Processes are intentionally flexible and context-dependent, provided they align with the framework and philosophy. + + + +## Product System +The technical asset being built to deliver value and generate a return on investment (ROI); the "asset being built". +This is the software, hardware, or service that customers use. It is the output of the **Delivery System**. + + + +## Psychological Safety +A system property in which individuals can speak up, admit uncertainty, report failure, and challenge assumptions without disproportionate personal risk. +Psychological safety is a prerequisite for learning, not a lack of accountability. + + + +## Queue +Any point at which work waits rather than progresses. +Queues increase lead time, hide risk, and delay feedback. They are a primary cost centre in delivery systems. + + + +## Slack +Deliberately unallocated capacity within a system that enables learning, recovery, improvement, and adaptation. +Slack is not waste. It is a **structural requirement** for resilience and long-term performance. + + + +## Socio-Technical System +The interaction between people and technology, including tooling, automation, documentation, CI/CD, observability, and operational workflows. +Technical systems always encode assumptions about communication, authority, and responsibility. + + + +## Stewardship +Leadership responsibility exercised over time, focused on system health rather than short-term optimisation. +Stewardship prioritises sustainability, learning, and long-term value over immediate efficiency. + + + +## System +A set of interacting elements whose collective behaviour produces outcomes over time. +In Firmitas, systems include human, organisational, socio-technical, delivery, technical, and economic systems. + + + +## System Health +The ability of a system to deliver value predictably, adapt to change, and recover from failure without degrading over time. +System health is prioritised over short-term efficiency. + + + +## Technical Debt +The accumulated cost of decisions that make future change more difficult, expensive, or risky. +Firmitas treats technical debt as a **strategic and economic concern**, not merely a technical one. + + + +## Utilisation +The proportion of available capacity that is actively engaged in work. +Firmitas explicitly rejects utilisation maximisation as a primary management objective, as it degrades flow and resilience. + + + +## Value +A meaningful benefit realised by a customer or the business. +Value is contextual, outcome-based, and realised only when solutions perform effectively in real conditions. + + + +## Work in Progress (WIP) +All work that has been started but not yet completed. +High WIP increases queues, delays feedback, and reduces predictability. Firmitas treats WIP as a controllable system variable. + + + +## Whole-System Optimisation +The practice of improving overall system outcomes, even when it reduces apparent efficiency in individual components. +Firmitas consistently prioritises whole-system optimisation over local gains. + + + +This glossary is intentionally comprehensive. +Its purpose is not academic precision, but **shared understanding**, enabling consistent interpretation and application of the Firmitas philosophy across teams, roles, and contexts. diff --git a/content/firmitas/appendices/b-firmitas-templates.md b/content/firmitas/appendices/b-firmitas-templates.md new file mode 100644 index 0000000..9a35b11 --- /dev/null +++ b/content/firmitas/appendices/b-firmitas-templates.md @@ -0,0 +1,281 @@ +--- +title: "Appendix B: Firmitas Templates" +--- + +# Appendix B: Firmitas Templates + +This appendix provides a **canonical set of lightweight, pragmatic templates** used throughout Firmitas. +They are intentionally minimal, text-first, and compatible with a Docs-as-Code approach. + +These templates are **not mandatory artefacts**. +They exist to reduce cognitive load, improve consistency, and preserve decision context where it matters. + +Teams should tailor them, but **not dilute their intent**. + + + +## B.1 Vision Brief Template + +**Purpose:** +Establish shared understanding of *why* the product exists and *what problem it serves*. + +```markdown +# Vision Brief – + +## Purpose +Why does this product exist? +What customer problem or opportunity does it address? + +## Target Users / Customers +Who is this for? +What context do they operate in? + +## Value Proposition +What meaningful value does this deliver to the customer? +Why is this valuable to the business? + +## Success Criteria (Outcomes) +How will we know this is successful? +Focus on outcomes, not outputs. + +## Constraints +Known constraints (regulatory, technical, commercial, time). + +## Strategic Risks +Key risks that could invalidate the vision. + +## Non-Goals +Explicitly state what this product is *not* trying to do. +``` + + + +## B.2 Architecture Decision Record (ADR) Template + +**Purpose:** +Capture **why** a significant decision was made, not just what was chosen. + +```markdown +# ADR-: + +## Status +Proposed | Accepted | Superseded | Deprecated + +## Context +What problem are we solving? +What forces, constraints, or trade-offs exist? + +## Decision +What decision has been made? + +## Rationale +Why this option over alternatives? +What trade-offs were accepted? + +## Consequences +Positive and negative consequences. +What does this enable or constrain? + +## Alternatives Considered +Briefly list rejected options and why. + +## Follow-Ups +Actions, reviews, or revisit conditions. +``` + + + +## B.3 Request for Comment (RFC) / Design Proposal Template + +**Purpose:** +Enable **collaborative design thinking** before commitment. + +```markdown +# RFC: + +## Problem Statement +What problem are we trying to solve? +Why now? + +## Proposed Approach +High-level description of the proposal. + +## Impacted Systems +What systems, teams, or interfaces are affected? + +## Trade-Offs +What do we gain? +What do we give up? + +## Risks & Unknowns +Key uncertainties or assumptions. + +## Open Questions +What feedback or decisions are needed? + +## Decision Owner +Who will decide and by when? +``` + + + +## B.4 Requirements Template (System / Product Level) + +**Purpose:** +Define intent clearly without over-specification. + +```markdown +# Requirements – <System / Product Name> + +## Scope +What is in scope? +What is explicitly out of scope? + +## Functional Requirements +- FR-001: <Statement> +- FR-002: <Statement> + +## Non-Functional Requirements +Performance, reliability, security, safety, usability, etc. + +## Regulatory / Compliance Requirements +Standards, certifications, legal obligations. + +## Assumptions +Key assumptions that influence design. + +## Acceptance Criteria +How will these requirements be verified? +``` + + + +## B.5 Requirements Traceability Matrix (RTM) – Lightweight Form + +```markdown +| Requirement ID | Design Ref | Implementation Ref | Test Ref | Status | +|-||--|-|--| +| FR-001 | ADR-004 | Repo:module-x | TC-012 | Done | +``` + + + +## B.6 Risk Register Template + +```markdown +# Risk Register – <Product / Programme> + +| Risk ID | Description | Likelihood | Impact | Mitigation | Owner | Status | +|--|-||--||-|--| +| R-001 | Component lead time | High | High | Alternate supplier | Ops | Open | +``` + + + +## B.7 Incident Review / Postmortem Template + +```markdown +# Incident Review – <ID / Date> + +## Summary +What happened? + +## Impact +Who was affected and how? + +## Timeline +Key events in chronological order. + +## Root Causes (Systemic) +What system conditions enabled this? + +## What Went Well +What helped contain or resolve the issue? + +## What Didn’t Go Well +Where did the system fail? + +## Actions +Concrete improvements to prevent recurrence. + +## Learning Captured +What should we remember next time? +``` + + + +## B.8 C4 Architecture Diagram Index Template + +```markdown +# Architecture Overview – <System Name> + +## Context Diagram +Link or embed diagram. + +## Container Diagrams +- Container A – Purpose +- Container B – Purpose + +## Component Diagrams +Links to detailed views. + +## Key Architectural Principles +Non-negotiable design principles. + +## Known Constraints +Technical or organisational constraints. +``` + + + +## B.9 Delivery Health Check Template + +```markdown +# Delivery Health Check – <Team / Product> + +## Flow +Is work flowing predictably? + +## Feedback +How fast do we learn from customers and failures? + +## Quality +Are defects escaping into production? + +## Sustainability +Is pace sustainable? +Is slack present? + +## Risks +What keeps us awake at night? + +## Actions +What system adjustments are needed? +``` + + + +## B.10 Adoption & Tailoring Checklist + +```markdown +# Firmitas Adoption Checklist + +- Philosophy understood and agreed +- Leadership responsibilities explicit +- Guardrails defined +- Core artefacts selected +- Slack explicitly protected +- Feedback loops established +- Risk visibility in place +- Governance lightweight and intentional +``` + + + +## B.11 Template Usage Guidance + +- Templates are **starting points**, not contracts. +- Prefer clarity over completeness. +- Text-first beats tools-first. +- Review templates periodically and retire those that no longer add value. +- Treat templates as **system components**, subject to evolution. diff --git a/content/firmitas/appendices/c-architectural-principles-for-firmitas.md b/content/firmitas/appendices/c-architectural-principles-for-firmitas.md new file mode 100644 index 0000000..9aa25c8 --- /dev/null +++ b/content/firmitas/appendices/c-architectural-principles-for-firmitas.md @@ -0,0 +1,290 @@ +--- +title: "Appendix C: Architectural Principles for Firmitas" +--- + +# Appendix C: Architectural Principles for Firmitas + +This appendix defines a **pragmatic, principle-led set of architectural principles** aligned to the Firmitas philosophy. +They draw inspiration from established architectural thinking (including TOGAF-style principles) but are **explicitly adapted** for modern, multi-disciplinary product development rather than enterprise IT alone. + +These principles are intended to: +- Guide architectural decisions at all levels (system, solution, component) +- Act as **architectural guardrails**, not rigid rules +- Support autonomy while preserving coherence +- Optimise for long-term system health and value delivery + +They should be **few, durable, and enforceable through conversation**, not bureaucracy. + + + +## C.1 Principle Structure + +Each principle is expressed using a consistent structure: + +- **Statement** – The principle itself +- **Rationale** – Why this matters +- **Implications** – What this means in practice + + + +## C.2 Foundational Architectural Principles + +### AP-01: Architecture Exists to Enable Value Delivery + +**Statement** +Architecture must directly enable the delivery of customer and business value. + +**Rationale** +Architecture that optimises elegance, theoretical purity, or control at the expense of delivery speed and learning is counterproductive. Architecture is a means, not an end. + +**Implications** +- Architectural work must be justified in terms of outcomes +- Architectural decisions should shorten feedback loops, not lengthen them +- “Nice to have” architectural complexity should be actively challenged + + + +### AP-02: Optimise for Whole-System Health + +**Statement** +Architectural decisions must optimise the health of the entire system, not individual components. + +**Rationale** +Local optimisation creates systemic fragility, hidden coupling, and long-term delivery degradation. + +**Implications** +- Trade-offs are evaluated across human, organisational, delivery, and technical systems +- Component efficiency may be sacrificed for system resilience +- Architectural reviews consider downstream operational impact + + + +### AP-03: Architecture Must Assume Change + +**Statement** +All architectures must be designed with the expectation of change. + +**Rationale** +Requirements, technologies, markets, and constraints will evolve. Architectures that assume stability accumulate risk rapidly. + +**Implications** +- Prefer modularity, clear interfaces, and replaceable components +- Avoid irreversible early decisions where possible +- Explicitly document assumptions and revisit them + + + +### AP-04: Decisions Are More Important Than Diagrams + +**Statement** +The quality and clarity of architectural decisions matter more than the completeness of architectural documentation. + +**Rationale** +Unrecorded decisions decay organisational memory and force teams to re-learn context through failure. + +**Implications** +- Use ADRs to capture key decisions and trade-offs +- Diagrams support understanding but do not replace decision rationale +- Architecture is judged by decision quality, not artefact volume + + + +## C.3 Human and Organisational Alignment Principles + +### AP-05: Architecture Must Reduce Cognitive Load + +**Statement** +Architecture should minimise unnecessary cognitive load on teams. + +**Rationale** +High cognitive load degrades decision quality, increases error rates, and slows learning. + +**Implications** +- Prefer simple, consistent patterns +- Avoid unnecessary technology diversity +- Make ownership and responsibilities explicit + + + +### AP-06: Conway’s Law Is a Design Constraint + +**Statement** +Architecture must acknowledge and deliberately respond to organisational communication structures. + +**Rationale** +Ignoring Conway’s Law leads to brittle architectures that conflict with how teams actually work. + +**Implications** +- Team boundaries should align with architectural boundaries +- Architectural refactoring may require organisational change +- Temporary misalignment should be explicit and managed + + + +### AP-07: Authority Must Match Responsibility + +**Statement** +Teams responsible for systems must have sufficient architectural decision authority. + +**Rationale** +Responsibility without authority creates bottlenecks, delays, and disengagement. + +**Implications** +- Architectural guardrails replace centralised approval +- Escalation paths are explicit and rare +- Decisions are made as close to the work as possible + + + +## C.4 Delivery and Evolution Principles + +### AP-08: Enable Fast, Honest Feedback + +**Statement** +Architecture must support rapid feedback from real usage and failure. + +**Rationale** +Feedback is the primary mechanism for learning and improvement in complex systems. + +**Implications** +- Observability is a first-class architectural concern +- Failure modes are designed, not ignored +- Testability and diagnosability influence design choices + + + +### AP-09: Build for Incremental Validation + +**Statement** +Systems should be architected so that value and risk can be validated incrementally. + +**Rationale** +Late validation amplifies cost, risk, and rework. + +**Implications** +- Support partial integration and staged releases +- Avoid designs that require “big bang” validation +- Enable prototyping and early system integration + + + +### AP-10: Slack Is a Structural Requirement + +**Statement** +Architectures must allow slack for learning, recovery, and improvement. + +**Rationale** +Systems designed for maximum utilisation are brittle and fail catastrophically under stress. + +**Implications** +- Capacity headroom is planned, not accidental +- Performance targets include resilience margins +- Teams are not architecturally forced into constant firefighting + + + +## C.5 Governance and Risk Principles + +### AP-11: Governance Through Principles, Not Permission + +**Statement** +Architectural governance should guide decisions rather than approve them. + +**Rationale** +Permission-based governance increases latency and suppresses learning. + +**Implications** +- Principles are explicit and widely understood +- Exceptions are discussed, not hidden +- Governance focuses on outcomes and risks + + + +### AP-12: Make Risk Explicit and Visible + +**Statement** +Architectural risk must be made visible and actively managed. + +**Rationale** +Hidden risk accumulates interest and surfaces late, when options are limited. + +**Implications** +- High-risk decisions are addressed early +- Risk trade-offs are documented +- Technical debt is tracked as an economic concern + + + +### AP-13: Security and Safety Are Architectural Properties + +**Statement** +Security and safety must be designed into the architecture, not added later. + +**Rationale** +Retrofitting safety or security is expensive and unreliable. + +**Implications** +- Threat modelling and hazard analysis inform design +- Compliance requirements influence architectural structure +- Safety-critical paths are isolated and protected + + + +## C.6 Technology and Platform Principles + +### AP-14: Prefer Proven Before Novel + +**Statement** +Prefer proven technologies unless novelty delivers clear, justified value. + +**Rationale** +Novelty increases uncertainty, learning cost, and risk. + +**Implications** +- New technology adoption requires explicit rationale +- Experiments are isolated and time-bound +- Exit strategies are defined early + + + +### AP-15: Minimise Irreversible Decisions + +**Statement** +Architectures should delay irreversible decisions until sufficient information is available. + +**Rationale** +Early irreversible commitments amplify uncertainty and lock in poor assumptions. + +**Implications** +- Use abstraction where it buys optionality +- Avoid premature standardisation +- Revisit early decisions intentionally + + + +### AP-16: Platforms Exist to Enable Teams + +**Statement** +Platforms exist to reduce cognitive load and accelerate delivery for stream-aligned teams. + +**Rationale** +Platforms that impose friction or control undermine their own purpose. + +**Implications** +- Platform teams operate as internal service providers +- Platform success is measured by team enablement +- Forced adoption without value is avoided + + + +## C.7 Applying These Principles + +These principles: +- Are **context-sensitive**, not absolute +- May occasionally conflict, requiring judgement +- Should be revisited as the organisation evolves + +Their purpose is not compliance, but **better decisions, faster learning, and healthier systems**. + +Architecture under Firmitas is a leadership responsibility. +These principles exist to make that responsibility explicit, shared, and actionable. diff --git a/content/firmitas/appendices/d-minimal-balanced-and-regulated-firmitas-profiles.md b/content/firmitas/appendices/d-minimal-balanced-and-regulated-firmitas-profiles.md new file mode 100644 index 0000000..d844dc7 --- /dev/null +++ b/content/firmitas/appendices/d-minimal-balanced-and-regulated-firmitas-profiles.md @@ -0,0 +1,322 @@ +--- +title: "Appendix D: Minimal, Balanced, and Regulated Firmitas Profiles" +--- + +# Appendix D: Minimal, Balanced, and Regulated Firmitas Profiles + +This appendix defines **three canonical Firmitas profiles**. +They are not maturity levels and not prescriptive methodologies. +They are **intentional configurations** of the same philosophy and framework, tuned to different organisational, product, and regulatory contexts. + +Firmitas is deliberately designed to scale **both up and down** without changing its core principles. + +The profiles differ in: +- Degree of formality +- Depth of artefacts +- Governance intensity +- Validation rigor +- Regulatory alignment + +They do **not** differ in philosophy. + + + +## D.1 Why Profiles Exist + +A common failure mode in engineering frameworks is assuming one “correct” level of process. + +Firmitas explicitly rejects this. + +Instead, it recognises that: +- Over-structure kills speed and learning in small systems +- Under-structure kills safety, trust, and predictability in complex or regulated systems +- The *same organisation* may need different profiles for different products + +Profiles exist to: +- Reduce cognitive overhead when tailoring +- Prevent accidental over- or under-engineering +- Make trade-offs explicit rather than implicit + + + +## D.2 Profile Overview + +| Dimension | Minimal | Balanced | Regulated | +||--|-|--| +| Primary Goal | Speed & learning | Sustainable delivery | Safety, assurance & compliance | +| Typical Context | Startups, internal tools | Scale-ups, IoT, SaaS | Medical, automotive, defence | +| Governance | Principles only | Lightweight gates | Formal gates & reviews | +| Documentation | Minimal viable | Living, structured | Comprehensive & auditable | +| Validation | Continuous, informal | Continuous + formal | Formal V&V + audits | +| Traceability | Implicit | Selective | Mandatory | +| Slack | Explicit but informal | Explicit & planned | Explicit & governed | + + + +## D.3 Minimal Firmitas Profile + +### D.3.1 Intended Context + +Minimal Firmitas is appropriate where: +- Teams are small and highly collaborative +- Risk of harm is low +- Regulatory burden is minimal +- Speed of learning outweighs predictability + +Typical examples: +- Early-stage startups +- Internal business tools +- Early product discovery phases +- Prototypes and pilots + + + +### D.3.2 Core Characteristics + +Minimal Firmitas emphasises: +- Fast feedback +- Low ceremony +- High trust +- Direct communication + +Governance exists primarily through **shared understanding**, not artefacts. + + + +### D.3.3 Artefact Set (Minimal) + +Required: +- Vision Brief (1–2 pages) +- Lightweight backlog +- ADRs for irreversible decisions only + +Optional: +- High-level architecture diagram +- Risk list (informal) +- Delivery health check (occasional) + +Explicitly avoided: +- Heavy traceability matrices +- Formal stage gates +- Detailed upfront specifications + + + +### D.3.4 Validation & Quality + +- Continuous testing embedded in development +- Informal acceptance based on outcomes +- Peer review over formal sign-off +- Quality is owned by the team, not a function + + + +### D.3.5 Risks and Failure Modes + +Common risks: +- Accidental architectural drift +- Hidden technical debt +- Implicit decisions becoming permanent + +Mitigations: +- Regular architectural conversations +- Periodic “decision archaeology” +- Explicit slack for refactoring and learning + + + +## D.4 Balanced Firmitas Profile + +### D.4.1 Intended Context + +Balanced Firmitas is the **default profile**. + +It fits contexts where: +- Products are customer-facing +- Multiple teams or disciplines are involved +- Some regulatory or contractual obligations exist +- Long-term maintainability matters + +Typical examples: +- IoT platforms +- Commercial SaaS +- Embedded products +- Regulated but non-safety-critical systems + + + +### D.4.2 Core Characteristics + +Balanced Firmitas balances: +- Speed with predictability +- Autonomy with coherence +- Learning with accountability + +Governance is **visible but lightweight**. + + + +### D.4.3 Artefact Set (Balanced) + +Required: +- Vision Brief +- System-level requirements +- Architecture overview (C4 context/container) +- ADRs for significant decisions +- Risk register +- Delivery health checks + +Optional: +- RFCs for complex changes +- Selective traceability (RTM where it adds value) +- Formal test plans for critical paths + + + +### D.4.4 Validation & Quality + +- Continuous integration and automated testing +- Defined acceptance criteria +- Regular system-level validation +- Quality engineering embedded, not siloed + + + +## D.4.5 Governance Model + +- Lightweight transition gates at major lifecycle boundaries +- Architectural guardrails instead of approvals +- Exceptions discussed openly, not hidden + + + +### D.4.6 Risks and Failure Modes + +Common risks: +- Gradual process creep +- Tool-driven rather than outcome-driven behaviour +- Governance becoming implicit again + +Mitigations: +- Periodic governance reviews +- Explicit removal of low-value artefacts +- Leadership ownership of system health + + + +## D.5 Regulated Firmitas Profile + +### D.5.1 Intended Context + +Regulated Firmitas applies where: +- Failure can cause harm +- Compliance is mandatory +- External audits are expected +- Traceability is non-negotiable + +Typical examples: +- Medical devices +- Automotive systems +- Aerospace and defence +- Energy and critical infrastructure + + + +### D.5.2 Core Characteristics + +Regulated Firmitas: +- Preserves learning and iteration +- Adds formality where risk demands it +- Treats compliance as a design constraint, not an afterthought + +It is **not waterfall**, even when heavily governed. + + + +### D.5.3 Artefact Set (Regulated) + +Required: +- Vision Brief +- Formal system and sub-system requirements +- Full RTM +- Architecture documentation (C4 + detailed views) +- ADRs (comprehensive) +- Risk register & hazard analysis +- Formal test plans and reports +- Configuration and change records + + + +### D.5.4 Validation & Verification + +- Explicit V&V strategy +- Formal reviews and sign-offs +- Independent verification where required +- Evidence generation integrated into delivery flow + + + +### D.5.5 Governance Model + +- Defined lifecycle gates with clear entry/exit criteria +- Formal architectural and safety reviews +- Clear accountability and escalation paths + +Governance is designed to: +- Reduce uncertainty +- Support audits +- Protect system integrity + + + +### D.5.6 Slack in Regulated Systems + +Slack is **mandatory**, not optional. + +- Capacity reserved for defect resolution +- Time allocated for audits and reviews +- Margin built into schedules and designs + +Regulated systems without slack fail catastrophically. + + + +## D.6 Moving Between Profiles + +Profiles are **not permanent**. + +Products may: +- Start Minimal, then become Balanced +- Operate Balanced with Regulated subsystems +- Temporarily tighten governance after incidents + +Key rule: +> Change profiles deliberately, not accidentally. + + + +## D.7 Leadership Responsibilities Across Profiles + +Leadership must: +- Select the appropriate profile consciously +- Explain *why* a profile is chosen +- Protect slack regardless of profile +- Prevent profile drift through neglect + +Firmitas succeeds when **intentionality replaces habit**. + + + +## D.8 Summary + +Minimal, Balanced, and Regulated Firmitas are not different frameworks. +They are different expressions of the same philosophy. + +What changes is **how much structure is required to protect system health**. + +What never changes: +- Focus on outcomes +- Respect for people +- Systemic thinking +- Learning over blame +- Long-term value creation diff --git a/content/firmitas/appendices/e-end-to-end-worked-example.md b/content/firmitas/appendices/e-end-to-end-worked-example.md new file mode 100644 index 0000000..5a1fa6a --- /dev/null +++ b/content/firmitas/appendices/e-end-to-end-worked-example.md @@ -0,0 +1,263 @@ +--- +title: "Appendix E: End-to-End Worked Example" +--- + +# Appendix E: End-to-End Worked Example + +## A Single Product Through the Full Firmitas Lifecycle + +This appendix provides a **concrete, end-to-end example** of how Firmitas is applied to a single product across its full lifecycle. + +The goal is not to prescribe a “correct” implementation, but to **demonstrate how philosophy, framework, and process interact** in practice, and how artefacts remain lightweight, living, and value-focused. + + + +## E.1 Product Context + +### Product Overview + +**Product:** Smart Gas Metre (NB‑IoT enabled) +**Domain:** Regulated IoT / critical national infrastructure +**Disciplines:** Firmware, hardware, cloud software, manufacturing, operations +**Regulatory Context:** MID, security standards, safety requirements +**Profile Used:** Regulated Firmitas (with a Balanced delivery core) + + + +### Business Objectives + +- Enable accurate, remote gas metering +- Support prepayment and credit modes +- Reduce operational cost via remote management +- Meet regulatory, security, and safety obligations + +Value is realised **only when meters operate reliably in the field over many years**, not when features are shipped. + + + +## E.2 Philosophy in Action + +From day one, leadership explicitly sets context: + +- **Purpose:** Safe, reliable metering that customers trust +- **Constraints:** Regulation, security, battery life, field longevity +- **Non‑Goals:** Rapid feature churn, experimental hardware in production +- **Slack:** Explicit buffer for learning, audits, and field feedback + +This context enables teams to make local decisions without constant escalation. + + + +## E.3 Structured Foundation Phase + +### E.3.1 Vision Brief + +A concise Vision Brief is created: + +- Problem statement +- Customer and regulator needs +- Success criteria (accuracy, uptime, audit success) +- Explicit constraints +- Known risks + +This document is reviewed once, then evolves slowly. + + + +### E.3.2 System Requirements + +System-level requirements are captured: + +- Functional (metering, communications, updates) +- Non-functional (battery life, latency, reliability) +- Regulatory and safety requirements + +Requirements are written to **express intent**, not implementation. + + + +### E.3.3 High-Level Architecture + +Architecture work focuses on: + +- System boundaries (metre, cloud, head-end) +- Key interfaces (radio, security, manufacturing) +- Failure modes and recovery paths + +C4 Context and Container diagrams are created and version-controlled. + + + +### E.3.4 Early Architectural Decisions + +Key ADRs are written for: +- Communications protocol +- Security model +- Key hardware components + +Each ADR captures trade-offs and future implications. + + + +## E.4 Iterative Development Core + +### E.4.1 Breaking Down Work + +System requirements are decomposed into: +- Firmware features +- Hardware revisions +- Cloud services +- Manufacturing processes + +A lightweight RTM links: +- Requirement → ADR → implementation → tests + + + +### E.4.2 Firmware & Software Iterations + +Development proceeds in short cycles: + +- Firmware stubs against simulated hardware +- Cloud services mocked early +- Continuous integration on every change +- Automated tests executed on-host and on-target + +Feedback is constant and visible. + + + +### E.4.3 Hardware Iteration + +Hardware progresses in parallel: + +- Early prototypes +- Iterative PCB revisions +- Emulators used to decouple firmware from hardware delays + +Hardware lead times are treated as **explicit risks**, not surprises. + + + +### E.4.4 Managing Risk + +A living risk register tracks: +- Supply chain risks +- Security risks +- Performance uncertainties + +High-risk items are addressed early through spikes and prototypes. + + + +## E.5 Validation and Verification Arm + +### E.5.1 System Integration + +Software, firmware, and hardware converge: + +- Full system builds +- Integration testing +- Fault injection and failure testing + +Integration is incremental, not a “big bang”. + + + +### E.5.2 Formal Verification + +Formal V&V includes: +- Accuracy testing +- Environmental testing +- Security penetration testing +- Compliance verification + +Evidence is generated **as part of normal delivery**, not as a separate phase. + + + +### E.5.3 Acceptance and Release + +Release readiness is assessed against: +- System requirements +- Risk posture +- Operational readiness + +Sign-off is based on **confidence**, not checkbox completion. + + + +## E.6 Operational Feedback Loop + +### E.6.1 Field Deployment + +Once deployed: +- Telemetry feeds back into engineering +- Operational incidents are reviewed blamelessly +- Assumptions are validated or challenged + +The system continues to evolve. + + + +### E.6.2 Incident Learning + +When incidents occur: +- Root causes are systemic +- ADRs are updated if assumptions change +- Architecture evolves deliberately + +Failure strengthens the system. + + + +## E.7 Artefact Summary + +| Artefact | Nature | +|--|-| +| Vision Brief | Stable, concise | +| Requirements | Living, traceable | +| ADRs | Continuous | +| Architecture Diagrams | Updated as system evolves | +| Risk Register | Actively maintained | +| Test Evidence | Generated continuously | + +No artefact exists without purpose. + + + +## E.8 Profile Transitions + +During early discovery: +- Minimal Firmitas applied locally + +During development: +- Balanced practices dominate + +For release and compliance: +- Regulated profile governs + +Profile changes are explicit and communicated. + + + +## E.9 What This Example Demonstrates + +- Firmitas scales without changing philosophy +- Regulation does not require waterfall +- Slack enables learning and resilience +- Architecture is a delivery enabler +- Leadership is exercised through system design + + + +## E.10 Key Takeaway + +Firmitas works when: +- Context is explicit +- Systems are designed intentionally +- Learning is prioritised over blame +- Governance serves delivery, not the reverse + +This example is representative, not exhaustive. +Its purpose is to show **how the pieces fit together in reality**. diff --git a/content/firmitas/appendices/f-leadership-and-delivery-anti-patterns-firmitas-is-designed-to-prevent.md b/content/firmitas/appendices/f-leadership-and-delivery-anti-patterns-firmitas-is-designed-to-prevent.md new file mode 100644 index 0000000..45184e7 --- /dev/null +++ b/content/firmitas/appendices/f-leadership-and-delivery-anti-patterns-firmitas-is-designed-to-prevent.md @@ -0,0 +1,334 @@ +--- +title: "Appendix F: Leadership and Delivery Anti-Patterns Firmitas Is Designed to Prevent" +--- + +# Appendix F: Leadership and Delivery Anti-Patterns Firmitas Is Designed to Prevent + +This appendix documents **common leadership, architectural, and delivery anti-patterns** that repeatedly undermine engineering organisations. + +Firmitas does not exist to impose a new process. +It exists to **systematically prevent these failure modes** by design. + +Each anti-pattern is described with: +- What it looks like +- Why it is harmful +- How Firmitas counters it + +This appendix is intentionally direct. + + + +## F.1 Leadership Anti-Patterns + +### F.1.1 Proxy Leadership + +**What it looks like** +Leadership is replaced by frameworks, tools, or roles. Decisions are deferred to process rather than owned by leaders. + +**Why it is harmful** +- Responsibility becomes diluted +- Teams lose clarity on intent +- Governance becomes performative + +**How Firmitas counters it** +- Makes leadership a **non-delegable systems responsibility** +- Frames frameworks as support, not substitutes +- Forces explicit ownership of system health + + + +### F.1.2 Hero Leadership + +**What it looks like** +Delivery relies on a small number of individuals working excessive hours to “save” projects. + +**Why it is harmful** +- Masks systemic failures +- Creates fragility and burnout +- Prevents learning + +**How Firmitas counters it** +- Explicitly protects slack +- Treats heroics as system failure signals +- Optimises for sustainable throughput + + + +### F.1.3 Control Through Fear + +**What it looks like** +Metrics used to punish, audits used to intimidate, escalation used as threat. + +**Why it is harmful** +- Suppresses bad news +- Delays risk discovery +- Encourages gaming behaviour + +**How Firmitas counters it** +- Emphasises psychological safety +- Separates learning metrics from performance judgement +- Uses governance to guide, not punish + + + +## F.2 Organisational and Governance Anti-Patterns + +### F.2.1 Decision Bottlenecking + +**What it looks like** +All meaningful decisions flow through a small central group. + +**Why it is harmful** +- High decision latency +- Reduced ownership +- Local optimisation by avoidance + +**How Firmitas counters it** +- Uses architectural guardrails instead of approvals +- Pushes decisions to the edge +- Makes escalation rare and explicit + + + +### F.2.2 Governance Theatre + +**What it looks like** +Reviews, gates, and documents exist primarily to demonstrate compliance rather than reduce risk. + +**Why it is harmful** +- Adds cost without value +- Creates false confidence +- Encourages box-ticking + +**How Firmitas counters it** +- Requires every artefact to have a clear purpose +- Treats evidence generation as part of delivery +- Optimises governance for risk reduction, not optics + + + +### F.2.3 Tool-Led Transformation + +**What it looks like** +New tools are introduced as a substitute for fixing underlying system issues. + +**Why it is harmful** +- Increases cognitive load +- Avoids real change +- Creates superficial progress + +**How Firmitas counters it** +- Prioritises process consistency over tool uniformity +- Treats tooling as a socio-technical choice +- Forces justification of tools against outcomes + + + +## F.3 Delivery and Process Anti-Patterns + +### F.3.1 Cargo-Cult Agile + +**What it looks like** +Scrum ceremonies without autonomy, iterations without learning, backlogs without outcomes. + +**Why it is harmful** +- Creates motion without progress +- Erodes trust in agile thinking +- Hides systemic issues + +**How Firmitas counters it** +- Separates principles from practices +- Anchors delivery to outcomes +- Makes learning explicit + + + +### F.3.2 Waterfall Disguised as Agile + +**What it looks like** +Upfront design and commitment with iterative labels applied later. + +**Why it is harmful** +- Late discovery of risk +- False sense of adaptability +- Costly rework + +**How Firmitas counters it** +- Enforces incremental validation +- Encourages early integration +- Treats assumptions as revisitable + + + +### F.3.3 Output Obsession + +**What it looks like** +Success measured by features delivered, tickets closed, or utilisation maximised. + +**Why it is harmful** +- Disconnects work from value +- Encourages overproduction +- Increases WIP and delays feedback + +**How Firmitas counters it** +- Shifts focus to outcomes +- Treats WIP as a system variable +- Measures success through value realised + + + +## F.4 Architectural Anti-Patterns + +### F.4.1 Big Design Up Front (BDUF) + +**What it looks like** +Extensive upfront design attempting to eliminate uncertainty. + +**Why it is harmful** +- Assumes stability +- Locks in poor assumptions +- Delays learning + +**How Firmitas counters it** +- Assumes change by default +- Uses evolutionary architecture +- Preserves optionality + + + +### F.4.2 Accidental Architecture + +**What it looks like** +Architecture emerges unintentionally from local decisions. + +**Why it is harmful** +- Hidden coupling +- Inconsistent patterns +- Long-term fragility + +**How Firmitas counters it** +- Makes architecture a leadership concern +- Uses ADRs to surface intent +- Maintains architectural coherence + + + +### F.4.3 Platform as Control + +**What it looks like** +Platforms impose constraints primarily to enforce compliance or control teams. + +**Why it is harmful** +- Increases friction +- Reduces autonomy +- Encourages workarounds + +**How Firmitas counters it** +- Defines platforms as enablement mechanisms +- Measures platform success by team outcomes +- Treats adoption as value-driven + + + +## F.5 Quality and Risk Anti-Patterns + +### F.5.1 Late Quality + +**What it looks like** +Testing and validation deferred to the end of delivery. + +**Why it is harmful** +- Amplifies cost of defects +- Delays feedback +- Creates blame cycles + +**How Firmitas counters it** +- Integrates V&V continuously +- Shifts testing left +- Treats quality as systemic + + + +### F.5.2 Hidden Risk + +**What it looks like** +Risks known but unspoken due to fear or incentives. + +**Why it is harmful** +- Explodes late +- Forces crisis response +- Damages trust + +**How Firmitas counters it** +- Makes risk visibility explicit +- Links risk to leadership responsibility +- Normalises early escalation + + + +### F.5.3 Zero-Slack Systems + +**What it looks like** +Teams run at full utilisation with no buffer. + +**Why it is harmful** +- Eliminates learning capacity +- Turns small issues into crises +- Guarantees burnout + +**How Firmitas counters it** +- Treats slack as a structural requirement +- Plans capacity intentionally +- Protects recovery and improvement time + + + +## F.6 Cultural Anti-Patterns + +### F.6.1 Blame-Oriented Postmortems + +**What it looks like** +Incidents framed around who failed rather than what failed. + +**Why it is harmful** +- Suppresses information +- Prevents improvement +- Encourages risk hiding + +**How Firmitas counters it** +- Enforces blameless reviews +- Focuses on system conditions +- Converts failure into learning + + + +### F.6.2 Local Optimisation Culture + +**What it looks like** +Teams rewarded for local efficiency regardless of system impact. + +**Why it is harmful** +- Creates silos +- Degrades end-to-end flow +- Undermines collaboration + +**How Firmitas counters it** +- Optimises for whole-system outcomes +- Aligns incentives to shared success +- Makes dependencies visible + + + +## F.7 Summary + +Firmitas is intentionally designed to: +- Surface reality early +- Prevent silent failure modes +- Replace heroics with systems +- Trade false certainty for learning +- Optimise for long-term value + +Anti-patterns do not disappear through good intentions. +They disappear when **systems are redesigned to make them impossible or expensive**. + +That is the purpose of Firmitas. diff --git a/content/firmitas/appendices/g-bibliography-and-further-reading.md b/content/firmitas/appendices/g-bibliography-and-further-reading.md new file mode 100644 index 0000000..c88e1ca --- /dev/null +++ b/content/firmitas/appendices/g-bibliography-and-further-reading.md @@ -0,0 +1,174 @@ +--- +title: "Appendix G: Bibliography and Further Reading" +--- + +# Appendix G: Bibliography and Further Reading + +This bibliography is deliberate and opinionated. + +The works listed here directly inform the philosophy, structure, and delivery logic of Firmitas. They shape how Firmitas treats leadership, systems, risk, learning, architecture, flow, quality, and human behaviour. These are not optional influences; they are the intellectual foundations that prevent Firmitas from collapsing into a shallow hybrid of existing methods. + +Where appropriate, short context is provided to explain why each work matters. + + + +## G.1 Systems Thinking and Organisational Dynamics + +### Donella H. Meadows – *Thinking in Systems* +Foundational to Firmitas’s systems worldview. Reinforces feedback loops, leverage points, and long-term consequences. Firmitas’s rejection of local optimisation and short-term efficiency is rooted directly in this work. + +### W. Edwards Deming – *Out of the Crisis* +Introduces the idea that most failures are systemic, not individual. Firmitas’s leadership stance, quality philosophy, and rejection of blame are deeply Deming-aligned. + +### Russell Ackoff – *Re-Creating the Corporation* +Explains why optimising parts of a system degrades the whole. Directly informs Firmitas’s portfolio thinking and organisational design guidance. + + + +## G.2 Leadership, Authority, and Responsibility + +### L. David Marquet – *Turn the Ship Around!* +Strong influence on Firmitas’s leadership model. Authority is pushed to where information exists, while accountability remains with leadership. This aligns with Firmitas’s “context over control” stance. + +### Simon Sinek – *Start With Why*, *Leaders Eat Last*, *The Infinite Game* +Reinforces: +- Purpose-driven leadership +- Long-term, infinite-game thinking +- Leadership as stewardship of people and systems +Firmitas adopts these ideas explicitly while grounding them in delivery reality. + +### Jocko Willink & Leif Babin – *Extreme Ownership* +Informs Firmitas’s principle that leadership responsibility for system health cannot be delegated, even when authority is distributed. + +### General Stanley McChrystal – *Team of Teams* +Supports Firmitas’s emphasis on adaptability, decentralised execution, and information flow in complex environments. + +### Frederic Laloux – *Reinventing Organizations* +Influences Firmitas’s thinking on organisational evolution, trust, and self-management, without assuming idealised maturity. + + + +## G.3 Human Systems, Motivation, and Team Health + +### Tom DeMarco & Tim Lister – *Peopleware* +One of the strongest influences on Firmitas. +Key ideas adopted directly: +- Productivity is dominated by system design +- Interruptions destroy thinking +- Environment matters more than tools + +### Tom DeMarco – *Slack* +Provides the intellectual basis for Firmitas’s insistence that slack is structural, not waste. Zero-slack systems are fragile and unethical. + +### Dan Pink – *Drive* +Informs Firmitas’s treatment of autonomy, mastery, and purpose as system design goals rather than motivational techniques. + +### Susan Fowler – *Why Motivating People Doesn’t Work… and What Does* +Reinforces intrinsic motivation and autonomy-supportive leadership, aligning with Firmitas’s rejection of incentive-driven control. + +### Patrick Lencioni – *The Five Dysfunctions of a Team* +Used selectively to support Firmitas’s focus on trust, conflict, and accountability, without oversimplifying team dynamics. + +### Kim Scott – *Radical Candor* +Supports Firmitas’s stance on high-trust, high-accountability leadership communication. + + + +## G.4 Engineering Management and Leadership Practice + +### Camille Fournier – *The Manager’s Path* +Informs Firmitas’s guidance on engineering leadership roles, progression, and responsibility boundaries. + +### Will Larson – *An Elegant Puzzle* +Strong alignment with Firmitas’s systems-first view of engineering management, especially around scaling and organisational constraints. + +### Michael Lopp – *Managing Humans* +Reinforces Firmitas’s people-centred leadership stance with practical, experience-based insight. + +### Ben Collins-Sussman, Brian Fitzpatrick, Dan Pilone – *Team Geek* +Supports Firmitas’s emphasis on humility, trust, and collaboration in technical teams. + + + +## G.5 Software Engineering and Architecture + +### Frederick P. Brooks – *The Mythical Man-Month* +Firmitas’s scepticism of heroics, linear scaling, and schedule compression comes directly from Brooks. + +### John Ousterhout – *A Philosophy of Software Design* +Strong influence on Firmitas’s architectural principles: +- Complexity as the primary enemy +- Cognitive load as a first-class concern + +### Mark Richards & Neal Ford – *Fundamentals of Software Architecture* +### Mark Richards & Neal Ford – *Software Architecture: The Hard Parts* +Inform Firmitas’s architectural decision-making, trade-off analysis, and evolutionary design stance. + +### Neal Ford, Rebecca Parsons, Patrick Kua – *Building Evolutionary Architectures* +Directly informs Firmitas’s approach to change, fitness functions, and long-lived systems. + +### Nick Rozanski & Eóin Woods – *Software Systems Architecture* +Supports structured architectural reasoning without prescribing heavyweight frameworks. + + + +## G.6 Flow, Economics, and Risk + +### Donald G. Reinertsen – *Managing the Design Factory* +### Donald G. Reinertsen – *Principles of Product Development Flow* +Firmitas’s economic view of flow, batch size, queues, and cost of delay is heavily influenced by Reinertsen. + +### Tom DeMarco & Tim Lister – *Waltzing with Bears* +Informs Firmitas’s explicit risk management stance and rejection of false certainty in planning. + +### Douglas W. Hubbard – *How to Measure Anything* +Supports Firmitas’s pragmatic approach to measurement, uncertainty, and decision-making under imperfect information. + +### Tom DeMarco – *The Deadline* +A narrative reinforcement of systems thinking, leadership behaviour, and organisational dysfunctions relevant to delivery. + + + +## G.7 DevOps, Delivery, and Socio-Technical Systems + +### Forsgren, Humble, Kim – *Accelerate* +Empirical reinforcement of flow, feedback, and learning as predictors of performance. + +### Gene Kim et al. – *The DevOps Handbook* +Informs Firmitas’s socio-technical systems thinking and operational feedback loops. + +### Mary & Tom Poppendieck – *Lean Software Development* +Reinforces flow-based delivery and waste reduction without collapsing into minimalism. + + + +## G.8 Team Design and Communication + +### Matthew Skelton & Manuel Pais – *Team Topologies* +Direct influence on Firmitas’s team design guidance, cognitive load management, and interaction modes. + +### Brandenburg & Strohschneider – *Communication Patterns* +Supports Firmitas’s explicit treatment of communication as a designed system, not an emergent property. + + + +## G.9 Documentation, Knowledge, and Governance + +### Anne Gentle – *Docs as Code* +Direct influence on Firmitas’s documentation philosophy and artefact governance model. + +### The Agile Manifesto +Referenced as historical context and a source of values, not as a sufficient delivery model. + + + +## G.10 How to Use This Bibliography + +This bibliography is not meant to be read linearly. + +Instead: +- Treat it as a map of ideas +- Revisit works as organisational maturity increases +- Use books as lenses, not rulebooks + +Firmitas exists precisely because no single book, framework, or methodology is sufficient on its own. diff --git a/content/firmitas/appendices/h-engineering-practice-across-disciplines.md b/content/firmitas/appendices/h-engineering-practice-across-disciplines.md new file mode 100644 index 0000000..b563227 --- /dev/null +++ b/content/firmitas/appendices/h-engineering-practice-across-disciplines.md @@ -0,0 +1,1439 @@ +--- +title: "Appendix H: Engineering Practice Across Disciplines" +--- + +# Appendix H: Engineering Practice Across Disciplines + +Firmitas is intentionally multi-disciplinary. It exists for organisations building real products, not abstractions. This appendix addresses how Firmitas’s philosophy and framework translate into **day-to-day engineering practice across software, firmware, hardware, mechanical, and quality disciplines**, while preserving coherence at the system level. + +A common failure mode in product development is treating disciplines as loosely coupled silos, each operating with its own methods, artefacts, and success measures. This leads to local optimisation, late integration, brittle interfaces, and systemic risk that only becomes visible when it is expensive or irreversible. This appendix is explicitly designed to prevent that outcome. + +This section treats engineering practice as a **socio-technical system**. Code, schematics, CAD models, tests, simulations, and tooling are inseparable from communication patterns, decision rights, and organisational structure. Technical excellence is not achieved through tools or techniques alone, but through the alignment of people, practices, and architecture around a shared purpose. + +this appendix makes explicit that different disciplines operate on **different timescales and cost curves**. Software is cheap to change, hardware is not. Firmware sits in between. Mechanical and manufacturing decisions carry long-lived consequences. Firmitas does not attempt to flatten these differences. Instead, it provides a structured way to coordinate them through: +- Clear interfaces and ownership boundaries +- Iterative learning where it is cheap and early +- Deliberate commitment points where change becomes costly + +Quality engineering is treated as a **system-level property**, not a downstream function. Verification and validation are integrated into everyday practice across disciplines, with explicit traceability back to requirements, risks, and business intent. This allows Firmitas to align naturally with regulated and safety-critical environments without distorting delivery. + +Modern engineering realities are addressed directly. this appendix includes guidance on: +- Continuous integration across heterogeneous toolchains +- Simulation, emulation, and hardware-in-the-loop testing +- Managing technical debt and architectural drift over long lifecycles +- Integrating AI-assisted development and agent-based tooling responsibly + +The emphasis throughout is on **coherence over uniformity**. Firmitas does not mandate identical practices across disciplines. It mandates shared intent, shared interfaces, and shared accountability for outcomes. Practices are allowed to vary where it makes sense, provided they strengthen the overall system rather than fragment it. + +By the end of this appendix, the reader should understand: +- How different engineering disciplines fit together as a single delivery system +- How to manage integration, testing, and change across uneven iteration speeds +- How to maintain architectural and quality integrity over time +- How to apply Firmitas in modern, tool-rich, AI-augmented environments + +this appendix grounds Firmitas in engineering reality. It is where the framework proves that it can support the messy, constrained, high-stakes work of building real products, not just elegant diagrams. + +## Software and Firmware Development in Firmitas + +Software and firmware development sit at the centre of most modern products, even when the product itself is not “software-first”. In Firmitas, software and firmware are treated neither as special cases nor as generic commodities. They are **socio-technical systems embedded within larger product, organisational, and economic systems**. + +This chapter explains how software and firmware development operate within Firmitas, and how day-to-day engineering practice aligns with the philosophy, framework, and delivery shell established earlier in the book. + +The intent is not to prescribe a specific programming model, language, or tooling stack, but to define **the constraints, responsibilities, and expectations** under which software and firmware work must occur to support long-term system health. + + + +### Software and Firmware as System-Shaping Forces + +Software and firmware do more than implement requirements. They shape: + +- how products evolve after release +- how quickly defects can be corrected +- how safely systems fail +- how observable and operable products are in the field +- how tightly coupled future change becomes + +Because of this, software and firmware decisions are **architectural, operational, and economic decisions**, not just technical ones. + +Firmitas therefore treats software and firmware development as: +- a long-lived responsibility, not a project phase +- a source of leverage and risk +- a primary driver of product adaptability + + + +### The Role of Software and Firmware Within the Firmitas Delivery Shell + +Within the Firmitas delivery shell, software and firmware work spans all major phases: + +- contributing to early feasibility and architectural exploration +- driving learning within the Iterative Development Core +- anchoring continuous verification and validation +- supporting system integration and operational readiness + +This explicitly rejects the idea that software “starts after design” or that firmware is merely an implementation detail following hardware decisions. + +Instead, software and firmware: +- participate early in shaping constraints and trade-offs +- expose risk quickly through executable artefacts +- provide fast feedback loops even when other disciplines move more slowly + + + +### Requirements: From Intent to Executable Behaviour + +Firmitas does not treat requirements as static inputs handed to software teams. + +For software and firmware, requirements evolve across levels: +- system-level intent and constraints +- capability and feature definitions +- detailed behavioural specifications +- acceptance and verification criteria + +Key expectations include: +- traceability from system intent to implemented behaviour +- explicit handling of non-functional requirements such as timing, memory, power, security, and safety +- early identification of requirements that drive architectural decisions + +In embedded and firmware contexts in particular, non-functional requirements are often the primary risk drivers. Firmitas requires these to be surfaced early rather than discovered through late integration failures. + + + +### Architecture and Design in Software and Firmware + +Architecture in Firmitas is not a document phase. It is a **continuous decision-making activity**. + +For software and firmware this includes: +- defining clear component boundaries +- making interfaces explicit +- managing coupling deliberately +- choosing appropriate abstractions for the domain + +Firmitas encourages: +- simple, comprehensible designs over cleverness +- modularity that supports testing and change +- architectures that align with team ownership boundaries + +Architectural decisions are captured using ADRs, ensuring that rationale survives personnel changes and that future teams understand why constraints exist, not just that they do. + + + +### Iterative Development and Incremental Learning + +Software and firmware are primary engines of iteration within Firmitas. + +The Iterative Development Core enables: +- early execution of real code on real or representative targets +- rapid feedback on assumptions +- incremental risk reduction + +Iterations are not defined by ceremony but by outcomes: +- a risk retired +- a capability proven +- a constraint validated +- a learning made explicit + +This applies equally to: +- application software +- embedded firmware +- real-time systems +- safety-related logic + +Iteration is bounded by system constraints, not by arbitrary sprint rules. + + + +### Testing, Verification, and Validation as Continuous Activities + +Firmitas treats testing as an integral part of development, not a downstream phase. + +For software and firmware this includes: +- unit testing for behaviour and edge cases +- integration testing across components +- hardware-in-the-loop testing where applicable +- fault injection and negative testing +- regression testing tied to change, not schedule + +Verification and validation are continuous: +- verification ensures the system was built correctly +- validation ensures the right system is being built + +Automation is strongly encouraged, but automation is not a substitute for engineering judgement. Tests must reflect meaningful system behaviour, not just code coverage targets. + + + +### Toolchains as Socio-Technical Systems + +Toolchains are not neutral. They encode assumptions about: +- workflow +- responsibility +- authority +- quality gates +- feedback speed + +Firmitas evaluates software and firmware toolchains based on: +- how well they support fast, reliable feedback +- how they integrate documentation and code +- how visible system state and quality are +- how much cognitive load they impose + +A toolchain that technically “works” but fragments understanding or slows learning is treated as a system smell, not a success. + + + +### Managing Change in Software and Firmware Systems + +Change is inevitable. Fragility is optional. + +Firmitas expects software and firmware systems to: +- tolerate change without cascading failure +- make the cost of change visible +- degrade gracefully under unexpected conditions + +Practices that support this include: +- modular architectures +- disciplined interface management +- explicit deprecation strategies +- versioned communication contracts +- controlled branching and release strategies + +Change control exists to manage risk, not to prevent evolution. + + + +### Firmware-Specific Considerations + +Firmware development introduces additional constraints that Firmitas explicitly recognises: + +- limited resources (CPU, memory, power) +- hardware dependencies and lead times +- constrained update mechanisms +- safety and regulatory obligations +- tighter coupling between design and execution + +These constraints increase the importance of: +- early prototyping +- executable specifications +- realistic test environments +- close collaboration with hardware teams + +Firmitas does not relax discipline for firmware; it **raises it**, while still preserving iterative learning. + + + +### Software, Firmware, and Operational Responsibility + +In Firmitas, software and firmware teams do not “throw work over the wall”. + +They retain responsibility for: +- operability +- diagnosability +- update strategies +- incident response support +- long-term maintainability + +This reinforces: +- end-to-end ownership +- feedback from real-world use +- continuous improvement + +Operational pain is treated as a design signal, not as someone else’s problem. + + + +### Leadership Responsibilities in Software and Firmware Development + +Leaders are responsible for creating conditions in which good software and firmware can be built. + +This includes: +- protecting teams from chronic overload +- enforcing architectural discipline without micromanagement +- funding quality work that does not produce immediate features +- resisting deadline-driven degradation of system health +- making trade-offs explicit and visible + +Leadership failure in software systems almost always manifests later as technical debt, operational incidents, or loss of adaptability. + + + +### Summary + +In Firmitas: +- software and firmware are long-lived system assets +- architecture is continuous decision-making +- iteration is a learning mechanism, not a ritual +- testing and validation are integral to development +- toolchains are socio-technical systems +- operational responsibility is inseparable from delivery +- leadership behaviour shapes technical outcomes + +Software and firmware development are not isolated activities. +They are **central expressions of the organisation’s values, constraints, and system design**. + +The next chapter extends these principles into the physical world, addressing the unique challenges of **hardware and mechanical development** and their integration with software-led systems. :contentReference[oaicite:0]{index=0} + +## Hardware and Mechanical Development Considerations + +Firmitas treats hardware and mechanical engineering as **first-class system components**, not as downstream constraints to be worked around once “the real work” begins. In many organisations, hardware is framed as slow, rigid, and fundamentally incompatible with iterative delivery. That framing is incorrect. The problem is not hardware itself, but **how the surrounding delivery, decision, and organisational systems are designed**. + +This chapter explains how Firmitas integrates hardware and mechanical development into a coherent, system-level delivery model without pretending that hardware behaves like software. + + + +### Hardware and Mechanical Engineering as System Anchors + +Hardware and mechanical elements anchor products in physical reality. They introduce: + +- irreversibility at certain points +- long lead times and supply chain coupling +- manufacturing constraints +- regulatory and safety obligations +- cost structures that dominate unit economics + +These properties make hardware **high-leverage decisions**, not “implementation details”. + +In Firmitas, hardware and mechanical design: +- shape architectural boundaries +- define feasible iteration speed +- constrain or enable future change +- influence organisational structure and skills + +Treating them late or lightly creates systemic risk that no amount of software agility can compensate for. + + + +### The Fallacy of “Hardware Can’t Be Iterative” + +Firmitas explicitly rejects the false dichotomy that says: +> “Software can iterate, hardware cannot.” + +What is true is that **different parts of the system have different costs of change**. + +Hardware iteration exists, but it occurs through: +- staged commitment +- progressive fidelity +- deliberate decision points +- simulation and abstraction + +Firmitas designs the delivery system so that learning occurs **before irreversible commitments**, rather than attempting to iterate after cost has already been locked in. + + + +### Progressive Commitment in Physical Systems + +Firmitas applies the principle of **progressive commitment** to hardware and mechanical work. + +This means: +- early exploration is cheap and reversible +- decisions harden gradually +- commitment increases only as uncertainty decreases + +Typical progression includes: +- conceptual modelling and trade studies +- simulation and virtual prototyping +- low-fidelity physical prototypes +- engineering prototypes +- pre-production builds +- production tooling and manufacture + +The key is not the stages themselves, but **explicit recognition of when change cost increases**, and leadership responsibility for managing those transitions. + + + +### Early Hardware Involvement in System Architecture + +Hardware and mechanical engineers are involved from the earliest architectural discussions. + +This ensures: +- realistic constraints inform system design +- non-functional requirements are grounded in physics +- interfaces between disciplines are designed deliberately +- infeasible architectures are rejected early + +Firmitas treats architecture as a **cross-disciplinary activity**, not a software artefact with hardware “plugged in later”. + + + +### Managing Hardware–Software Interdependence + +In integrated products, the most severe failures occur at boundaries: +- hardware assumptions embedded in firmware +- mechanical tolerances affecting sensor behaviour +- thermal or power constraints undermining software guarantees + +Firmitas manages these risks by: +- making interfaces explicit and versioned +- using living Interface Control Documents (ICDs) +- aligning iteration cadences across disciplines +- ensuring integration happens early and repeatedly + +Integration is not a phase. It is a **continuous system property**. + + + +### Prototyping as a Learning Instrument + +Prototyping in Firmitas is not about impressing stakeholders. It is about **learning under constraint**. + +Hardware and mechanical prototypes are used to: +- validate assumptions +- expose failure modes +- inform architectural decisions +- reduce downstream risk + +This includes: +- rapid PCB spins +- mechanical mock-ups +- 3D-printed enclosures +- off-the-shelf component evaluation +- breadboards and dev kits + +The value of a prototype is measured by what it invalidates, not how polished it appears. + + + +### Simulation, Modelling, and Virtualisation + +Where physical iteration is expensive, Firmitas leans heavily on modelling. + +This includes: +- electrical simulation +- thermal modelling +- mechanical stress and fatigue analysis +- digital twins +- software-in-the-loop and hardware-in-the-loop testing + +Simulation does not replace physical testing, but it: +- shifts learning earlier +- reduces iteration cost +- narrows uncertainty before commitment + +Leaders must actively fund modelling capability. It is often the highest ROI investment in hardware-heavy systems. + + + +### Manufacturing, Supply Chain, and Economic Reality + +Hardware decisions are inseparable from manufacturing and supply chain systems. + +Firmitas explicitly considers: +- component availability and lifecycle +- supplier risk and dependency +- tooling lead times +- yield and testability +- unit cost versus non-recurring engineering (NRE) + +Engineering leadership is responsible for ensuring that: +- designs are manufacturable +- cost trade-offs are explicit +- supply chain risks are surfaced early + +Ignoring these realities leads to late redesign, missed markets, or fragile products. + + + +### Quality, Safety, and Regulatory Constraints + +Hardware and mechanical systems are often subject to: +- safety standards +- environmental regulations +- industry-specific compliance regimes + +Firmitas integrates these constraints into normal delivery flow by: +- embedding compliance requirements into system intent +- treating evidence as a living artefact +- aligning verification activities with iteration, not gates alone + +Quality is not inspected in at the end. It is **designed in from the start**. + + + +### Organising Teams Around Physical Systems + +Hardware and mechanical work place different cognitive and coordination demands on teams. + +Firmitas recognises that: +- excessive context switching is especially damaging +- unclear ownership leads to integration failure +- long feedback loops amplify mistakes + +Teams are organised to: +- own coherent subsystems +- minimise cross-team dependencies +- align accountability with decision authority + +This directly reflects the socio-technical principles established earlier in the book. + + + +### Change Management in Physical Domains + +Change in hardware systems must be managed deliberately, but not fearfully. + +Firmitas: +- distinguishes learning-driven change from churn +- evaluates change based on system impact, not sunk cost +- uses structured decision points to manage risk + +Suppressing change too early creates brittle designs. +Allowing uncontrolled change too late creates chaos. + +Leadership judgement, not process, is decisive here. + + + +### Leadership Responsibilities in Hardware-Centric Products + +In hardware and mechanical contexts, leadership responsibilities intensify. + +Leaders must: +- protect time for early exploration +- resist premature cost optimisation +- ensure cross-disciplinary collaboration +- fund prototyping and modelling +- make irreversible decisions explicit and conscious + +Most hardware failures are leadership failures long before they become technical ones. + + + +### Summary + +In Firmitas: +- hardware and mechanical systems are integral, not secondary +- iteration exists, but through progressive commitment +- architecture is cross-disciplinary +- prototyping and modelling are core learning tools +- manufacturing and economics are engineering concerns +- quality and compliance are system properties +- leadership behaviour determines outcomes + +Firmitas does not attempt to make hardware behave like software. +It designs **a delivery system that respects physical reality while preserving learning, control, and long-term adaptability**. + +The next chapter addresses how these disciplines are integrated into **whole systems**, focusing on dependency management, interfaces, and cross-domain coherence. + +## Integrated Systems and Dependency Management + +Modern products are rarely isolated. +They are networks of interdependent components, teams, technologies, suppliers, and decisions that evolve over time. + +Failures in complex products are far more often caused by **poorly managed dependencies** than by deficiencies in individual components. Firmitas treats integration and dependency management as a first-class systems problem, not a late-phase technical activity. + +### Integration Is a System Property, Not a Phase + +Traditional delivery models often treat integration as something that happens “later”: +after design, +after implementation, +after teams have optimised locally. + +This framing is fundamentally flawed. + +Integration is not an event. +It is a **continuous system behaviour** shaped by: +- architectural boundaries, +- organisational structure, +- communication patterns, +- tooling, +- and decision timing. + +Firmitas treats integration as something that must be designed for from the outset and continuously exercised throughout delivery. + +### Types of Dependencies in Integrated Systems + +Dependencies exist across multiple system layers simultaneously. Focusing on only one dimension guarantees blind spots elsewhere. + +Firmitas explicitly recognises and manages the following dependency types: + +#### Technical Dependencies +- Interfaces between software modules, firmware components, hardware subsystems, and external services. +- Coupling created by shared state, shared resources, timing assumptions, or hidden contracts. +- Version compatibility and backward/forward compatibility constraints. + +Poorly managed technical dependencies increase failure propagation and slow change. + +#### Organisational Dependencies +- Dependencies between teams created by ownership boundaries, approval paths, and reporting lines. +- Waiting on decisions, reviews, or resources controlled elsewhere. +- Hidden queues introduced by shared specialists or centralised functions. + +These dependencies are often invisible in code but dominant in delivery performance. + +#### Socio-Technical Dependencies +- Toolchains that require coordination across teams. +- Documentation formats that encode assumptions about who decides and who consumes. +- CI/CD pipelines that assume particular integration sequences or ownership models. + +Socio-technical dependencies frequently constrain flow more than technology itself. + +#### External and Supply Chain Dependencies +- Third-party components, vendors, cloud services, manufacturing partners, and regulators. +- Lead times, certification cycles, and contractual constraints. +- End-of-life risks and component obsolescence. + +These dependencies must be surfaced early, not discovered during validation. + +### Dependency Visibility as a Leadership Responsibility + +Dependencies do not manage themselves. +They must be made visible, discussed, and deliberately shaped. + +Firmitas places responsibility for dependency visibility squarely with leadership, not individual contributors. This includes: +- architects surfacing technical coupling, +- delivery leadership exposing organisational queues, +- and programme leadership making economic trade-offs explicit. + +Invisible dependencies always cost more than visible ones. + +### Designing for Loose Coupling and Clear Ownership + +The primary mechanism for managing dependencies is **design**, not coordination. + +Firmitas emphasises: +- clear ownership boundaries, +- explicit interfaces, +- and minimal coupling between components and teams. + +Key practices include: +- defining stable interfaces early, even if implementations evolve, +- assigning single accountable owners for interfaces and integration points, +- and avoiding shared mutable state wherever possible. + +Ownership clarity reduces coordination overhead and accelerates learning. + +### Interface Control as a Living System + +Interface Control Documents (ICDs) are often treated as static, bureaucratic artefacts. +In Firmitas, they are living system contracts. + +Effective ICDs: +- are version-controlled, +- evolve alongside the system, +- and are reviewed using the same mechanisms as code. + +They exist to reduce ambiguity, not to freeze design prematurely. + +Importantly, ICDs are socio-technical artefacts: they encode expectations about communication, responsibility, and change tolerance. + +### Integration Cadence and Early Validation + +Integration deferred is risk multiplied. + +Firmitas encourages: +- frequent integration, +- early system-level testing, +- and continuous validation of assumptions. + +This applies equally to: +- software-software integration, +- firmware-hardware integration, +- and system-environment interaction. + +Early integration surfaces systemic issues while they are still cheap to fix. + +### Dependency Management and Flow + +Every dependency introduces potential delay. +Every delay introduces queues. +Every queue increases cycle time and reduces feedback quality. + +Firmitas explicitly links dependency management to flow: +- fewer dependencies mean faster learning, +- clearer dependencies mean better planning, +- and well-managed dependencies reduce the need for heroic coordination. + +Local optimisation that increases dependency density is actively discouraged. + +### Economic Consequences of Dependency Decisions + +Dependency decisions are economic decisions. + +Tightly coupled systems: +- increase change cost, +- increase failure impact, +- and reduce optionality. + +Loosely coupled systems: +- preserve strategic flexibility, +- enable incremental evolution, +- and reduce long-term cost of ownership. + +Firmitas requires leaders to make these trade-offs explicit, rather than allowing them to emerge accidentally. + +### Integration as Continuous Learning + +Integration is not just about making things work together. +It is one of the richest sources of learning about the system. + +Every integration issue reveals: +- mismatched assumptions, +- unclear ownership, +- or systemic weaknesses. + +Firmitas treats integration failures as learning signals, not individual errors. This aligns directly with its commitment to psychological safety and continuous improvement. + +### Integration in Regulated and Safety-Critical Contexts + +In regulated environments, integration carries additional obligations: +- traceability, +- evidence generation, +- and formal verification. + +Firmitas integrates these requirements without reverting to late-stage integration by: +- maintaining continuous traceability, +- integrating verification into normal flow, +- and treating compliance artefacts as living documents. + +This preserves learning speed while satisfying external assurance needs. + +### Summary: Integration Is Leadership Work + +Integrated systems succeed or fail based on how dependencies are designed, owned, and evolved. + +Firmitas makes this explicit: +- integration is not a technical afterthought, +- dependency management is not a coordination tax, +- and system health is not emergent by accident. + +Engineering leadership is responsible for shaping the conditions under which integration becomes routine, safe, and informative rather than painful and destabilising. + +The next chapter builds directly on this foundation by formalising **architecture decision-making as a living, traceable practice**, rather than a one-off design activity. + +## Architecture Decision Records and Living Design + +Architecture is not a phase. +It is not a document. +It is not a role. + +Architecture is the **accumulated set of decisions that shape how a system behaves, changes, fails, and evolves over time**. + +Firmitas treats architecture as a continuous, decision-centric activity that must remain visible, reviewable, and evolvable throughout the life of a product. Architecture Decision Records (ADRs) and living design artefacts are the primary mechanisms by which this happens. + +This chapter formalises how Firmitas uses ADRs and living design to turn architecture from an implicit, fragile knowledge base into an explicit, resilient system. + + + +### Architecture as a Decision System + +Every meaningful architectural choice encodes assumptions about: +- change frequency +- ownership boundaries +- failure modes +- performance constraints +- cost and risk tolerance + +When these assumptions remain implicit, they: +- become invisible constraints +- are rediscovered through failure +- are defended emotionally rather than rationally + +Firmitas reframes architecture as a **decision system**, where: +- decisions are surfaced explicitly +- trade-offs are documented +- and rationale is preserved over time + +ADRs are not bureaucracy. They are memory. + + + +### Why Traditional Architecture Documentation Fails + +Traditional architecture documentation often fails because it: +- attempts to be comprehensive rather than decision-focused +- is produced upfront and decays rapidly +- is detached from actual delivery work +- serves compliance rather than understanding + +The result is shelfware that: +- no one trusts +- no one updates +- and no one uses when it matters + +Firmitas deliberately rejects “big design up front” and “big documents later”. +Instead, it optimises for **just enough design, captured at the moment of decision**. + + + +### What an Architecture Decision Record Is (and Is Not) + +An ADR is a lightweight, structured record of a **significant architectural decision**. + +It exists to answer: +- What decision did we make? +- Why did we make it? +- What alternatives did we consider? +- What trade-offs did we accept? +- What are the consequences? + +An ADR is: +- narrow in scope +- explicit about context +- honest about uncertainty + +An ADR is not: +- a design specification +- a justification after the fact +- a permanent truth + +Decisions can be revisited. ADRs make that possible without rewriting history. + + + +### When to Create an ADR + +Firmitas does not mandate ADRs for every choice. +They are used when a decision: +- is hard to reverse +- affects multiple teams or systems +- introduces long-term constraints +- carries significant risk or cost +- is likely to be questioned later + +Examples include: +- architectural patterns and styles +- protocol or platform selection +- data ownership boundaries +- build vs buy decisions +- deployment and operational models + +The trigger is **decision significance**, not hierarchy or ceremony. + + + +### ADRs as Socio-Technical Artefacts + +ADRs are not just technical records. +They are socio-technical artefacts that: +- shape how teams communicate +- clarify who decides what +- reduce conflict by making trade-offs explicit + +Well-written ADRs: +- depersonalise disagreement +- enable asynchronous collaboration +- reduce reliance on tribal knowledge + +They shift conversations from “who decided this?” to “under what assumptions did we decide this?”. + + + +### Living Design Over Static Blueprints + +Firmitas pairs ADRs with **living design artefacts**. + +Living design means: +- diagrams evolve as the system evolves +- documentation reflects reality, not aspiration +- design artefacts are version-controlled +- updates happen alongside code changes + +Common living design artefacts include: +- C4 diagrams at appropriate levels +- interface definitions +- deployment views +- data flow diagrams +- failure and resilience models + +The goal is shared understanding, not exhaustive description. + + + +### Diagrams as Thinking Tools, Not Decoration + +In Firmitas, diagrams exist to: +- reduce cognitive load +- support reasoning +- expose coupling and assumptions + +They are not marketing material. + +Good diagrams: +- answer a specific question +- target a defined audience +- avoid false precision +- are easy to update + +Bad diagrams attempt to explain everything and end up explaining nothing. + + + +### Version Control and Review of Design Decisions + +All ADRs and living design artefacts are: +- stored in version control +- reviewed via pull requests +- evolved incrementally + +This achieves several things: +- design changes are auditable +- rationale is preserved alongside implementation +- discussion happens close to the decision +- ownership is explicit + +Design review becomes part of normal engineering flow, not a separate governance ritual. + + + +### ADRs and Governance Without Bureaucracy + +Firmitas uses ADRs to enable **lightweight governance**. + +Rather than central approval boards: +- guardrails are expressed through principles +- significant deviations are documented via ADRs +- accountability is achieved through transparency + +This allows: +- decentralised decision-making +- central visibility +- and post-hoc review without control theatre + +Governance emerges from shared understanding, not enforcement. + + + +### Revisiting and Superseding Decisions + +Systems change. +Assumptions expire. +Constraints shift. + +Firmitas expects ADRs to be: +- revisited when context changes +- superseded when better options emerge +- referenced when diagnosing problems + +A superseded ADR is not a failure. +It is evidence of learning. + +What matters is that change is **conscious**, not accidental. + + + +### Architectural Integrity Over Time + +Architectural decay rarely comes from bad initial design. +It comes from: +- unrecorded decisions +- local optimisations +- silent trade-offs +- time pressure without reflection + +ADRs and living design provide friction in the right places: +- they slow down irreversible decisions just enough +- they force trade-offs into the open +- they preserve intent across team and time boundaries + +This protects long-term system health without paralysing delivery. + + + +### Leadership Responsibilities Around Architecture + +Engineering leaders are responsible for: +- ensuring architectural decisions are visible +- protecting time for design thinking +- discouraging undocumented shortcuts +- valuing clarity over heroics + +They do not need to make every decision. +They must ensure that decisions are made **well, consciously, and transparently**. + +Architecture fails most often when leadership treats it as someone else’s problem. + + + +### Summary + +In Firmitas: +- architecture is a continuous decision process +- ADRs capture intent, trade-offs, and consequences +- living design maintains shared understanding +- governance is achieved through transparency +- learning is preserved over time + +Architecture is not about predicting the future. +It is about **making today’s decisions survivable tomorrow**. + +The next chapter builds directly on this foundation by formalising **documentation itself as a delivery enabler**, rather than an overhead. + +## Documentation as an Enabler (Docs as Code) + +Documentation is not a by-product of engineering. +It is not evidence created after the fact. +It is not something to be “kept in sync” if time allows. + +In Firmitas, documentation is treated as a **first-class delivery mechanism**. +It is part of the socio-technical system through which intent, decisions, risk, quality, and learning flow. + +This chapter defines how Firmitas reframes documentation from overhead into leverage through a **Docs-as-Code** approach. + + + +### Why Documentation Fails in Most Organisations + +Most documentation fails for systemic reasons, not because engineers are careless. + +Common failure modes include: +- documentation created to satisfy a gate or audit, then abandoned +- large documents written upfront, divorced from delivery +- ownership assigned to “someone else” +- tools optimised for publishing rather than thinking +- documentation treated as static truth rather than evolving knowledge + +The result is predictable: +- engineers stop trusting documentation +- decisions move into chat logs and heads +- audits become painful archaeology exercises +- organisations relearn the same lessons repeatedly + +Firmitas starts from a different premise: +**documentation exists to support decision-making, learning, and coordination under uncertainty**. + + + +### Documentation as a Socio-Technical System + +Documentation is not just text. +It is a socio-technical system spanning: +- people and incentives +- tools and workflows +- governance and quality expectations +- organisational memory + +Every documentation system encodes assumptions about: +- who is allowed to change what +- what level of precision is expected +- how disagreement is resolved +- what “good enough” looks like + +Firmitas explicitly designs this system rather than letting it emerge accidentally. + + + +### The Core Principle: Documentation Must Live Where Work Lives + +In Firmitas: +- documentation lives alongside code +- documentation changes follow the same workflow as code +- documentation is version-controlled +- documentation is reviewed, not approved in isolation + +This has several effects: +- documentation stays close to reality +- changes are incremental, not heroic +- ownership is clear +- history is preserved automatically + +If a document cannot be changed with a pull request, it will decay. + + + +### What “Docs as Code” Means in Practice + +Docs-as-Code is not a tool choice. +It is a way of working. + +Practically, it means: +- text-based formats (Markdown, AsciiDoc) by default +- diagrams as text where possible (Mermaid, PlantUML) +- Git as the source of truth +- CI pipelines for validation and publishing +- pull requests for review and discussion + +This aligns documentation with the same delivery system engineers already understand. + + + +### Documentation Types in Firmitas + +Firmitas deliberately limits the number of documentation types, focusing on **decision-bearing artefacts**. + +Key categories include: + +#### Vision and Intent +Documents that explain: +- why the product exists +- who it serves +- what success means + +These are concise, narrative, and stable. + +#### Requirements and Constraints +Statements of: +- functional intent +- non-functional constraints +- regulatory and safety obligations + +They evolve, but changes are explicit and traceable. + +#### Architectural and Design Artefacts +Living views of: +- system structure +- interfaces +- deployment +- failure modes + +Paired with ADRs to preserve rationale. + +#### Verification, Validation, and Evidence +Test plans, results, and traceability that: +- demonstrate conformance +- support audits +- enable learning + +These are generated continuously, not assembled at the end. + + + +### Documentation and Cognitive Load + +Documentation exists to **reduce cognitive load**, not increase it. + +Firmitas applies the same principles here as for team design: +- minimise duplication +- avoid unnecessary precision +- surface only what matters to the audience + +This means: +- different views for different needs +- no single “master document” +- clarity over completeness + +A document that tries to answer every question answers none well. + + + +### Documentation and Decision Transparency + +Documentation is the primary mechanism for decision transparency. + +In Firmitas: +- significant decisions are documented explicitly +- rationale is preserved +- alternatives are visible +- consequences are acknowledged + +This reduces: +- repeated debates +- personalisation of disagreement +- institutional memory loss + +It also enables new team members to onboard into *why*, not just *what*. + + + +### Docs as Code and Governance Without Friction + +Docs-as-Code enables governance without heavyweight process. + +Instead of: +- formal review boards +- static templates +- central approval queues + +Firmitas relies on: +- shared standards +- visible changes +- peer review +- automated checks + +Governance becomes an emergent property of the system, not an external control layer. + + + +### Documentation in Regulated and Safety-Critical Contexts + +Firmitas does not reject formal documentation. +It rejects **wasteful documentation**. + +Docs-as-Code works particularly well in regulated environments because: +- version history is automatic +- traceability can be generated, not curated +- evidence is produced as work happens +- audits become queries, not archaeology + +Formal outputs (PDFs, baselined documents) are treated as **snapshots of a living system**, not the system itself. + + + +### Documentation, Slack, and Long-Term Performance + +High-quality documentation requires slack. + +When teams are permanently overloaded: +- documentation degrades first +- decisions go undocumented +- risk accumulates silently + +Firmitas treats slack as a **system investment**: +- time to explain +- time to reflect +- time to improve artefacts + +This is not inefficiency. +It is what allows systems to remain understandable and adaptable over time. + + + +### Leadership Responsibilities Around Documentation + +Leaders are responsible for: +- valuing clarity over volume +- protecting time for documentation +- modelling good documentation behaviour +- resisting performative documentation demands + +When leaders treat documentation as “admin”, teams follow. +When leaders treat it as infrastructure, teams invest. + + + +### Anti-Patterns to Avoid + +Firmitas explicitly warns against: +- documentation owned by a single role +- “final” documents +- copy-paste templates without intent +- documentation created purely for audits +- tools chosen for aesthetics over integration + +Each of these undermines trust and long-term system health. + + + +### Summary + +In Firmitas: +- documentation is part of the delivery system +- Docs-as-Code aligns documentation with engineering flow +- living artefacts preserve intent and learning +- governance emerges from transparency +- slack enables quality documentation + +Documentation is not about recording the past. +It is about **enabling better decisions in the future**. + +The next chapter builds on this by examining how **toolchains and automation** shape the socio-technical system that Firmitas operates within. + +## Toolchains, Automation, and Socio-Technical Fit + +Tooling does not merely support delivery. +It shapes behaviour, constrains choices, encodes assumptions, and silently defines how work actually gets done. + +In Firmitas, toolchains and automation are treated as **socio-technical systems**, not neutral utilities. Poorly chosen tools can undermine autonomy, distort incentives, and increase cognitive load. Well-chosen tools amplify good practices, reinforce flow, and make quality the default outcome rather than an act of heroism. + +This chapter defines how Firmitas approaches tools and automation deliberately, pragmatically, and without falling into the tool-first trap. + + + +### The Tool Fallacy + +Most organisations overestimate the impact of tools and underestimate the systems they operate within. + +Common failure patterns include: +- adopting tools before clarifying purpose +- forcing teams to adapt to tools rather than the reverse +- mistaking visibility for control +- standardising tools to solve organisational dysfunction + +Tools do not fix broken systems. +They tend to **amplify whatever already exists**, good or bad. + +Firmitas explicitly rejects tool-led transformation. + + + +### Tools as Behaviour-Shaping Systems + +Every tool encodes assumptions about: +- who is allowed to act +- when decisions can be made +- what “good” looks like +- what is visible and what is hidden + +For example: +- ticket systems encode assumptions about predictability and work decomposition +- CI pipelines encode assumptions about ownership and quality gates +- documentation tools encode assumptions about authorship and authority + +Firmitas treats tools as **policy embedded in software**, and therefore subject to the same scrutiny as any other system design decision. + + + +### Socio-Technical Fit Over Tool Uniformity + +Firmitas prioritises **socio-technical fit** over uniformity. + +This means: +- different teams may use different tools +- standardisation is justified only where it reduces friction +- diversity is acceptable where it increases effectiveness without harming the wider system + +Uniformity is a cost, not a virtue. +It must earn its place. + +The guiding question is always: +> Does this tool reduce cognitive load and improve flow at the system level? + + + +### Tooling and Autonomy + +Tools can either enable autonomy or quietly destroy it. + +Autonomy-supporting tools: +- allow teams to make local decisions +- provide fast feedback +- minimise external approvals +- make consequences visible + +Autonomy-eroding tools: +- introduce central queues +- require permission for routine actions +- hide system state +- reward compliance over outcomes + +Firmitas favours tools that **extend team capability** rather than increase oversight. + + + +### Automation as Risk Reduction, Not Speed Theatre + +Automation is often sold as a way to “go faster”. +Firmitas frames it differently. + +Automation exists primarily to: +- reduce variability +- catch errors early +- make quality repeatable +- free human capacity for judgement and creativity + +Speed is a side-effect of reliability, not the goal. + +Automating a broken process simply allows you to fail faster and more consistently. + + + +### Continuous Integration as a System Health Signal + +CI is not a build server. +It is a **system health indicator**. + +In Firmitas: +- broken builds signal upstream problems +- flaky tests indicate architectural or ownership issues +- long pipelines reveal coupling and batch size problems + +CI pipelines are diagnostic tools, not just delivery mechanisms. + +Leaders should treat CI failures as **system feedback**, not team failure. + + + +### Toolchains Across Disciplines + +Firmitas explicitly acknowledges that: +- software, firmware, hardware, and mechanical disciplines use different tools +- forcing artificial convergence increases friction +- integration happens through interfaces, not tool sameness + +The goal is not a single toolchain, but **interoperable toolchains**: +- shared identifiers +- traceable artefacts +- clear ownership boundaries + +Integration is achieved through information flow, not tool mandates. + + + +### Documentation, Tooling, and Docs as Code + +Documentation tools are part of the delivery system. + +Firmitas favours: +- text-first formats +- version-controlled documentation +- automation for publishing and validation +- low-friction contribution paths + +Tools that require specialised roles to update documentation create bottlenecks and decay. + +Docs-as-Code only works when the tooling lowers the barrier to contribution. + + + +### Observability, Feedback, and Learning + +Toolchains should make system behaviour observable. + +This includes: +- build and test results +- deployment state +- operational telemetry +- incident data + +Observability supports: +- fast feedback +- evidence-based decision-making +- blameless learning + +A system that cannot be observed cannot be responsibly governed. + + + +### Slack, Tooling, and Long-Term Effectiveness + +Good tooling requires slack. + +Teams under constant pressure: +- bypass automation +- disable checks +- accumulate fragile workarounds + +Firmitas explicitly protects time for: +- improving pipelines +- refactoring automation +- reducing toil + +This aligns directly with the concept of **slack as a prerequisite for resilience**, not a sign of inefficiency. + + + +### Leadership Responsibilities Around Tooling + +Leaders are responsible for: +- resisting tool-driven change narratives +- ensuring tools serve clear purposes +- removing tools that add friction without value +- funding maintenance and improvement of automation + +Neglected toolchains become invisible technical debt with real organisational cost. + + + +### Anti-Patterns to Avoid + +Firmitas explicitly warns against: +- mandating tools to solve trust problems +- measuring productivity through tool activity +- building approval gates into automation +- conflating standardisation with maturity +- outsourcing thinking to dashboards + +Each of these creates false confidence and systemic brittleness. + + + +### Summary + +In Firmitas: +- tools are socio-technical systems +- automation exists to reduce risk and cognitive load +- socio-technical fit matters more than uniformity +- observability enables learning +- slack enables sustainable automation + +Toolchains should make the *right* thing easier, the *wrong* thing harder, and the *system* more understandable over time. + +The next section shifts focus from practice to assurance, examining **quality engineering and regulation as leadership responsibilities**, not compliance exercises. diff --git a/content/firmitas/appendices/i-quality-regulation-and-assurance.md b/content/firmitas/appendices/i-quality-regulation-and-assurance.md new file mode 100644 index 0000000..954d59a --- /dev/null +++ b/content/firmitas/appendices/i-quality-regulation-and-assurance.md @@ -0,0 +1,1190 @@ +--- +title: "Appendix I: Quality, Regulation, and Assurance" +--- + +# Appendix I: Quality, Regulation, and Assurance + +Quality, regulation, and assurance are where many delivery frameworks quietly fail. Either they are treated as afterthoughts, bolted on once “real work” is finished, or they dominate delivery to the point where learning, ownership, and adaptability collapse. this appendix exists to establish a different position: **quality and assurance are system properties that must be designed in, governed deliberately, and led actively**. + +Firmitas treats quality as inseparable from leadership, architecture, and delivery. It is not the responsibility of a single role, phase, or function. It emerges from how requirements are framed, how decisions are made, how work is verified, and how risk is surfaced and managed over time. In this sense, quality is a consequence of system design, not individual diligence. + +This section makes explicit that **regulation is not the enemy of good engineering**. Poorly interpreted regulation is. Many organisations struggle not because standards such as ISO 9001, IEC 62304, ISO 26262, or sector-specific defence and safety regimes are unreasonable, but because they are applied mechanically, without understanding their intent. this appendix focuses on aligning Firmitas’s artefacts and practices to regulatory *outcomes*, not checklist compliance. + +Assurance is treated as an ongoing activity, not a final gate. Evidence is generated continuously through living artefacts, traceability, automated verification, and transparent decision records. This allows Firmitas to satisfy audit and certification requirements without creating parallel documentation systems or slowing delivery unnecessarily. + +A central theme of this section is **proportionality**. Not all products require the same level of rigour. Not all risks deserve equal treatment. Firmitas supports lightweight, balanced, and regulated operating profiles precisely so that assurance effort scales with risk, impact, and context. Over-assurance wastes capacity and destroys slack. Under-assurance creates latent failure. Leadership judgement is required to navigate that boundary. + +this appendix also clarifies roles and responsibilities. Quality is not delegated away from leadership, and assurance is not synonymous with policing. Engineers, architects, testers, managers, and auditors each play distinct but complementary roles within a coherent assurance system. When those roles are misaligned, quality becomes performative rather than real. + +By the end of this section, the reader should understand: +- How Firmitas embeds quality into everyday delivery +- How regulatory standards map naturally onto Firmitas artefacts +- How to generate assurance evidence continuously, not retrospectively +- How to scale assurance effort without destroying flow or ownership + +this appendix reframes quality and regulation as **enablers of trust**, not constraints on delivery. When designed correctly, they allow organisations to move faster with confidence, reduce systemic risk, and sustain long-term product integrity. + +## Quality Engineering as a Leadership Discipline + +Quality is not a phase. +It is not a department. +It is not a gate at the end of delivery. + +Quality is a **system property**, and like all system properties, it is shaped primarily by leadership decisions, organisational design, and delivery structure. Firmitas positions quality engineering as a **leadership discipline**, not a compliance function and not something that can be delegated away once targets are set. + +This chapter makes that responsibility explicit. + + + +### Why Quality Fails in Otherwise Competent Organisations + +Most quality failures do not occur because engineers do not care about quality. +They occur because the surrounding system makes quality difficult, invisible, or irrational. + +Common systemic causes include: +- incentives that reward speed over correctness +- late discovery of defects due to long feedback loops +- separation of “delivery” from “assurance” +- quality treated as documentation rather than behaviour +- leadership signals that schedule matters more than integrity + +In such systems, individuals behave rationally, even when outcomes are poor. + +Firmitas starts from a different assumption: +**if quality is important, the system must make quality the easiest path**, not the hardest. + + + +### Quality as an Emergent Property of Systems + +Quality does not emerge from testing alone. +It emerges from the interaction of: +- people and skills +- architecture and design decisions +- feedback loops +- tooling and automation +- organisational incentives +- time and slack + +Treating quality as a downstream activity ignores where most defects are actually introduced: **upstream decisions made under pressure and uncertainty**. + +Firmitas therefore treats quality as something that must be designed into: +- leadership behaviour +- delivery flow +- decision-making structures +- and system architecture + + + +### Leadership Accountability for Quality + +In Firmitas, leadership accountability for quality is explicit and non-negotiable. + +Leaders are responsible for: +- setting quality expectations that are credible, not aspirational +- aligning incentives so quality is rewarded, not punished +- protecting time for verification, refactoring, and learning +- refusing to trade integrity for short-term optics + +Quality failures are leadership failures first, technical failures second. + +This does not mean leaders write test cases. +It means they **shape the conditions under which quality is either possible or impossible**. + + + +### Quality Engineering vs Quality Control + +Firmitas draws a hard distinction between: +- **quality control**, and +- **quality engineering**. + +Quality control focuses on: +- detection +- inspection +- catching defects + +Quality engineering focuses on: +- prevention +- system design +- reducing defect introduction in the first place + +Both have a role, but quality engineering dominates long-term outcomes. + +Testing finds bugs. +Quality engineering changes the system so fewer bugs are created. + + + +### Shifting Quality Left Without Creating Chaos + +“Shift left” is often misunderstood as “test earlier”. +Firmitas interprets it more broadly. + +Shifting quality left means: +- making requirements clearer +- making assumptions explicit +- validating designs before implementation +- using prototypes to surface risk +- designing architectures that are testable and observable + +This reduces rework and increases learning speed. +It does not require heavyweight process. +It requires **intentional leadership choices**. + + + +### Verification, Validation, and Continuous Evidence + +Firmitas distinguishes clearly between: +- **verification**: did we build the thing right? +- **validation**: did we build the right thing? + +Both are continuous activities, not end-stage events. + +Quality evidence is generated: +- as work happens +- through automation where possible +- and through disciplined review where judgement is required + +Evidence is not “prepared for audit”. +It is a natural by-product of a well-functioning system. + + + +### Quality, Architecture, and Change Cost + +Architecture and quality are inseparable. + +Architectural decisions determine: +- how easy systems are to test +- how failures propagate +- how safely changes can be made +- how observable behaviour is in production + +Firmitas explicitly connects: +- modularity with defect containment +- clear interfaces with verification ease +- loose coupling with safer evolution + +Poor architecture guarantees quality pain, no matter how much testing follows. + + + +### Quality Across Disciplines + +In integrated products, quality spans: +- software correctness +- firmware timing and reliability +- hardware tolerances and durability +- mechanical fit, form, and function +- system-level behaviour under stress + +Firmitas rejects siloed quality ownership. + +Quality engineering is: +- cross-disciplinary +- system-oriented +- and coordinated through shared intent and artefacts + +No discipline gets to “throw quality over the wall”. + + + +### Quality in Regulated and Safety-Critical Contexts + +In regulated environments, quality has formal expression through standards and audits. +Firmitas does not resist this. +It reframes it. + +Instead of: +- compliance-driven documentation +- late-stage evidence creation +- defensive audit preparation + +Firmitas emphasises: +- living assurance +- continuous traceability +- evidence generated through normal work + +This reduces audit stress while increasing real quality. + + + +### Quality, Slack, and Sustainability + +Sustained quality is impossible without slack. + +Teams with no slack: +- cut corners +- skip verification +- defer refactoring +- normalise risk + +Firmitas treats slack as: +- a quality enabler +- a risk control mechanism +- and a leadership responsibility + +This aligns directly with long-term performance and resilience. + + + +### Measuring Quality Without Corrupting Behaviour + +Poor metrics destroy quality faster than no metrics at all. + +Firmitas avoids: +- vanity metrics +- individual defect counts +- simplistic “quality scores” + +Instead, it favours: +- defect escape rates +- system reliability trends +- rework levels +- customer impact +- and qualitative signals from teams + +Metrics exist to support learning, not control. + + + +### Common Quality Anti-Patterns + +Firmitas explicitly warns against: +- separating delivery and quality ownership +- treating testing as a role rather than a capability +- quality gates that appear late +- audits that reward paperwork over behaviour +- leaders overruling quality concerns for schedule reasons + +Each of these creates long-term systemic damage. + + + +### Summary + +In Firmitas: +- quality is a system property +- quality engineering is a leadership discipline +- prevention matters more than detection +- architecture and quality are inseparable +- slack enables sustained excellence +- evidence emerges from normal work + +Quality is not about perfection. +It is about **building systems that behave predictably, safely, and honestly under real conditions**. + +The next chapter builds on this foundation by addressing how Firmitas approaches **compliance without paralysis**, preserving agility while meeting formal obligations. + +## Compliance Without Paralysis + +Compliance is often treated as an external force imposed on engineering. +Something to be satisfied. +Something to survive. +Something that inevitably slows everything down. + +Firmitas rejects that framing. + +Compliance is not the opposite of agility. +Poorly designed systems are. + +This chapter explains how Firmitas enables **compliance without paralysis** by treating regulation, standards, and assurance as system constraints to be engineered into the delivery model, not layered on top after the fact. + + + +### Why Compliance Breaks Delivery Systems + +In many organisations, compliance fails for predictable reasons: + +- Compliance is addressed late, once design and delivery decisions are already locked in. +- Evidence is created retrospectively rather than generated as work happens. +- Responsibility for compliance is separated from responsibility for delivery. +- Standards are interpreted rigidly rather than intelligently. +- Audits become exercises in documentation archaeology. + +The result is a brittle system where: +- teams slow down defensively, +- quality becomes performative, +- and compliance work crowds out learning and improvement. + +Firmitas treats this as a **systems failure**, not a people failure. + + + +### Compliance as a Design Constraint, Not a Phase + +In Firmitas, compliance is treated the same way as: +- safety, +- security, +- performance, +- or cost. + +It is a **design constraint**. + +This means: +- compliance requirements inform architecture early, +- delivery flow is shaped to generate required evidence naturally, +- and verification activities are integrated, not bolted on. + +Compliance stops being a separate stream of work and becomes part of normal engineering. + + + +### Separating Intent from Interpretation + +Standards such as ISO 9001, IEC, or DO-178 do not prescribe how to build systems. +They articulate **intent**. + +Most compliance pain comes from confusing: +- the intent of a standard +- with a particular historical interpretation of it. + +Firmitas explicitly encourages: +- understanding the *why* behind requirements, +- mapping intent to modern delivery practices, +- and challenging outdated interpretations that no longer serve risk reduction. + +This is not non-compliance. +It is responsible compliance. + + + +### Living Evidence Instead of Static Artefacts + +Traditional compliance models rely on static artefacts: +documents frozen in time, assembled for audits. + +Firmitas replaces this with **living evidence**. + +Evidence is generated through: +- version control history, +- pull request discussions, +- automated test results, +- traceability links, +- and operational data. + +Formal artefacts still exist, but they are **snapshots of a living system**, not the system itself. + +This dramatically reduces: +- audit effort, +- document drift, +- and last-minute compliance panic. + + + +### Traceability Without Theatre + +Traceability is often implemented as a reporting obligation. +Firmitas treats it as a **learning and assurance mechanism**. + +Effective traceability: +- connects intent to outcomes, +- supports impact analysis, +- and enables confident change. + +Firmitas avoids traceability theatre by: +- embedding trace links where work happens, +- generating matrices automatically where possible, +- and avoiding manual duplication. + +Traceability exists to support understanding, not to satisfy spreadsheets. + + + +### Continuous Verification and Validation + +Compliance frameworks assume verification and validation. +Firmitas makes them continuous. + +This includes: +- automated verification at every change, +- incremental validation of assumptions, +- and system-level testing aligned with risk. + +By the time formal validation is required, the system has already demonstrated compliance behaviours repeatedly. + +Final validation becomes confirmation, not discovery. + + + +### Audits as System Inspections, Not Trials + +Firmitas reframes audits fundamentally. + +An audit is not: +- a test of obedience, +- or a trial of individuals. + +It is an inspection of the system. + +A healthy Firmitas system allows auditors to: +- follow decisions through time, +- inspect evidence in context, +- and understand how risks are managed. + +This reduces defensiveness and increases trust on both sides. + + + +### Roles and Accountability in Compliant Systems + +Compliance fails when accountability is fragmented. + +Firmitas is explicit: +- leadership owns compliance outcomes, +- teams own compliant behaviour, +- specialists advise and enable, not police. + +Quality, safety, and regulatory experts are: +- embedded in delivery discussions, +- involved early in decision-making, +- and focused on risk reduction, not paperwork generation. + + + +### Balancing Formal Gates and Continuous Flow + +Some compliance contexts require formal gates. +Firmitas does not deny this. + +Instead, it: +- minimises the number of gates, +- ensures gates review evidence, not documents, +- and uses them as decision points, not control theatre. + +Gates exist to protect irreversible commitments, not to slow learning. + + + +### Compliance, Slack, and Ethical Responsibility + +Sustainable compliance requires slack. + +Teams operating permanently at capacity: +- hide problems, +- defer evidence, +- and normalise risk. + +Firmitas explicitly treats slack as: +- a compliance enabler, +- a safety mechanism, +- and an ethical obligation. + +This aligns compliance with long-term performance rather than short-term optics. + + + +### Common Compliance Anti-Patterns + +Firmitas explicitly warns against: +- compliance teams disconnected from delivery, +- documentation created only for audits, +- manual evidence collation, +- treating standards as immutable processes, +- and leaders overriding compliance concerns for schedule reasons. + +Each of these increases long-term risk, even if short-term delivery appears faster. + + + +### Summary + +In Firmitas: +- compliance is a system property +- standards constrain design, not creativity +- evidence is generated continuously +- audits inspect systems, not people +- leadership owns compliant outcomes +- slack enables ethical, sustainable assurance + +Compliance without paralysis is not achieved by doing *less* compliance. +It is achieved by **designing systems that make compliance the natural outcome of good engineering**. + +The next chapter builds directly on this by making the mapping explicit, showing how Firmitas aligns with formal standards without distortion. + +## Mapping Firmitas to Standards +*(ISO 9001, IEC 62304 / 61508, DO-178C, and Related Frameworks)* + +This chapter makes something explicit that is often left implicit or hand-waved: + +**Firmitas is not “instead of” standards. +Firmitas is how standards are satisfied without distorting delivery.** + +Where many organisations bolt compliance on top of delivery, Firmitas integrates assurance into the system design itself. This chapter shows how Firmitas aligns with formal standards **by intent**, not by ritual, and why that distinction matters. + + + +### Why Mapping Matters (and Why It Is Often Done Poorly) + +Standards do not fail organisations. +**Interpretations do.** + +Most compliance pain comes from: +- treating standards as prescriptive processes +- assuming documentation equals assurance +- mapping activities instead of outcomes +- creating parallel “compliance systems” disconnected from real work + +This produces: +- duplicate effort +- false confidence +- and brittle audit artefacts + +Firmitas takes a different approach. + +It maps: +- **system behaviours to standard intent** +- **evidence generation to real work** +- **governance to risk, not ceremony** + +The result is *stronger* compliance with *less* friction. + + + +### The Core Principle: Intent Over Form + +All major standards share common intent, regardless of domain: + +- clarity of purpose +- controlled change +- demonstrable quality +- risk management +- accountability +- traceability +- learning from failure + +Firmitas explicitly designs for these intents. + +It does **not**: +- mandate document formats +- prescribe stage-gated sequences +- require heavyweight artefacts unless risk demands them + +Instead, it ensures the system can always answer: +- *Why was this decision made?* +- *How do we know this is correct?* +- *What happens if it changes?* +- *Where is the evidence?* +- *Who is accountable?* + +Those answers are what auditors are actually looking for. + + + +### ISO 9001 – Quality Management Systems + +**ISO 9001 focuses on system capability, not technical detail.** + +Key ISO 9001 intents and Firmitas alignment: + +**Leadership & Accountability** +- ISO requires leadership ownership of quality. +- Firmitas explicitly places quality and compliance as leadership responsibilities (Chapters 39–40). + +**Process Control** +- ISO requires defined, controlled processes. +- Firmitas provides *guardrails*, not rigid workflows, ensuring consistency without rigidity. + +**Continuous Improvement** +- ISO mandates learning and improvement. +- Firmitas embeds feedback loops, retrospectives, and incident learning as system features. + +**Evidence-Based Decision Making** +- ISO requires objective evidence. +- Firmitas generates evidence continuously through: + - version control + - test automation + - ADRs + - delivery metrics + +ISO 9001 compliance emerges naturally from a functioning Firmitas system, rather than being enforced externally. + + + +### IEC 62304 / IEC 61508 – Safety-Critical Systems + +Safety standards raise the bar. +Firmitas meets it by **making safety a system concern, not a documentation exercise**. + +Key intents and alignment: + +**Risk Management** +- Firmitas treats risk as a first-class input to planning, prioritisation, and design. +- Hazards drive architecture, verification depth, and gating decisions. + +**Traceability** +- Requirements → design → implementation → verification is maintained as living traceability, not static matrices. + +**Separation of Concerns** +- Safety-related functions are architecturally isolated where required. +- This aligns with both safety integrity levels and fault containment expectations. + +**Verification Rigor** +- Verification depth scales with risk. +- Formal methods and additional controls are applied where justified, not universally. + +Firmitas supports safety-critical delivery **without flattening everything into worst-case bureaucracy**. + + + +### DO-178C and Similar Aviation Standards + +DO-178C is often cited as incompatible with modern delivery. +This is largely untrue. + +DO-178C cares about: +- determinism +- traceability +- verification evidence +- configuration control +- independence of review + +Firmitas addresses these through: +- strong configuration management +- explicit decision records +- independent review via pull requests and design forums +- automated and repeatable verification pipelines +- controlled baselining at defined transition points + +What Firmitas avoids is **false sequencing**. +Learning happens continuously. +Formal acceptance happens deliberately. + + + +### Traceability: What Standards Actually Require + +Standards do not require: +- giant spreadsheets +- manual trace updates +- duplicated artefacts + +They require: +- demonstrable linkage +- impact awareness +- controlled change + +Firmitas implements traceability as: +- lightweight identifiers +- tool-assisted linking +- automation where possible +- human review where judgement matters + +Traceability supports **engineering confidence first**, audit confidence second. + + + +### Configuration Management and Change Control + +Firmitas aligns strongly with configuration control expectations by default: + +- version control is the system of record +- changes are reviewed, discussed, and approved +- baselines are explicit and intentional +- rollback is possible and understood + +Change control is not a board. +It is a **capability**. + +Formal change boards exist only where risk justifies them. + + + +### Evidence, Audits, and “Show Me” + +Auditors do not want more documents. +They want clearer stories. + +Firmitas enables this by making it easy to: +- trace decisions over time +- inspect how quality is assured +- see how risk is managed +- understand how learning happens + +An audit becomes a guided walk through the system, not a defensive performance. + + + +### Common Mapping Anti-Patterns + +Firmitas explicitly avoids: +- “one artefact per clause” thinking +- mapping tools instead of behaviours +- freezing documents to look compliant +- parallel compliance teams +- retrofitting evidence + +Each of these increases cost while reducing actual assurance. + + + +### Summary + +Firmitas does not weaken standards. +It **fulfils their intent more honestly**. + +- Standards describe *what must be true* +- Firmitas designs *systems that make it true* + +Compliance emerges from: +- leadership accountability +- clear architecture +- disciplined delivery +- continuous evidence +- and sustainable pace + +The next chapter builds on this by addressing **how evidence is curated, presented, and sustained over time**. + +## Evidence, Auditability, and Living Assurance + +This chapter addresses a recurring failure mode in engineering organisations: + +**The belief that assurance is something you prepare for, rather than something you continuously generate.** + +Firmitas rejects the idea that evidence is an output created at the end of delivery, or something assembled reactively for audits. Instead, it treats evidence as a **by-product of a healthy system**, generated naturally through disciplined work, clear decisions, and continuous learning. + +Auditability is not a phase. +Assurance is not a role. +Evidence is not paperwork. + +They are all **system properties**. + + + +### The Core Shift: From Compliance Theatre to Living Assurance + +Most organisations experience audits as disruptive, stressful, and adversarial. This is not because auditors are unreasonable, but because the system has been optimised to *appear* compliant rather than *be* controlled. + +Common symptoms include: +- evidence created after the fact +- documents that do not reflect reality +- parallel “quality” systems disconnected from delivery +- heroics before audits +- brittle artefacts that collapse under scrutiny + +Firmitas reframes assurance around a simple principle: + +> **If you cannot explain how a decision was made, how quality was assured, and how risk was managed at any point in time, the system is already failing.** + +Living assurance means: +- evidence exists because the work was done properly +- audits become inspection, not performance +- confidence is continuous, not episodic + + + +### What Auditors Actually Need (and What They Don’t) + +Auditors rarely ask for: +- long documents +- polished templates +- heavyweight reports + +They ask for: +- clarity +- consistency +- traceability +- accountability +- evidence of control + +Firmitas ensures the system can always answer: +- What was the intent? +- Who decided? +- What options were considered? +- What risks were identified? +- How was it verified? +- What changed, and why? +- What was learned? + +When those answers are embedded in the way work is done, audits become straightforward. + + + +### Evidence as a Continuous Output of the System + +In Firmitas, evidence is produced continuously through normal activity: + +- Version control records *what* changed and *when* +- Pull requests capture *review*, *challenge*, and *approval* +- ADRs preserve *decision rationale* +- Test automation produces *objective verification data* +- CI pipelines provide *repeatable, time-stamped proof* +- Incident reviews generate *learning evidence* +- Metrics show *system behaviour over time* + +None of these exist *for* assurance. +They exist for **engineering effectiveness**. +Assurance simply consumes them. + + + +### Living Artefacts and Evidence Freshness + +A critical distinction in Firmitas is between: +- **living artefacts**, and +- **frozen evidence** + +Living artefacts: +- evolve with the system +- reflect current understanding +- support day-to-day decision-making + +Frozen evidence: +- is created intentionally at defined points +- represents a declared baseline +- supports formal acceptance, release, or certification + +Firmitas makes baselining explicit: +- not everything is frozen +- not everything needs to be +- freezing is a *decision*, not a default + +This avoids the trap of: +- outdated documentation +- false confidence +- change avoidance driven by paperwork fear + + + +### Traceability Without Fragility + +Traceability in Firmitas is: +- intentional +- risk-driven +- proportionate + +It is not: +- exhaustive by default +- manually maintained +- decoupled from delivery + +Traceability exists to answer impact questions: +- If this changes, what else is affected? +- If this fails, what risk is exposed? +- If this is audited, what evidence supports it? + +Tooling may assist, but the principle is architectural: +**traceability must follow the flow of value**, not fight it. + + + +### Evidence Ownership and Accountability + +A key failure mode in traditional models is assigning “evidence ownership” to quality teams. + +Firmitas is explicit: +- engineers own the evidence of their work +- leaders own the assurance of the system +- quality functions enable, not police + +Evidence without ownership is meaningless. +Ownership without accountability is dangerous. + +Leadership remains accountable for: +- ensuring evidence exists +- ensuring it is credible +- ensuring it reflects reality + +This aligns assurance directly with leadership responsibility, not delegation. + + + +### Audits as System Inspection, Not Events + +In a Firmitas organisation: +- audits are not “prepared for” +- audits do not require special behaviour +- audits reveal system health, not performance anxiety + +Auditors are shown: +- how work actually flows +- where decisions are made +- how risk is managed +- how quality is assured +- how learning occurs + +The confidence comes from transparency, not polish. + + + +### Incident Evidence and Learning as Assurance + +One of the strongest forms of assurance is **how an organisation responds to failure**. + +Firmitas treats: +- incidents +- near misses +- escapes +- unexpected behaviour + +as evidence of system maturity when handled well. + +Blameless post-incident analysis, documented learning, and systemic improvement demonstrate: +- control +- accountability +- and resilience + +Hiding failure erodes assurance. +Learning from it strengthens trust. + + + +### The Role of Slack in Living Assurance + +Living assurance requires **slack**. + +Without slack: +- evidence is rushed +- reviews are skipped +- learning is deferred +- assurance becomes performative + +Firmitas explicitly recognises slack as: +- a prerequisite for quality +- a buffer against uncertainty +- an enabler of reflection and learning + +Organisations that eliminate slack also eliminate assurance, whether they realise it or not. + + + +### Summary + +Living assurance is not an overlay. +It is a consequence of a well-designed system. + +Firmitas ensures: +- evidence is generated continuously +- auditability is inherent +- quality is visible +- learning is preserved +- leadership accountability is explicit + +When assurance is alive, audits become routine. +When assurance is dead, audits become theatre. + +The next chapter builds on this by addressing **how organisations respond to failure and convert incidents into durable improvement**. + +## Incident Management and Learning Systems + +Incident management is one of the clearest windows into the true health of an engineering system. + +Not how often incidents occur. +Not how quickly teams recover. +But **how the organisation responds, learns, and adapts afterwards**. + +Firmitas treats incidents not as operational failures to be minimised or hidden, but as **signals from the system**. They reveal mismatches between intent and reality, between assumptions and behaviour, between design and use. + +This chapter defines how Firmitas frames incident management as a **learning system**, not a control mechanism. + + + +### Incidents Are System Signals, Not Human Errors + +A foundational premise of Firmitas is this: + +> People behave rationally within the systems they are placed in. + +When incidents occur, the root cause is almost never: +- incompetence, +- negligence, +- or lack of effort. + +Instead, incidents emerge from: +- misaligned incentives, +- unclear ownership, +- brittle architectures, +- overloaded teams, +- poor feedback loops, +- or decisions made under constrained context. + +Firmitas therefore rejects blame-oriented incident handling. Blame suppresses learning. Suppressed learning guarantees recurrence. + + + +### The Purpose of Incident Management in Firmitas + +Incident management exists to serve three purposes: + +1. **Restore service safely and predictably** +2. **Understand what the system revealed** +3. **Improve future system behaviour** + +Speed of recovery matters, but learning matters more. +An organisation that recovers quickly but learns nothing is accumulating invisible risk. + + + +### Separating Response from Learning + +Firmitas explicitly separates: +- **incident response** +- from **incident learning** + +During response: +- clarity, authority, and coordination matter +- decision-making must be simple +- communication must be direct + +During learning: +- reflection, curiosity, and psychological safety matter +- complexity must be explored +- assumptions must be challenged + +Blending the two leads to poor outcomes in both. + + + +### Blamelessness Is Necessary but Not Sufficient + +Firmitas adopts blameless post-incident practices, but with an important clarification: + +Blameless does **not** mean: +- no accountability +- no standards +- no consequences + +It means: +- accountability is assigned to systems, decisions, and structures +- learning is prioritised over punishment +- leadership examines how the system shaped behaviour + +Blamelessness creates the conditions for truth. +Leadership responsibility ensures truth leads to change. + + + +### What a Firmitas Incident Review Looks Like + +A Firmitas incident review focuses on understanding, not verdicts. + +Key questions include: +- What was the system trying to do? +- What assumptions were embedded in design or process? +- What signals were missed or ignored? +- What constraints shaped decisions at the time? +- Where did feedback loops fail? +- What trade-offs were implicitly made? + +The goal is not to identify a single root cause, but to build a **rich, shared understanding** of system behaviour. + + + +### Incidents Across System Types + +Incidents rarely exist in a single domain. + +Firmitas explicitly examines incidents across: +- **Human systems** (fatigue, incentives, trust) +- **Organisational systems** (handoffs, governance, escalation paths) +- **Socio-technical systems** (tooling, CI/CD, observability) +- **Delivery systems** (batch size, validation timing, queues) +- **Technical systems** (architecture, coupling, failure modes) +- **Economic systems** (cost pressure, deadlines, prioritisation) + +Learning that ignores any of these dimensions is incomplete. + + + +### Learning Must Result in System Change + +An incident review that produces a document but no change is worse than useless. +It creates the illusion of improvement while preserving risk. + +Firmitas requires that learning results in at least one of: +- architectural change +- process adjustment +- policy clarification +- tooling improvement +- training or capability development +- incentive realignment + +If nothing changes, the system has not learned. + + + +### Incident Trends and Weak Signals + +Firmitas treats incident analysis as both: +- **event-based**, and +- **trend-based** + +Recurring low-severity issues are often more dangerous than rare major failures. They signal chronic system stress, eroding margins and resilience. + +Leadership is expected to: +- monitor incident patterns +- detect weak signals early +- act before catastrophic failure forces attention + +This is a core leadership responsibility, not an operational afterthought. + + + +### Psychological Safety as a Prerequisite for Learning + +Incident learning collapses without psychological safety. + +If teams fear: +- blame, +- reputation damage, +- or career impact, + +they will: +- hide problems, +- sanitise narratives, +- and stop reporting near misses. + +Firmitas explicitly ties: +- incident learning effectiveness +- to leadership behaviour +- to incentive structures +- to workload sustainability + +A system that punishes honesty cannot learn. + + + +### Slack Enables Reflection and Learning + +Learning takes time. +Time requires slack. + +Firmitas explicitly connects incident learning to the concept of slack: +- time to analyse properly +- space to explore contributing factors +- capacity to implement improvements + +Organisations that eliminate slack also eliminate learning, regardless of how many postmortems they schedule. + + + +### Incidents as Leading Indicators of System Health + +In Firmitas, incidents are not just failures. +They are **diagnostic tools**. + +Handled well, they reveal: +- architectural weaknesses +- organisational misalignments +- governance blind spots +- cultural pathologies + +This makes incident management one of the most valuable leadership instruments available. + + + +### Summary + +In Firmitas: +- incidents are signals, not sins +- learning is the primary objective +- blamelessness enables truth +- leadership owns system change +- slack enables reflection +- resilience is built through learning, not avoidance + +An organisation’s incident practices tell you everything you need to know about its maturity. + +The next section builds on this foundation by examining how **AI, automation, and agent-based systems change the nature of incidents, accountability, and learning**. diff --git a/content/firmitas/appendices/index.md b/content/firmitas/appendices/index.md new file mode 100644 index 0000000..2bc5bf5 --- /dev/null +++ b/content/firmitas/appendices/index.md @@ -0,0 +1,9 @@ +--- +title: Appendices +--- + +# Appendices + +The appendices come from the first edition of Firmitas. Appendices A to G are its reference material. Appendices H and I are its sections on engineering practice across disciplines, and on quality, regulation and assurance. The second edition does not yet cover these subjects in chapters of their own. + +The [Templates](../../templates/index.md) and [Glossary](../../glossary/index.md) sections of this site hold the current templates and terms. diff --git a/content/firmitas/images/cone-of-uncertainty.png b/content/firmitas/images/cone-of-uncertainty.png new file mode 100644 index 0000000..36153b5 Binary files /dev/null and b/content/firmitas/images/cone-of-uncertainty.png differ diff --git a/content/firmitas/images/front-cover.png b/content/firmitas/images/front-cover.png new file mode 100644 index 0000000..2c8859b Binary files /dev/null and b/content/firmitas/images/front-cover.png differ diff --git a/content/firmitas/index.md b/content/firmitas/index.md index a679385..af85f56 100644 --- a/content/firmitas/index.md +++ b/content/firmitas/index.md @@ -5,6 +5,64 @@ author: Mark Bacon # Firmitas -Firmitas: a ways-of-working framework for engineering teams. +**A Framework for Sustainable Engineering Delivery: Designing the Systems That Make Delivery Possible** -This section is moving in from docs-WoW. +![The cover of Firmitas](images/front-cover.png){ width="40%" } + +[[CAPTION:Figure]] Firmitas, by Mark Bacon. + +Most engineering programmes that fail do not fail suddenly. They fail slowly, visibly and predictably, in ways the people closest to the work could see coming. Firmitas is about why that happens and what to do about it. Part One describes how programmes fail. Part Two sets out the framework that responds to it. Part Three is about making it stick. + +© 2025 Mark Bacon. Firmitas is licensed under a [Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International licence](https://creativecommons.org/licenses/by-nc-nd/4.0/). + +## Contents + +| Contents | +| --- | +| [Preface](01-preface.md) | +| [Acknowledgements](02-acknowledgements.md) | +| [Introduction](03-introduction.md) | +| [Part One: Why Programmes Fail](part-1-why-programmes-fail/index.md) | +|     [Chapter 1: The Programme That Failed Predictably](part-1-why-programmes-fail/01-the-programme-that-failed-predictably.md) | +|     [Chapter 2: Estimation: Why It Fails and What Honest Looks Like](part-1-why-programmes-fail/02-estimation-why-it-fails-and-what-honest-looks-like.md) | +|     [Chapter 3: The Commitment That Wasn't Theirs](part-1-why-programmes-fail/03-the-commitment-that-wasnt-theirs.md) | +|     [Chapter 4: What the System Did With the Information](part-1-why-programmes-fail/04-what-the-system-did-with-the-information.md) | +|     [Chapter 5: The Learning That Doesn't Happen](part-1-why-programmes-fail/05-the-learning-that-doesnt-happen.md) | +|     [Chapter 6: Engineering Is a Systems Discipline](part-1-why-programmes-fail/06-engineering-is-a-systems-discipline.md) | +| [Part Two: The Framework](part-2-the-framework/index.md) | +|     [Chapter 7: Leadership Owns the System](part-2-the-framework/07-leadership-owns-the-system.md) | +|     [Chapter 8: Delivery Knowledge Lives Closest to the Work](part-2-the-framework/08-delivery-knowledge-lives-closest-to-the-work.md) | +|     [Chapter 9: Commitments, Estimates, and Honest Plans](part-2-the-framework/09-commitments-estimates-and-honest-plans.md) | +|     [Chapter 10: Risk, Issues, and the Register That Actually Works](part-2-the-framework/10-risk-issues-and-the-register-that-actually-works.md) | +|     [Chapter 11: Requirements: From Customer Need to Testable Specification](part-2-the-framework/11-requirements-from-customer-need-to-testable-specification.md) | +|     [Chapter 12: Slack, Flow, and Why Efficiency Thinking Fails](part-2-the-framework/12-slack-flow-and-why-efficiency-thinking-fails.md) | +|     [Chapter 13: Architecture as Delivery Infrastructure](part-2-the-framework/13-architecture-as-delivery-infrastructure.md) | +|     [Chapter 14: The Delivery Model](part-2-the-framework/14-the-delivery-model.md) | +|     [Chapter 15: Quality as a System Property](part-2-the-framework/15-quality-as-a-system-property.md) | +|     [Chapter 16: Governance, Guardrails, and Decision Authority](part-2-the-framework/16-governance-guardrails-and-decision-authority.md) | +|     [Chapter 17: Teams, Cognitive Load, and Organisation for Flow](part-2-the-framework/17-teams-cognitive-load-and-organisation-for-flow.md) | +|     [Chapter 18: Communication, Information Flow, and Documentation](part-2-the-framework/18-communication-information-flow-and-documentation.md) | +|     [Chapter 19: Technical Debt and Change Management](part-2-the-framework/19-technical-debt-and-change-management.md) | +|     [Chapter 20: Capability, People, and Building for the Long Term](part-2-the-framework/20-capability-people-and-building-for-the-long-term.md) | +|     [Chapter 21: Portfolio Flow and Prioritisation](part-2-the-framework/21-portfolio-flow-and-prioritisation.md) | +|     [Chapter 22: Business Alignment, Economics, and the Customer](part-2-the-framework/22-business-alignment-economics-and-the-customer.md) | +|     [Chapter 23: AI, Automation, and Human Accountability](part-2-the-framework/23-ai-automation-and-human-accountability.md) | +| [Part Three: Making It Stick](part-3-making-it-stick/index.md) | +|     [Chapter 24: The Leader Who Has to Change](part-3-making-it-stick/24-the-leader-who-has-to-change.md) | +|     [Chapter 25: Assessing Readiness and Adopting Progressively](part-3-making-it-stick/25-assessing-readiness-and-adopting-progressively.md) | +|     [Chapter 26: Common Failure Modes and Anti-Patterns](part-3-making-it-stick/26-common-failure-modes-and-anti-patterns.md) | +|     [Chapter 27: Measuring What Actually Matters](part-3-making-it-stick/27-measuring-what-actually-matters.md) | +|     [Chapter 28: What Firmitas Cannot Fix](part-3-making-it-stick/28-what-firmitas-cannot-fix.md) | +|     [Chapter 29: The Long View](part-3-making-it-stick/29-the-long-view.md) | +| [Appendices](appendices/index.md) | +|     [Appendix A: Glossary of Terms](appendices/a-glossary-of-terms.md) | +|     [Appendix B: Firmitas Templates](appendices/b-firmitas-templates.md) | +|     [Appendix C: Architectural Principles for Firmitas](appendices/c-architectural-principles-for-firmitas.md) | +|     [Appendix D: Minimal, Balanced, and Regulated Firmitas Profiles](appendices/d-minimal-balanced-and-regulated-firmitas-profiles.md) | +|     [Appendix E: End-to-End Worked Example](appendices/e-end-to-end-worked-example.md) | +|     [Appendix F: Leadership and Delivery Anti-Patterns Firmitas Is Designed to Prevent](appendices/f-leadership-and-delivery-anti-patterns-firmitas-is-designed-to-prevent.md) | +|     [Appendix G: Bibliography and Further Reading](appendices/g-bibliography-and-further-reading.md) | +|     [Appendix H: Engineering Practice Across Disciplines](appendices/h-engineering-practice-across-disciplines.md) | +|     [Appendix I: Quality, Regulation, and Assurance](appendices/i-quality-regulation-and-assurance.md) | + +[[CAPTION:Table]] The contents of Firmitas. diff --git a/content/firmitas/part-1-why-programmes-fail/01-the-programme-that-failed-predictably.md b/content/firmitas/part-1-why-programmes-fail/01-the-programme-that-failed-predictably.md new file mode 100644 index 0000000..4c039a7 --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/01-the-programme-that-failed-predictably.md @@ -0,0 +1,113 @@ +--- +title: "Chapter 1: The Programme That Failed Predictably" +--- + +# Chapter 1: The Programme That Failed Predictably + +The contract was signed on a Tuesday in October. + +The client needed a complex integrated product - hardware, firmware, and software working together - delivered within fourteen months. The budget was fixed. The penalty clauses were standard. The account team had worked hard to win the business and the commercial director had committed to a date that made the business case work. + +Nobody had spoken to the engineering team before the contract was signed. + +That is not unusual. In most organisations, the sequence runs: bid, win, commit, then tell engineering. The assumption embedded in that sequence - that engineering will find a way to deliver whatever has been sold - is so common that most people inside those organisations no longer notice it. It feels like the natural order of things. Sales sells. Engineering delivers. The gap between what was sold and what is deliverable is a problem for later. + +Later, in this case, arrived at the end of week three. + +--- + +The engineering team spent the first three weeks doing what good engineering teams do when handed a commitment they did not make. They read the contract. They mapped the requirements. They identified the unknowns. They produced an honest assessment of what the programme would actually take. + +The lead systems engineer brought the assessment to the first programme review. It contained a three-point estimate: best case twelve months, most likely seventeen months, worst case twenty-two months. Attached to the estimate was a risk register. Fourteen items. Three rated red. + +The first red item was the hardware-software interface. The specification existed but contained ambiguities that would require resolution with the client before firmware development could proceed meaningfully. Until those ambiguities were resolved, any firmware written against the current specification was written against assumptions that might prove wrong. + +The second red item was a third-party software component the architecture depended on. It had not been evaluated against the programme's specific requirements. The supplier had confirmed it was suitable. The engineering team had not yet been able to verify that claim independently. + +The third red item was the test environment. The programme required a hardware-representative test environment to validate the integrated system. Procuring, building, and qualifying that environment would take time. Until it existed, integration testing would be delayed - and integration testing was where the real risks in this kind of programme almost always surfaced. + +The lead systems engineer presented all of this clearly. The three red items were not hypothetical concerns. They were specific, named, bounded risks, each with a defined impact on the delivery estimate and a defined window within which a decision or action could prevent that impact from materialising. + +The programme board noted the risks. They asked for mitigation plans. They confirmed the delivery date was unchanged. + +The programme moved on to a review of task status. + +--- + +By month four, the hardware-software interface ambiguities had not been resolved. Two rounds of correspondence with the client had produced clarifying questions rather than answers. The firmware team had made the only rational response to sustained ambiguity - they had written against their best interpretation of the specification. That interpretation would later turn out to be wrong in three places. The rework cost six weeks. + +The third-party software component had been evaluated. It was not suitable as specified. It could be made suitable with modifications, but the supplier's timeline for those modifications was longer than the programme could absorb. The architecture required a redesign. The team identified a viable alternative in week eighteen. Implementing it took four weeks and required changes to elements of the system that had already been completed and tested. + +The test environment was procured in month five. Qualification took longer than expected. It was available for integration testing in month seven. By that point the programme was carrying risks that had been visible since week three, accumulating since month one, and never resolved - because the decisions required to resolve them had never been made. + +At month six the programme manager told the client there would be a delay. The revised delivery date was eight months beyond the original commitment. The client was not surprised. They had been watching the slow responses and the revised timelines accumulating in their inbox. They had suspected for some time that the programme was in difficulty. + +The engineering team was not surprised either. They had known since week three. + +The people who were surprised were the ones who had been reading the status reports. + +--- + +The status reports had been mostly green. Task completion rates were tracking reasonably. The risk register had been mostly amber. Nothing in the formal programme reporting had indicated the scale of what was coming. The reporting had been technically accurate in a narrow sense - the tasks that were being tracked were being completed - but it had been describing a programme that existed on paper rather than the programme that existed in reality. + +This is worth pausing on, because it is one of the most consistent patterns in engineering programme failure. The formal reporting system and the informal engineering reality had diverged at month one and continued to diverge throughout. Every engineer on the programme knew this. Most of the programme management layer knew it too, in the way that people know things they have chosen not to act on. Only the people at the top of the governance structure - the people with the authority to make the decisions the programme needed - were reading the green task status and the amber risk register and concluding that the programme was roughly on track. + +The risk register had not lied to them. It had contained three red items since week three. But those three red items had been reviewed, noted, mitigations had been requested, and the conversation had moved on. The register had been used as a reporting artefact rather than a decision instrument. It had told the programme board what the risks were. The programme board had not treated that information as a set of decisions requiring owners, authority, and action. They had treated it as information to be noted. Noted and filed. + +The window to change the outcome had been at week three. Not at month six when the delay was announced. Not at month four when the rework began. At week three, when a lead systems engineer put three specific, actionable risks in front of people who had the authority to act on them, and those people asked for mitigation plans and moved on to task status. + +--- + +After the delay was announced, a lessons-learned exercise was commissioned. + +It concluded that the programme had suffered from insufficient risk management, inadequate requirements definition, and a failure to escalate issues in a timely manner. + +These conclusions were not wrong. But they were incomplete in a way that made them useless - and dangerous. + +The risk register had contained three red items since week three. They had been reviewed at every programme board. They had not been acted on. That is not insufficient risk management. That is risk management that was present, visible, and systematically bypassed in favour of preserving a delivery date that had been set before anyone understood what the programme required. + +The requirements had ambiguities that the engineering team had identified and flagged. The flags had been noted. The ambiguities had not been resolved. That is not inadequate requirements definition. That is a requirements problem that was known, named, and left unresolved because resolving it would have required a difficult conversation with the client at a moment when nobody wanted to have it. + +The issues had been escalated. The lead systems engineer had put three red items in front of the programme board at the first review. The programme board had asked for mitigation plans, confirmed the delivery date, and moved on. That is not a failure to escalate. That is escalation that was received, acknowledged, and not acted upon. + +The lessons-learned exercise produced a set of process improvements. New templates for risk registers. A revised escalation protocol. A requirement for more frequent programme reviews. These changes were implemented on the next programme. + +The next programme failed in the same way. + +--- + +What happened here was not unusual. The same sequence - honest assessment, noted risks, unchanged date, deferred decisions, accumulated failure, surprised senior management, blame distributed downward - plays out across engineering organisations in every sector, at every scale, with a consistency that suggests it is not a series of individual failures but a single systemic one. + +The engineers on this programme were not incompetent. The programme manager was not negligent. The programme board members were not indifferent to the outcome. Everyone involved was doing what their system had shaped them to do. The engineers produced an honest assessment and flagged the risks because that was their job. The programme board noted the risks and moved on because that was what programme boards in their organisation did with risk registers. The commercial director held the date because dates in fixed-price contracts are not moved without significant commercial and reputational cost. The lessons-learned exercise blamed process rather than structure because blaming process produces recommendations that are easier to implement than the ones that would actually change the outcome. + +Each individual in this story behaved rationally within the system they were in. The system produced a predictable failure. The failure was attributed to the individuals. The system was adjusted at the edges. The next programme began. + +--- + +There is a variant of this pattern worth naming, because it produces an additional layer of cost that is rarely counted. + +When an organisation delivers for a third party - a customer, a client under contract - the overrun carries a defined, bounded cost. Contractual penalties, reputational damage, a difficult client relationship. Real costs, but contained ones. The delivering organisation bears those costs and moves on. The failure is painful but it does not reach inside the business and damage its ability to operate. + +When an organisation delivers a product or platform for its own use - building the infrastructure it needs to run as a business - the overrun hits twice. The first hit is the delivery overrun. The second is the operational consequence: every week the platform does not exist is a week the business operates on workarounds, on legacy systems, on manual processes, on capabilities it does not yet have. And when the platform eventually arrives - built under cost pressure, with risks deferred, with quality shortcuts taken to hit a date - it becomes the infrastructure the business depends on. The technical debt is not deferred to a future project. It is baked into the system that now runs the business. The cost of delivery decisions made under pressure becomes an ongoing operational cost that the project budget never captured and the lessons-learned exercise never measured. + +Organisations delivering for their own use tend to feel the consequences of poor delivery more acutely for this reason. They take the full hit - delivery cost, operational lateness cost, and the long-term cost of what was built under pressure - with none of the insulation that the distance between a delivering organisation and an external client provides. That proximity to consequence can, when leadership is willing to act on it, create the conditions for more honest delivery governance. It does not always do so. The same failure patterns occur. But the cost of them is harder to ignore. + +--- + +This book is about why programmes fail in the way this one did - and what it would take to build organisations where that failure pattern is no longer the default. + +The answer is not better risk register templates. It is not more frequent programme reviews. It is not a revised escalation protocol. These are the responses of an organisation that has correctly identified the symptoms and incorrectly identified the cause. + +The cause is structural. The people who made the decisions that determined the outcome of this programme were not the people who bore the consequences of those decisions. The commercial director who committed to the date did not write the firmware. The programme board members who filed the risk register did not spend six weeks on rework. The account team who won the contract did not spend month seven integrating a system in a test environment that should have been available in month two. + +This separation - between the people who make delivery decisions and the people who live with the consequences of those decisions - is not an accident. It is the predictable product of how most engineering organisations are structured. Authority flows upward. Knowledge stays down. The people with the most accurate understanding of what a programme requires have the least authority to act on that understanding. The people with the most authority to act have the least visibility of what the programme actually needs. + +Firmitas is built on the conviction that closing that gap - aligning the authority to make delivery decisions with the knowledge required to make them well - is the primary structural challenge of engineering programme governance. Everything else in this book follows from that conviction. + +The next chapter examines the belief that most commonly prevents that gap from being closed: the assumption that engineers are being conservative, that their estimates are negotiating positions, and that pressure will extract the real number. + +It is almost always wrong. Understanding why - and understanding the specific, measurable damage it causes - is where the structural argument begins. + +--- + +*End of Chapter 1* diff --git a/content/firmitas/part-1-why-programmes-fail/02-estimation-why-it-fails-and-what-honest-looks-like.md b/content/firmitas/part-1-why-programmes-fail/02-estimation-why-it-fails-and-what-honest-looks-like.md new file mode 100644 index 0000000..4570068 --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/02-estimation-why-it-fails-and-what-honest-looks-like.md @@ -0,0 +1,139 @@ +--- +title: "Chapter 2: Estimation: Why It Fails and What Honest Looks Like" +--- + +# Chapter 2: Estimation: Why It Fails and What Honest Looks Like + +The programme in Chapter 1 failed at the moment a commitment was made without an honest estimate. It was compounded at the moment the honest estimate arrived and was not used. + +Understanding why requires understanding what estimation actually is, what it can and cannot achieve, what corrupts it, and what the governance layer's response to it reveals about the system surrounding the programme. + +Estimation is not a precise science. It is a structured act of professional judgement — the best available assessment, at a specific point in time, of what a specific programme requires given what is currently known about it. The quality of an estimate depends on who produces it, what information they have access to, and the environment in which they are working. All three of these factors are shaped by the system surrounding the programme, not just by the capability of the individuals doing the estimating. + +This is the point most often missed in how organisations respond to estimation failures. When a programme overruns its estimate, the failure is attributed to the estimate. The process is improved. The template is revised. The governance is tightened. None of this addresses the cause — because in most cases the estimate was not the failure. The environment that shaped the estimate, and the governance response to it, was. + +--- + +## The cone of uncertainty + +Before examining what corrupts estimation, it is necessary to establish what honest estimation can achieve — and specifically, to be precise about the relationship between how well a programme is understood and how accurately it can be estimated. + +The cone of uncertainty describes that relationship. At the beginning of a programme — before requirements are defined, before architecture is established, before the key technical risks have been investigated — the uncertainty in any estimate is large. Not because the estimators are incompetent, but because the programme is genuinely not yet understood well enough to be estimated precisely. The honest acknowledgement of this state is a wide range. A precise single-point estimate at this stage is not an accurate estimate. It is manufactured precision applied to genuine uncertainty. The uncertainty does not disappear because the governance layer demands a number. It becomes invisible — and invisible uncertainty is the most dangerous kind. + +As the programme progresses — as requirements are defined, architecture is established, and the major technical risks are resolved — the cone narrows. The range between the optimistic and pessimistic cases reduces. By the time the programme is well understood and the significant risks have been investigated, estimates can be substantially more precise. + +![Cone of Uncertainty](../images/cone-of-uncertainty.png) + +[[CAPTION:Figure]] The cone of uncertainty. + +The governance implication is direct. Demanding a precise single-point estimate before requirements are defined is not discipline. It is the demand for certainty that the programme's current state cannot support. The honest response to such a demand is a range — this is what we know, this is what we do not yet know, and this is the range within which the outcome is likely to sit given the current state of understanding. + +Most organisations do not accept range estimates. They accept single-point estimates because single-point estimates can be put in a project plan and tracked against. The consequence is that engineers produce single-point estimates that reflect either genuine precision — when it exists — or manufactured precision that does not, while the uncertainty the cone represents is transferred silently to the risk register or absorbed into the programme's eventual overrun. + +The cone of uncertainty does not disappear because the organisation refuses to acknowledge it. It persists as hidden risk until it surfaces as visible cost. The programme in Chapter 1 was at the wide end of the cone when the committed date was set. Nobody acknowledged the width. The uncertainty was assumed away rather than managed. By month six, it had become an eight-month overrun. + +--- + +## What corrupts estimation — the management side + +### More detail produces false precision, not accuracy + +Detailed task-level estimates feel more rigorous than high-level ones. A plan with two hundred line items appears to have been carefully considered. It is not necessarily more accurate than a plan with twenty. + +Estimation accuracy is determined by how well the scope is understood — not by how granular the task breakdown is. A detailed estimate built on poorly understood scope is precisely wrong rather than approximately right. The false precision of a comprehensive task plan gives the governance layer confidence it has not earned. The plan appears rigorous. The underlying assumptions on which the granular estimates rest have not been validated. + +As McConnell documents in *Software Estimation*, the estimate that feels most precise is often the least reliable, because its precision has been constructed from detail on top of uncertainty that has not been resolved. The wide end of the cone cannot be narrowed by decomposing tasks more finely. It can only be narrowed by resolving the underlying uncertainty — by doing the work of defining requirements, validating the architecture, and investigating the key technical risks. + +A high-level range estimate grounded in genuine scope understanding is more useful to programme governance than a detailed task plan built on scope assumptions. Precision that does not reflect reality is not a virtue. It is a liability that obscures the work still required to earn it. + +### Estimates are not commitments + +Many organisations treat the act of asking for an estimate and receiving one as the creation of a commitment. The engineering team has produced a number. They are now accountable for delivering to it — regardless of what changes after the estimate was made, regardless of whether the conditions it depended on were met, regardless of whether scope changed or identified risks were managed. + +This conflation is one of the primary incentives for producing estimates that are known to be wrong but are defensible. If the estimate will become the commitment regardless of its content, the rational response is to produce a number the team believes it can be held to rather than one that honestly represents what the work requires. + +Estimation and commitment are not the same act. An estimate is the best current assessment of what a programme requires given current understanding. A commitment is an agreement to deliver a specific outcome under specific conditions — a subject addressed fully in Chapter 9. Treating estimates as commitments destroys the conditions for honest estimation, because engineers who know their estimate will be used as a commitment calibrate it to what they can commit to rather than what they honestly assess. + +### Past performance is not this programme + +Estimation by analogy — this programme is similar to the last one, so it will take a similar amount of time — is one of the most common sources of optimistic baselines. Historical data provides useful context. It is not a substitute for assessment of the specific programme. + +The differences between programmes matter at least as much as the similarities. A different team composition changes the estimate. A different regulatory environment changes it. A different level of integration complexity changes it. The estimate that does not account for what is different about this programme from the one being used as the analogue is an estimate of the past programme applied to the present one. It will be wrong by the amount the two programmes differ — which is usually more than the people making the analogy have acknowledged. + +### Parallel working does not compress timelines proportionally + +The belief that adding teams or running work streams in parallel reduces the overall timeline in direct proportion to the resource added is persistent and wrong. Adding people to a programme increases coordination overhead, creates integration risk at every interface, and introduces dependencies that serial working avoids. + +Brooks identified this in 1975: adding people to a late programme makes it later. The communication overhead between team members grows non-linearly with headcount. The integration cost between parallel work streams is not captured in the estimates for the individual streams. The timeline compression from parallelisation is always less than the resource increase suggests — and at high levels of parallelisation, the coordination overhead can increase rather than reduce overall duration. Five decades after *The Mythical Man-Month*, this dynamic continues to produce the same failures in programmes that add resource in response to slippage without understanding what the addition will cost them. + +--- + +## What corrupts estimation — the engineering side + +### The estimate covers what is known, not what exists + +Engineers estimate the work they can see — the tasks they understand, the components they know how to build, the integrations they have done before. They systematically underestimate — not through carelessness but because it is structurally invisible — the work that is not yet known. The integration effort across discipline boundaries. The rework that ambiguous requirements will produce. The time required to validate assumptions about third-party components. The coordination overhead of working across teams that have different cadences and different assumptions about shared interfaces. + +The known work is estimable. The unknown work is where most overruns live. An estimate that does not explicitly account for what is not yet known — that does not build in investigation time for open questions and contingency for the risks that will materialise — is an estimate of the programme the team currently imagines, not the programme that exists. The gap between the two is the overrun. + +### The happy path is not the most likely path + +Estimates produced by imagining how the work will proceed when everything goes well describe a programme that does not exist. Complex engineering programmes do not follow the happy path. Integration failures. Specification ambiguities requiring rework. Architectural assumptions that turn out to be wrong. These are not edge cases. They are the predictable reality of building complex systems under uncertainty. + +An estimate based on the happy path is an estimate of best case, not most likely. The most likely case accounts for the variation that programmes of this type reliably experience. In the three-point model described in the next section, best case and most likely are different numbers — because best case requires specific conditions to hold that do not always hold. + +### Non-functional requirements are not secondary + +Estimates in software and firmware development frequently focus on functional delivery — the capabilities and behaviours the system must provide. Performance, security, reliability, maintainability, testability, and regulatory compliance are treated as properties that will be addressed once the functionality is in place. + +This is not a planning approach. It is a deferred cost accumulation strategy. Non-functional requirements are not properties that can be added to a completed system without significant rework. They are properties of the architecture and the implementation choices made throughout development. A system built without explicit consideration of its performance requirements will not meet those requirements without redesign. A system built without explicit consideration of testability will be expensive and slow to verify. + +An estimate that omits non-functional requirements is an estimate of a prototype. The cost of the gap between the prototype and the product is paid during validation and in operation — substantially later and substantially more expensively than it would have been paid during development. + +### The estimate does not state its conditions + +This is the most consequential engineering-side estimation failure — and the most common. Engineers produce estimates that are honest given their current understanding but do not make explicit what that understanding depends on. + +The estimate assumes the interface specification is as currently documented. It assumes the third-party component behaves as the supplier claims. It assumes the test environment will be available when the integration phase begins. These are conditions — things that must be true for the estimate to hold. When they are not stated, the estimate appears unconditional. The governance layer accepts it as unconditional. When the conditions are not met and the estimate proves wrong, the engineering team appears to have failed to deliver what they committed to. + +Stating conditions explicitly is precisely what the three-point estimate with attached risks achieves. The estimate is honest given current understanding. The conditions that understanding depends on are stated. The risks that would invalidate the conditions are named and priced. The governance layer knows what it is accepting — not just a number but the conditions and risks that give the number its meaning. When the governance layer accepts the number and not the conditions, as happened in Chapter 1, the commitment it has made is not the commitment the estimate described. + +--- + +## The estimation environment + +All of the corruptions described above are influenced by the environment in which estimation occurs. The most important environmental variable is not the capability of the estimators. It is whether the environment makes honest estimation rational. + +In an environment where honest estimates are accepted, used as the basis for planning, and supported with the conditions they require, engineers produce honest estimates. The conditions that make the estimate achievable are addressed. The risks are managed. The estimate proves accurate not because the estimators were prescient but because the system was designed to make it accurate. + +In an environment where honest estimates are challenged, compressed, and overridden, engineers learn that honesty is not what the environment rewards. They calibrate to what the environment will accept. The calibration may be toward inflation — produce a higher number that will survive the inevitable compression and still be achievable. Or it may be toward the committed date — produce whatever number gets the programme board to proceed. + +Neither calibration produces a useful estimate. Both produce numbers that look like estimates but are responses to governance pressure. And the programme is managed against those numbers — which is why it fails in the way Chapter 1 describes. + +--- + +## What good estimation looks like + +Good estimation is a continuous process of narrowing the cone — of replacing uncertainty with understanding through investigation, validation, and the progressive resolution of open questions. + +At the early stages, an honest estimate is a range. It acknowledges the width of the cone at that point. It identifies the specific investigations that will narrow it — the requirements work that will resolve scope ambiguity, the architectural spikes that will validate technical assumptions, the supplier evaluations that will confirm or refute dependency claims. It commits to a review point at which, once those investigations are complete, the estimate will be updated. + +As the programme progresses and the cone narrows, the estimate becomes more precise. At the point where the programme is well enough understood to support a three-point estimate, that estimate is produced. Best case is what happens if the specific conditions that enable it are all met — and those conditions must be stated explicitly, not assumed. Most likely is the realistic professional judgement of what the programme will take given the variation that programmes of this type experience. Worst case is what happens if the named risks materialise — and those risks must be named specifically. + +The governance layer's response to estimation at each stage should match the stage. At the wide end of the cone, accept the range, fund the investigations that will narrow it, and treat the range as honest current information. At a later stage, accept the three-point estimate with its conditions and risks, make the decisions the conditions require, and govern the programme against the most likely case — not the best case presented as the plan. + +--- + +## When the estimation environment breaks down + +When an engineering team operates in a governance environment that compresses estimates, treats them as commitments, and applies pressure to reduce numbers that were produced honestly, a specific and predictable consequence follows. + +In some teams, the response is to produce numbers that are higher than the honest estimate — because experience has taught the team that honest numbers will be reduced, and the reduced number needs to be achievable. Management commonly refers to this as sandbagging. + +It is worth naming this directly. In low-trust environments where estimates are routinely overridden, some teams do build protection into their numbers as rational self-protection. This is not dishonesty. It is adaptation to an environment that has made honest estimation systematically unrewarding. The team is behaving rationally in the system they are in. + +What matters is what that adaptation costs. Each inflated estimate reinforces the governance layer's belief that engineering estimates cannot be trusted. The governance layer applies more pressure. The next estimate is more inflated. The cycle compounds. Over time, the governance layer and the engineering team lose the ability to have an honest conversation about what a programme requires, because neither side trusts the numbers the other produces. The programme is managed on fictions. The outcome is determined by that, not by the capability of the people involved. + +The correct response is not more pressure. It is a different environment — one in which honest estimates are accepted, the conditions they depend on are managed, and the gap between a commercial commitment and an honest estimate is treated as a risk to be managed rather than a number to be suppressed. + +When that environment exists, the inflated numbers stop. Not because engineers have been challenged into honesty but because honesty has become the rational response. And the programme can be governed against reality rather than against a plan that was manufactured to avoid it. diff --git a/content/firmitas/part-1-why-programmes-fail/03-the-commitment-that-wasnt-theirs.md b/content/firmitas/part-1-why-programmes-fail/03-the-commitment-that-wasnt-theirs.md new file mode 100644 index 0000000..e884980 --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/03-the-commitment-that-wasnt-theirs.md @@ -0,0 +1,201 @@ +--- +title: "Chapter 3: The Commitment That Wasn't Theirs" +--- + +# Chapter 3: The Commitment That Wasn't Theirs + +At some point in the life of almost every engineering programme, a date is set that the engineering team did not set. + +It may arrive as a contract commitment, agreed by the commercial team before engineering was consulted. It may arrive as a board-level announcement, made for strategic or political reasons, that the product will be available by a specific quarter. It may arrive as a customer expectation, established in pre-sales conversations that engineering was not part of, which hardened into a contractual obligation before anyone thought to check whether it was achievable. + +However it arrives, the effect is the same. The programme begins with a commitment that was made by people who will not have to keep it, handed to people who had no part in making it, presented as a fixed constraint within which delivery must occur. + +This is the single most common structural cause of engineering programme failure. Not incompetence. Not insufficient process. Not inadequate tools or methodologies. The commitment that arrived already made, from above, before the work was understood. + +--- + +## The reverse-engineered plan + +When a date is set before the scope is understood, planning does not produce a plan. It produces a rationalisation. + +The planning process runs backwards. The date is fixed. The scope is known, approximately. The task is to construct a sequence of activities that will connect the current state to the required outcome by the required date. This is not planning. It is the construction of a narrative that makes the date look achievable. And in most organisations, the people doing the constructing know it. + +The reverse-engineered plan has several distinguishing characteristics that experienced engineers recognise immediately. + +Contingency is absent or minimal. Every task is estimated at best case. Dependencies are shown as sequential when they are actually parallel, or as parallel when they are actually sequential, depending on which arrangement produces the right end date. Risks appear on the risk register but are not reflected in the plan - the plan assumes they will be managed away, not that they will materialise and require time to address. Integration phases are compressed to whatever space remains after the development tasks have been allocated. Testing is scheduled for the end, with whatever time is left. + +The plan, when complete, shows delivery on the required date. Everyone in the planning exercise knows the plan is not achievable as written. Nobody says so clearly, because saying so would require challenging the date, and challenging the date would require either a difficult internal conversation or a difficult client conversation, neither of which anyone is incentivised to initiate. + +The plan is presented to programme governance. It is approved. The programme begins. + +At some point - usually between months three and six, depending on the complexity of the programme and the severity of the compression - the plan encounters the work. The work does not conform to the plan's assumptions. The integration phase that was allocated three weeks requires eight. The third-party dependency that was assumed to be available turns out to need qualification. The test environment that was scheduled to be ready at month four is ready at month six. + +Each individual divergence from the plan is reported as a delay. Each delay is treated as an execution failure. The plan was achievable. The team has not achieved it. The corrective response is pressure, increased reporting frequency, and questions about whether the team is the right team. + +None of this addresses the cause. The plan was not achievable. It was never achievable. It was constructed to show a date, not to reflect a reality. + +--- + +## The iron triangle ignored + +Every engineering programme operates within three constraints: time, cost, and quality. These three constraints are related. Changing one changes the others. This relationship is not controversial - it is one of the oldest and most consistently validated observations in engineering programme management. + +Delivering faster costs more, because you need more resource in parallel, or you need to buy time through shortcuts that will cost you later. Delivering cheaper takes longer, because you have less resource. Delivering faster and cheaper requires accepting lower quality - fewer features, more defects, more technical debt, a system that is less maintainable and less resilient than one built with adequate time and resource. + +These trade-offs are real. They are not negotiable. The physics of complex systems does not make exceptions for commercial commitments. + +Most programme governance structures behave as though the triangle does not apply to them. The client wants it delivered on the original date, within the original budget, to the original scope and quality. The programme board does not want to have the conversation about trade-offs. The commercial team does not want to tell the client that the scope is not achievable in the time for the money. Nobody wants to be the person who says that something has to give. + +The result is that all three constraints are nominally maintained while the programme quietly compromises on quality in ways that are never formally acknowledged. Features are descoped without being formally removed from the requirements. Testing is compressed without the acceptance criteria being formally revised. Technical debt accumulates without being formally recognised as a risk. The programme delivers something that looks, on paper, like what was contracted, while actually delivering something materially different - less complete, less tested, less maintainable than the original specification required. + +This is not dishonesty in the individual sense. It is the rational collective response to a governance structure that insists all three constraints are fixed while providing no mechanism for managing the tension between them. The programme team does what it must to show progress against a plan that does not reflect reality. The quality that is quietly compromised is the only constraint that can be degraded without immediately showing on a dashboard. + +The cost appears later. In production defects. In integration failures. In the support burden of a system that was not built to be maintained. In the next programme that inherits the technical debt of this one. By that point, the people who made the governance decisions that created the pressure are rarely in the room where the cost is being counted. + +--- + +## The commitment made on someone else's behalf + +The most damaging variant of the commitment problem is the one that appears most normal. + +An executive signs a contract. A sales director confirms a delivery date to a client. A product manager commits to a feature release in a roadmap presentation. A programme director announces a go-live date to the business. In each case, a commitment has been made. In each case, the commitment was made by someone who will not personally deliver against it. + +The commitment then flows down through the organisation. By the time it reaches the people who will actually do the work, it has the weight of contractual obligation, management expectation, and commercial necessity behind it. Challenging it is not simply a matter of offering a different estimate. It is a matter of challenging something that is already agreed, already communicated, already built into someone else's plans. + +This is the structural mechanism behind the failure in Chapter 1. The commercial director did not intend to set an unachievable date. They were doing their job - winning business, making commercial commitments, managing client relationships. The engineering team did not intend to fail to deliver. They were doing their job - assessing the work honestly, flagging the risks, producing the best estimate they could with the information available. + +The failure was not in either of those individual actions. It was in the gap between them. A commitment was made before the assessment was possible. The assessment, when it came, showed that the commitment was at risk. The governance structure had no mechanism for connecting the two - for treating the gap between the commitment and the honest assessment as a risk to be managed rather than an awkward fact to be noted and moved past. + +The commitment was not the engineering team's. It became their problem the moment it was handed to them. + +--- + +## The cycle + +The diagram below shows how this plays out across a programme - and across the programmes that follow it. + +```mermaid +flowchart TD + A["Commitment made<br>without engineering input"] --> B["Engineering produces<br>honest three-point estimate"] + B --> C["Gap identified<br>Commitment vs estimate"] + + C --> D{"Governance response"} + + %% Negative loop (common) + D -->|Common| E["Compress estimate<br>towards commitment<br>Risk noted but accepted"] + E --> F["Programme proceeds<br>on unachievable plan"] + F --> G["Risks materialise<br>predictably"] + G --> H["Programme fails<br>or overruns"] + H --> I["Accountability pushed<br>to delivery teams"] + I --> J["Conclusion:<br>Engineers cannot deliver"] + J --> K["Next programme:<br>More control<br>Less trust<br>More pressure"] + K --> A + + %% Positive loop (rare) + D -->|Rare| L["Acknowledge gap<br>Make trade-offs visible"] + L --> M["Rebaseline or renegotiate<br>scope / time / cost"] + M --> N["Programme proceeds<br>on honest plan"] + N --> O["Risks actively managed"] + O --> P["Programme delivers<br>to realistic expectations"] + P --> Q["Conclusion:<br>Estimates are reliable"] + Q --> R["Next programme:<br>More trust<br>Better decisions"] + R --> A + + %% Styling + style A fill:#F5F5F5,stroke:#666,color:#222 + style B fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style C fill:#FAEEDA,stroke:#BA7517,color:#412402 + style D fill:#FFF4CC,stroke:#B38F00,color:#4D3B00 + + %% Negative path + style E fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style F fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style G fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style H fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style I fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style J fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style K fill:#FCEBEB,stroke:#E24B4A,color:#501313 + + %% Positive path + style L fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style M fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style N fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style O fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style P fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style Q fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style R fill:#E1F5EE,stroke:#0F6E56,color:#04342C +``` + +The left-hand path - the common one - does not produce a single failure. It produces a cycle. Each iteration of the cycle reinforces the conditions that cause the next failure. Trust in engineering estimates decreases. Control increases. The gap between the people who make commitments and the people who keep them widens. The next programme is harder to run well than the one before it. + +The right-hand path exists. It requires a specific moment of governance courage - the willingness to treat the gap between the commitment and the honest estimate as a risk to be managed, not a problem to be suppressed. It requires someone with sufficient authority to say: the honest estimate does not match the commitment. That is information. What are we going to do about it? + +That moment almost never occurs in organisations where the left-hand path has become normal. Because the left-hand path has taught everyone in the governance structure that the gap is not a risk to be managed. It is an awkward fact to be noted, filed, and moved past on the way to task status. + +--- + +## The commercial and contractual reality + +It would be incomplete to discuss the committed date without acknowledging the context in which most committed dates are made. + +Fixed-price contracts are a commercial reality. Clients need to plan their own programmes and cannot do so against an open-ended delivery estimate. Competitive bids require price and schedule commitments. Boards and investors expect delivery dates. Programme roadmaps require milestones. These are not unreasonable requirements. They are the conditions under which engineering organisations operate. + +The problem is not the existence of commitments. The problem is the absence of honesty about what commitments are. + +A commitment made before scope is understood is not a delivery commitment. It is a commercial gesture - a statement of intent that creates expectations in the absence of evidence. Most fixed-price contract dates are commercial gestures. They represent the date the client needs, or the date the market opportunity requires, or the date the business case depends on. They do not represent the date that a genuine assessment of the work would produce. + +This distinction matters because it changes what the organisation should do when the honest estimate diverges from the committed date. + +When the honest estimate diverges from a genuine, evidence-based commitment - one produced by people who understood the work and made a specific promise about what they would deliver - the divergence is a problem with the estimation. The commitment was made in good faith based on understanding. The estimate has shown that the understanding was incomplete. That requires investigation. + +When the honest estimate diverges from a commercial gesture - a date set before the work was understood - the divergence is not a problem with the estimation. The estimation is doing its job: revealing the gap between aspiration and reality. The problem is the gap itself, and the question is what to do about it. + +What most organisations do is suppress the gap. The estimate is compressed toward the commitment. The risk register is noted. The programme proceeds on the commercial gesture, now dressed as a plan. + +What they should do is manage the gap. This requires a conversation - usually an uncomfortable one - about what is genuinely achievable and what trade-offs are available. It may require a conversation with the client about revised expectations. It may require an internal conversation about what can be descoped, what additional resource could be justified, or what the real cost of the overrun will be if the date is held and the quality compromised. + +None of those conversations are easy. All of them are cheaper than the conversation that happens at month six when the programme has failed in exactly the way the honest estimate predicted it would. + +The principle is simple even if the practice is difficult: a plan that requires honest risks to be suppressed is not a plan. It is a schedule waiting to become an incident report. + +--- + +## When the engineering team is handed the commitment + +The person in the most difficult position in this story is not the commercial director who made the commitment. They made a commercial decision, rationally, within their authority and their frame of reference. It is not the programme board member who noted the risk register and moved on. They responded to the information in the way their governance culture had taught them to respond to it. + +The most difficult position belongs to the senior engineer or programme manager who receives the commitment, understands immediately that it is not achievable on an honest plan, and must decide what to do about it. + +In most organisations, the options are narrower than they should be. + +Challenging the commitment directly requires political capital, a clear chain of evidence, and confidence that the challenge will be received as professional honesty rather than obstruction. In many organisations, that confidence is not warranted. The engineer who challenges the committed date is the engineer who is not a team player, who does not understand the commercial reality, who is being negative. + +Accepting the commitment and planning around it - the reverse-engineered plan - produces a programme that everyone knows is not achievable. The engineer who does this has bought themselves a quieter month and a much harder programme. + +The middle path - accepting the commitment as a working target while formally documenting the gap, attaching the risks explicitly, and establishing clear trigger points at which the gap will require escalation - is the most rational response and the one Firmitas is designed to support. It does not require the engineer to challenge the commitment directly. It requires them to make the conditions of the commitment visible and to maintain that visibility throughout the programme, so that when the risks materialise, the governance structure cannot claim surprise. + +That middle path is not risk-free. It requires a governance structure that treats risk registers as decision instruments rather than reporting artefacts. In the programme described in Chapter 1, that governance structure did not exist. The risks were visible. The decisions were not made. The outcome was predictable. + +Building a governance structure where that middle path is not just available but normal - where the gap between commitment and honest estimate is treated as a programme risk to be managed rather than a fact to be suppressed - is one of the primary structural objectives of this framework. + +--- + +## The double cost of getting it wrong + +One pattern worth naming specifically, because it is often invisible in programme retrospectives, is what happens when an organisation delivers a complex platform or product for its own operational use. + +When delivering for an external client under contract, the cost of an overrun is bounded. Penalty clauses, reputational damage, a difficult client relationship. Real costs, but defined ones. The delivering organisation absorbs them and moves on. + +When an organisation delivers for its own use - building the infrastructure it needs to operate - the overrun hits twice. The first hit is the delivery overrun itself. The second is the operational cost of the delay: every week the platform does not exist is a week the business operates on workarounds, legacy systems, or manual processes, incurring cost and creating risk that the project budget never captured. And when the platform eventually arrives - built under pressure, with risks deferred, with quality compressed to meet a date - the technical debt does not disappear. It becomes part of the operational infrastructure the business depends on. + +This second hit is almost always invisible in programme governance. The project cost is on one ledger. The operational cost of lateness is on another. The long-term cost of quality shortcuts is on a third. The decisions that produced all three costs were made in the first programme review, when the risk register was noted and the task status was reviewed. But by the time all three costs become visible, the connection to that governance decision has been lost. + +The principle is the same whether the delivery is internal or external: the cost of a commitment that was not made by the people who have to keep it is almost always higher than it appears at the point where it is made. The difference is that in internal delivery, the organisation has nowhere else to send the bill. + +--- + +## The next step + +The chapters so far have described a failure that originates in a commitment, persists through a governance structure that cannot process honest information, and compounds through a cycle that makes the next failure more likely than the last. + +Chapter 4 examines what that failure looks like from the inside - not the timeline of events, but the six interacting systems that produce it. Understanding those systems is the prerequisite for changing them. diff --git a/content/firmitas/part-1-why-programmes-fail/04-what-the-system-did-with-the-information.md b/content/firmitas/part-1-why-programmes-fail/04-what-the-system-did-with-the-information.md new file mode 100644 index 0000000..677d0e2 --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/04-what-the-system-did-with-the-information.md @@ -0,0 +1,163 @@ +--- +title: "Chapter 4: What the System Did With the Information" +--- + +# Chapter 4: What the System Did With the Information + +The programme in Chapter 1 did not fail because the information was absent. It failed because the system did not know what to do with the information it had. + +The lead systems engineer produced an honest three-point estimate. The risk register named three specific, actionable risks. The window for intervention was clearly visible at week three. All of this information existed. It was in the room. It was in the documents. It was in the heads of every engineer on the programme. + +And the programme failed anyway. + +This is the question that process improvements - better templates, revised escalation protocols, more frequent reviews - never adequately answer. If the information existed, why did the system not act on it? What was it about the way the programme was organised, governed, and structured that caused accurate, timely, actionable information to be received, noted, and not used? + +The answer is not a single cause. It is six interacting systems, each of which contributed something to the failure, and none of which can be fixed in isolation without the others reasserting the conditions that produced it. + +--- + +## Thinking in systems + +Before naming those six systems, it is worth being precise about what a system is and why the distinction matters. + +A system, in the sense used throughout this book, is not a process or a methodology or an organisational chart. It is a set of interacting elements - people, structures, incentives, tools, constraints - that together produce outcomes. The behaviour of a system is not determined by any single element. It emerges from the interactions between elements. Change one element without changing the interactions and the system reasserts itself. The new template gets used in the same way as the old template. The revised escalation protocol gets applied by the same governance culture that ignored the previous one. + +This is why the lessons-learned exercise in Chapter 1 produced no lasting change. It identified elements - the risk register format, the escalation protocol, the review frequency - and changed them. It did not change the interactions. The new elements were absorbed by the same system and produced the same outcomes. + +Engineering organisations are complex systems. Their behaviour is produced by the interaction of human systems, organisational systems, socio-technical systems, delivery systems, technical systems, and economic systems. None of these can be optimised in isolation without degrading the whole. Understanding how each contributed to the failure in Chapter 1 is the prerequisite for building a framework that addresses them as a coherent whole rather than as a series of independent problems. + +--- + +## The six systems + +### Human systems + +Human systems encompass the motivation, trust, incentives, psychological safety, communication patterns, and informal power structures that determine how people actually behave - as opposed to how the organisational chart says they should behave. + +In the programme in Chapter 1, the human system was operating under a specific and identifiable constraint: it was not safe to say certain things clearly. + +The lead systems engineer produced an honest three-point estimate and a risk register with three red items. That was an act of professional courage in an environment where the commercial commitment had already been made and the expectation of delivery was already established. Most engineers in that position would have felt the pressure to produce a more comfortable number. The fact that an honest assessment was produced at all is evidence that the human system had not entirely collapsed. + +But the human system had shaped the response to that honest assessment in a way that neutralised it. The programme board members who noted the risks and moved on were not behaving irrationally. They were behaving in the way that programme boards in their organisation behaved when faced with a risk register that contained uncomfortable information. The cultural norm - ask for mitigation plans, note the risks, move to task status - was the learned response of a human system that had not been designed to make hard decisions uncomfortable to avoid. + +The human system had also shaped the behaviour of everyone downstream of the programme board. The engineers who wrote firmware against an ambiguous specification, making a decision rather than escalating indefinitely, were behaving rationally in a system where escalation was slow, uncertain, and unlikely to produce a timely resolution. The programme manager who reported mostly-green task status was behaving rationally in a system that measured task completion and not programme health. + +People behave in ways the system makes rational. The human system in Chapter 1 made avoidance rational and honesty expensive. Until that changes, no amount of process improvement will change what the system produces. + +### Organisational systems + +Organisational systems include the operating model, governance structures, funding mechanisms, reporting lines, portfolio management, and performance evaluation frameworks that determine who is allowed to decide what, how priorities are set, and what success looks like institutionally. + +The organisational system in Chapter 1 had a specific structural flaw that is common across the industry: decision authority was concentrated at a level that lacked operational knowledge, while operational knowledge was concentrated at a level that lacked decision authority. + +The programme board had the authority to make the decisions the programme needed - to resolve the hardware-software interface ambiguity with the client, to validate the third-party component independently, to procure the test environment early enough to be useful. The programme board did not have the operational understanding to recognise that those decisions were urgent, that the window for making them was closing, and that not making them was not a neutral act but a choice with a specific and predictable consequence. + +The engineering team had that operational understanding. They had named the risks. They had attached timelines to them. They had presented the information in a form that should have been sufficient to generate decisions. The engineering team did not have the authority to make the decisions themselves - the client conversations, the procurement approvals, the architectural trade-offs that required budget and business sign-off were all above their authority level. + +This is the authority-experience gap operating at the level of organisational structure. It is not a matter of individual incompetence or bad faith. It is the predictable output of an organisational system that concentrated decision authority in a governance layer insulated from delivery reality. + +The performance evaluation framework reinforced this. Programme board members were evaluated on commercial outcomes - the contract was won, the relationship was maintained, the date was held as long as possible. They were not evaluated on whether the governance decisions they made in programme reviews were good decisions. The cost of those decisions - the six weeks of rework, the four-week architecture redesign, the two months of delayed integration testing - was attributed to execution rather than governance. The organisational system ensured that the people who made the decisions that produced the failure never bore the cost of it. + +## Socio-technical systems + +Socio-technical systems sit at the intersection of people and technology. They include the toolchains, documentation practices, reporting tools, communication platforms, and workflows through which work gets done and information flows. + +The risk register in Chapter 1 failed as a socio-technical system. Not because it contained wrong information - it contained correct information - but because it was designed as a reporting tool rather than a decision instrument. + +A risk register designed as a reporting tool produces reports. It captures risks, rates them, attaches mitigation plans, and generates a document that can be reviewed in programme governance. It confirms that risk management is happening. It does not generate decisions, because it is not designed to. It does not escalate automatically when a trigger date passes without action. It does not connect a risk item to the programme schedule in a way that makes the impact of non-action visible. It does not distinguish between a risk that has been mitigated and a risk that has been given a mitigation plan that requires a decision nobody has made. + +The three red items in Chapter 1 had been given mitigation plans. The mitigation plans required decisions - a client conversation, an independent component evaluation, a procurement approval - that were above the engineering team's authority. Those decisions were not made. The risks remained live. The register showed amber because mitigation plans existed. The system had been designed to produce a status, not to drive a decision. + +The task status reporting system had the same property. It was designed to show what had been completed, not to show whether the programme was healthy. Green task completion in a programme with three unresolved red risks and a plan built on best-case assumptions looks the same in a task status report as green task completion in a programme with no significant risks and a realistic plan. The socio-technical system was not designed to surface the difference. + +### Delivery systems + +Delivery systems define how work moves from intent to outcome. They include feedback loops, batch sizes, handoffs, validation mechanisms, risk management practices, and release strategies. + +The delivery system in Chapter 1 had a feedback loop that was too long and too slow to be useful. + +The honest estimate was produced at week three. The first material consequence of the unresolved risks - the six weeks of rework on the firmware written against an ambiguous specification - arrived at month four. The connection between the week-three risk and the month-four consequence was clear in retrospect. It was not visible in the delivery system's feedback loops because those feedback loops were designed around task completion rather than risk materialisation. + +The delivery system had also been structured in a way that deferred integration. In complex hardware-firmware-software programmes, integration is where the real risks surface. Hidden dependencies, incompatible assumptions, timing constraints that only become apparent when components operate together - these are not visible in unit development but become visible immediately in integration. Deferring integration to a late phase - as the reverse-engineered plan had done, allocating whatever time remained after the development tasks had been filled - is the delivery system equivalent of deciding not to look at the problem until it is too late to fix it cheaply. + +The batch size of the work was also too large. Firmware was being developed against a full, unvalidated specification rather than against a small, validated slice of it. The hardware-software interface ambiguities had not been resolved before development began in earnest. When the ambiguities turned out to matter, the rework was not limited to a small, recently-completed piece of work. It extended across everything that had been built on the ambiguous assumptions. + +A delivery system designed for learning - one with short feedback loops, small batch sizes, and early integration - would have surfaced the firmware specification problem in week four rather than month four. At week four, it is a two-week problem. At month four, it is a six-week problem with downstream consequences. + +### Technical and architectural systems + +Technical and architectural systems include the software, firmware, hardware, infrastructure, and integration architecture of what is being built. Architecture is not just a technical concern - it encodes assumptions about how change occurs, who owns what, and where failure is tolerated. + +The architectural dependency on the third-party software component in Chapter 1 was a technical system risk that had been identified but not managed. The component was a load-bearing element of the architecture. Its suitability had been assumed rather than validated. When the validation finally occurred, the assumption proved wrong and the architectural dependency had to be redesigned. + +This is a pattern that appears consistently in programmes that fail. An architectural assumption - that a component is suitable, that an interface is stable, that a dependency can be managed - is made early, before the evidence is available to validate it. The assumption is not flagged as an architectural risk. It is treated as a design decision. When it turns out to be wrong, the redesign cost is classified as a delivery problem rather than an architectural one. + +The technical system had also accumulated hidden coupling. The architecture redesign required changes to components that had already been completed and tested - because those components had been built in ways that assumed the validity of the architecture that was being redesigned. That coupling was not visible on the architecture diagram. It was embedded in the implementation. When it surfaced, it expanded the scope of the rework significantly beyond what a surface-level assessment of the architectural change would have suggested. + +### Economic systems + +Economic systems include the market and competitive forces, cost structures, regulatory constraints, pricing models, and commercial incentives within which the programme operated. + +The economic system had produced the committed date in the first place. The client needed the product by a specific date because their own programme depended on it. The delivering organisation had bid competitively to win the contract and had accepted the date as a commercial necessity. The penalty clauses created a financial incentive to maintain the date even when the honest estimate showed it was unachievable. The fixed-price structure meant that the cost of overruns would be borne by the delivering organisation, creating pressure to minimise reported risk rather than manage actual risk. + +These economic incentives were not incidental to the failure. They were structural inputs to it. The commercial director who committed to the date was responding rationally to the economic system. The programme board that held the date despite the risk register was responding rationally to the economic system. The engineering team that was asked to find a way to make the date work was being asked to solve an economic problem with engineering effort - which is not a solvable problem, but is a very common request. + +The economic system also shaped the lessons-learned exercise. The cost of the overrun was real and visible - delay penalties, additional resource, client relationship damage. The cost of the governance decisions that produced the overrun - the decision to note the risk register rather than act on it, the decision to confirm the delivery date rather than surface the gap - was invisible in the economic accounting. It had been distributed across the programme as execution costs rather than governance costs. The economic system had no mechanism for attributing the cost of a governance decision to the governance decision. + +--- + +## How the systems interacted + +The diagram below shows how the six systems interacted to produce the failure in Chapter 1. Each system contributed something. None of them was sufficient alone. The failure was produced by their interaction. + +```mermaid +flowchart TB + HS["Human system<br>Fear makes honesty expensive"] + OS["Organisational system<br>Authority separated from knowledge"] + ST["Socio-technical system<br>Risk register used for reporting<br>not decision-making"] + DS["Delivery system<br>Long feedback loops<br>Late integration<br>Large batches"] + TA["Technical system<br>Unvalidated architectural assumptions"] + ES["Economic system<br>Fixed price<br>Penalty clauses<br>Date pressure"] + + F["Programme failure"] + + %% Interactions (system behaviour) + HS -->|Suppresses transparency| ST + ES -->|Incentivises concealment| HS + OS -->|Disconnects decision from reality| ST + ST -->|Masks true status| OS + DS -->|Delays feedback| ST + TA -->|Creates hidden risk| DS + DS -->|Amplifies rework| ES + + %% Convergence + HS --> F + OS --> F + ST --> F + DS --> F + TA --> F + ES --> F + + %% Styling + style HS fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style OS fill:#FAEEDA,stroke:#BA7517,color:#412402 + style ST fill:#E6F1FB,stroke:#185FA5,color:#042C53 + style DS fill:#EEEDFE,stroke:#534AB7,color:#26215C + style TA fill:#E1F5EE,stroke:#0F6E56,color:#04342C + style ES fill:#FFF4CC,stroke:#B38F00,color:#4D3B00 + style F fill:#2B2B2B,stroke:#000,color:#FFFFFF +``` + +The important observation is not that any one of these systems caused the failure. It is that each one was necessary. If the human system had been designed to make honesty safe rather than expensive, the risk information would have been acted on rather than noted. If the organisational system had aligned decision authority with delivery knowledge, the programme board would have had the operational understanding to treat the risk register as urgent rather than routine. If the socio-technical system had been designed as a decision instrument rather than a reporting tool, the three red items would have generated escalation rather than mitigation plans. If the delivery system had used short feedback loops and early integration, the firmware specification problem would have surfaced in week four rather than month four. If the technical architecture had treated the third-party dependency as an architectural risk, it would have been validated before the architecture was built around it. If the economic system had attributed the cost of governance decisions to governance rather than execution, the incentive to suppress risk would have been weaker. + +Change any one of these systems in isolation and the others reassert themselves. Improve the risk register template without changing the human system and the new template is filed with the same consequences as the old one. Align decision authority with delivery knowledge without changing the economic system and the financial incentives to suppress risk remain. This is why process improvements applied to individual elements consistently fail to change outcomes. The failure is systemic. The response must be too. + +--- + +## What this means for the framework + +Part One has described a failure - in Chapter 1, through a specific programme - and explained it - in Chapters 2, 3, and 4, through the belief system, the structural mechanism, and the interacting systems that produce it. + +The framework in Part Two is built as a direct response to what Part One has described. Every element of the framework addresses one or more of the six systems identified in this chapter. The principles that open Part Two are not abstract values. They are structural positions designed to address specific failure mechanisms. The governance model, the estimation approach, the risk register treatment, the delivery model, the team design principles - each is the structural response to a named problem. + +Chapter 5 completes Part One's diagnosis by describing what happens when the failure cycle runs without interruption - and specifically, what the organisation learns from it, and why what it learns makes the next failure more likely than the last. diff --git a/content/firmitas/part-1-why-programmes-fail/05-the-learning-that-doesnt-happen.md b/content/firmitas/part-1-why-programmes-fail/05-the-learning-that-doesnt-happen.md new file mode 100644 index 0000000..4628e6e --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/05-the-learning-that-doesnt-happen.md @@ -0,0 +1,135 @@ +--- +title: "Chapter 5: The Learning That Doesn't Happen" +--- + +# Chapter 5: The Learning That Doesn't Happen + +Every organisation that runs the failure cycle described in the previous chapters believes it learns from it. + +The lessons-learned exercise is commissioned. The post-mortem is conducted. The retrospective is held. The findings are documented. The recommendations are presented. The process improvements are agreed. The next programme begins with the improvements in place. + +And fails in the same way. + +This is not a coincidence. It is the predictable output of a learning system that is identifying the wrong cause and therefore producing the wrong lesson. The organisation is learning. It is just not learning the thing that would change the outcome. + +--- + +## What the organisation concludes + +When a programme fails in the way described in Chapter 1, the organisation's diagnostic process almost always arrives at one of a small set of conclusions. The team underestimated the complexity. The risks were not escalated in time. The requirements were not well enough defined. The programme needed better project management. The engineering team did not deliver what they committed to. + +Each of these conclusions has enough surface validity to be credible. The requirements were not well enough defined - true, in the sense that the hardware-software interface ambiguities were never resolved. The risks were not escalated in time - true, in the sense that the three red items never generated the decisions they required. The engineering team did not deliver what they committed to - true, in the sense that the programme missed the contracted date. + +But each conclusion locates the cause in the wrong place. The requirements were not well enough defined because a governance structure that had the authority and the client relationship to resolve the ambiguities did not do so. The risks were not escalated in time because the escalation mechanism - the programme board risk review - received the escalation and did not act on it. The engineering team did not deliver what they committed to because the commitment was not theirs - it was handed to them, already made, by people who were not in the room when the work was assessed. + +The misattribution is not dishonest. It is structural. The diagnostic process that produces these conclusions is the same governance structure that produced the failure. The people conducting the lessons-learned exercise are working within the same organisational system, with the same authority structures, the same incentives, and the same cultural norms that shaped the failure in the first place. They are looking at the programme from inside the system that produced it, and they are seeing what that system allows them to see. + +What the system does not allow them to see clearly is that the failure was governance failure rather than execution failure. Because seeing that would require the governance layer to conclude that the governance layer failed - and governance layers are not, in general, designed to reach that conclusion about themselves. + +--- + +## The wrong lesson and its consequences + +The wrong lesson produces a specific and predictable set of responses. + +More process is added. Risk register templates are revised. Escalation protocols are strengthened. Gate reviews are made more frequent. Status reporting is enhanced. Programme management capability is reviewed. A lessons-learned library is created so that future programmes can benefit from the experience. + +None of these responses address the cause. The cause was that the governance structure had the authority to act on accurate, timely information and did not. Adding more process to a governance structure that does not act on the information it receives produces more information that is not acted on. It does not change the governance behaviour. It changes the volume and format of the information while leaving the response to it unchanged. + +The consequences of the wrong lesson extend beyond the next programme. + +The first consequence is that the next programme begins under tighter control. The conclusion that the team underestimated the complexity, or did not escalate the risks, or did not deliver what they committed to, produces a response of increased oversight, more frequent reporting, and less latitude for engineering judgement. The team that failed - in the organisation's narrative - requires closer management. + +The second consequence is that trust in engineering estimates decreases. The programme failed. The engineering team produced the estimates. The estimates, in the organisation's narrative, were wrong - because the programme did not deliver to them. The fact that the original honest estimate of seventeen months was closer to the actual outcome than the committed date of fourteen months is not visible in this narrative, because the honest estimate was replaced by the compressed plan before the programme formally began. + +The third consequence is that the authority-experience gap widens. The governance layer that did not act on the risk register concludes that the risk register needs to be better, not that the governance layer's response to it needs to change. The engineering layer that produced the honest estimate and was overridden concludes - rationally - that honest estimates are not valued, and adjusts its behaviour accordingly. + +The fourth and most damaging consequence is that the engineers who held the accurate information and were overridden begin, gradually, to stop volunteering it. Not through obstruction or disengagement, but through the accumulated experience of having been right, and having been neither acknowledged nor heard. The engineer who flags a risk, watches it get noted and filed, watches the programme fail in exactly the way the risk predicted, and then watches the failure attributed to execution rather than governance, has learned something important about the value of flagging risks in this organisation. They will still do their jobs. They will still produce estimates. But the estimates will be less honest, the risks will be less specific, and the information the organisation most needs will be less available than it was before the cycle ran. + +This is how capable engineering organisations progressively lose access to the capability that would allow them to improve. Not through dramatic failures or mass departures, but through the slow erosion of the conditions that make honest information safe to surface. + +--- + +## The compounding cycle + +The failure cycle described in Chapter 3 does not simply repeat. It compounds. Each iteration produces conditions that make the next iteration more likely to fail and harder to recover from. + +```mermaid +flowchart TD + A["Programme fails<br>predictably"] --> B["Failure misdiagnosed<br>as execution issue<br>not governance failure"] + + B --> C["Governance response<br>More process<br>Tighter control<br>Less trust"] + + %% Core reinforcing loop + C --> D["Psychological safety drops<br>Honest reporting declines"] + D --> E["Next programme starts<br>with distorted information"] + E --> F["Early commitment made<br>before understanding<br>Estimates compressed<br>Risks recorded not acted on"] + F --> A + + %% Secondary reinforcing effects + C --> G["Authority–experience gap widens"] + G --> E + + C --> H["Engineering capability erodes<br>over time"] + H --> E + + %% Styling + style A fill:#2B2B2B,stroke:#000,color:#FFFFFF + style B fill:#FAEEDA,stroke:#BA7517,color:#412402 + style C fill:#FCEBEB,stroke:#E24B4A,color:#501313 + + style D fill:#FCEBEB,stroke:#E24B4A,color:#501313 + style E fill:#E6F1FB,stroke:#185FA5,color:#042C53 + style F fill:#E6F1FB,stroke:#185FA5,color:#042C53 + + style G fill:#EEEDFE,stroke:#534AB7,color:#26215C + style H fill:#E1F5EE,stroke:#0F6E56,color:#04342C +``` + +The compounding effect has a specific character that is worth understanding. It is not rapid. An organisation does not lose its engineering capability in one programme or two. The erosion is slow enough that it is not visible as a trend until it has been running for some time. Individual programmes fail for specific, identifiable reasons. The pattern connecting those reasons - the same misattribution, the same wrong lesson, the same tightening of control, the same erosion of trust - becomes visible only when someone looks across multiple programmes rather than within each one. + +By the time the pattern is visible, the capability loss is significant. The engineers who held the institutional knowledge of why previous architectures were designed as they were have moved on. The culture of honest risk escalation that existed before the first compounding cycle has been replaced by a culture of managed optimism. The governance structure that was already insulated from delivery reality is now further insulated by a layer of reporting designed to show progress rather than to surface problems. + +The organisation is not failing dramatically. It is failing gradually, in ways that feel manageable at each stage, until the accumulated cost of many cycles of the wrong lesson becomes impossible to ignore. + +--- + +## What learning would actually look like + +The contrast with an organisation that learns correctly from programme failure is instructive, not because it is common - it is not - but because it illustrates what the diagnostic process would need to be capable of producing in order to generate the right lesson. + +A governance structure that learns correctly from the programme in Chapter 1 would ask different questions in the lessons-learned exercise. + +Not: why did the engineering team not escalate the risks in time? But: why did the escalation mechanism - the programme board risk review at week three - receive the escalation and not generate the decisions it required? + +Not: why were the requirements not well enough defined? But: who had the authority and the client relationship to resolve the hardware-software interface ambiguities, and why did that resolution not happen in the window where it could have changed the outcome? + +Not: why did the engineering team not deliver what they committed to? But: who made the commitment, on what information, and what was the relationship between that commitment and the honest engineering assessment that was produced three weeks later? + +These questions are harder to ask from inside the governance structure, because they locate the cause in the governance structure rather than in the execution layer. They require a willingness to examine the governance decisions - the decision to note the risk register rather than act on it, the decision to confirm the delivery date rather than surface the gap, the decision to move to task status rather than stay on the three red items until the decisions they required had been made - as the primary cause of the failure rather than as the context in which the execution failure occurred. + +That willingness is rare. But it is not impossible. And in the organisations where it exists, the lessons-learned exercise produces something different: not more process, but changed governance behaviour. Not revised templates, but different conversations in programme reviews. Not tighter control, but clearer decision authority. Not less trust in engineering estimates, but better conditions for honest estimation. + +The outcome, over subsequent programmes, is measurably different. Not perfect - complex programmes will always carry risk and uncertainty and will sometimes fail for reasons that could not have been predicted. But the specific, preventable failure described in Part One - the failure of accurate information to reach the people who could act on it, the commitment made before the work was understood, the risk register filed rather than acted on - becomes less common. And when it does occur, the organisation is more likely to identify the real cause and less likely to compound it with the wrong lesson. + +--- + +## The original claim + +Part One has now described the specific structural mechanism that Firmitas is built to address. + +Engineering programmes fail when the people who make delivery commitments are not the people who have to keep them. When the gap between honest engineering assessment and commercial commitment is treated as a fact to be suppressed rather than a risk to be managed. When the governance structure receives accurate, timely information about programme risk and does not generate the decisions that information requires. When the failure that results is attributed to execution rather than governance. And when the lesson drawn from that failure makes the next one more likely - by adding process without changing behaviour, by tightening control without aligning authority with knowledge, by reducing trust in engineering judgement without examining the conditions that produced the judgement it distrusts. + +This is not a description of incompetent organisations or bad people. It is a description of a structural problem that affects capable organisations with well-intentioned people across every sector, at every scale, producing outcomes that nobody wanted and that could have been avoided if the system had been designed differently. + +The people closest to the work almost always know what is wrong. The question is whether the system allows that knowledge to reach the people who can act on it - and whether those people, when they receive it, have the operational understanding and the governance discipline to treat it as the decision request it is rather than the reporting artefact it is so often mistaken for. + +Firmitas is the structural response to that question. Not a process to be installed. Not a methodology to be adopted. A framework built on an explicit philosophy about where knowledge lives, who should own commitments, how governance should treat risk information, and what leadership actually means in the context of systems that consistently produce outcomes their participants did not intend. + +Part Two describes that framework. Every element of it connects back to what Part One has described. The principles are not values. They are structural positions. The governance model is not a preference. It is the direct response to a specific governance failure. The estimation approach, the risk register treatment, the delivery model, the team design - each is built around the specific failure mechanism that the programme in Chapter 1 so faithfully illustrated. + +The framework cannot prevent all programme failures. Complex systems operating under genuine uncertainty will always produce some outcomes that could not have been predicted or prevented. What the framework is designed to prevent is the specific, common, and entirely avoidable failure of accurate information not reaching the people who need to act on it - and of the people who needed to act on it not doing so. + +That failure is not inevitable. It is structural. Structural problems have structural solutions. + +Part Two is the solution. diff --git a/content/firmitas/part-1-why-programmes-fail/06-engineering-is-a-systems-discipline.md b/content/firmitas/part-1-why-programmes-fail/06-engineering-is-a-systems-discipline.md new file mode 100644 index 0000000..19507fa --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/06-engineering-is-a-systems-discipline.md @@ -0,0 +1,197 @@ +--- +title: "Chapter 6: Engineering Is a Systems Discipline" +--- + +# Chapter 6: Engineering Is a Systems Discipline + +Part One has described a failure. Not a hypothetical one, not an extreme case, but the specific, common, and entirely preventable failure that plays out across engineering organisations in every sector with a consistency that can only be explained by a shared structural cause. + +The cause is not incompetence. It is not bad faith. It is not insufficient process or inadequate tooling or the wrong methodology. It is the systematic disconnection between the people who hold delivery knowledge and the people who hold delivery authority — and the predictable consequences of that disconnection when it is compounded by a belief system that mistakes the symptoms of the problem for its cause and a learning system that draws the wrong lesson from each failure and uses it to make the next one more likely. + +Before presenting the framework that addresses this, it is necessary to establish the lens through which the framework is designed and should be read. That lens is systems thinking — and specifically, the application of systems thinking to engineering organisations as the primary unit of analysis for understanding why they behave as they do. + +This is not an abstract philosophical choice. It is a practical one. The failure in Chapter 1 was not produced by a single cause. It was produced by six interacting systems, each of which contributed something, and none of which could be addressed in isolation without the others reasserting the conditions that produced the failure. Any framework that addresses individual elements without addressing the interactions between them will produce the same outcome as the lessons-learned exercise: changes that look meaningful on paper and leave the underlying system unchanged. + +--- + +## What a system is + +A system, in the sense used throughout this book, is a set of interacting elements that together produce outcomes. The outcomes are not determined by any single element. They emerge from the interactions — the incentives, constraints, feedback loops, and dependencies that connect the elements to each other. + +This definition has a specific and important implication: if you want to change the outcomes a system produces, you must change the interactions, not just the elements. Replacing a risk register template does not change the governance culture that treats risk registers as reporting artefacts. Revising an escalation protocol does not change the authority structure that receives escalations and does not act on them. Adding a new programme governance role does not change the incentive structure that rewards the appearance of control over the reality of it. + +The corollary is equally important: local optimisation — making any single element of the system perform better in isolation — reliably degrades overall system performance when the interactions are not addressed. The firmware team that optimises its development process without addressing the ambiguous hardware-software interface is optimising locally while the system accumulates risk globally. The programme board that improves its risk register template without changing its decision behaviour is optimising the input while leaving the output unchanged. + +Engineering organisations are complex systems. Their behaviour emerges from the interaction of multiple system types — human systems, organisational systems, socio-technical systems, delivery systems, technical systems, and economic systems. No single framework element addresses all of them. But a framework that does not keep all of them in view will consistently find that changes intended to address one system are neutralised by the others. + +This is the foundational reason why engineering leadership is a systems discipline. Not because systems thinking is intellectually fashionable, but because the alternative — treating the components of engineering organisations as independent problems to be solved separately — has a long and well-documented record of failure. + +--- + +## The philosophy that constrains the framework + +Firmitas is structured in three layers. Understanding the relationship between these layers is essential to using the framework correctly — and to recognising when it is being misused. + +The first layer is philosophy. Philosophy defines what matters and why. It establishes the non-negotiable positions — the things that are true about how engineering organisations work, about where knowledge lives, about what leadership means, about what commitments are — from which everything else follows. The philosophy layer does not change with context. It is the stable foundation on which the framework rests. + +The second layer is framework. The framework translates philosophy into repeatable structure. It defines the decision logic, the governance mechanisms, the lifecycle shape, the artefact types, and the role boundaries through which the philosophy is applied in practice. The framework is stable but not static — it can evolve as understanding improves, but it does not change casually or under pressure. + +The third layer is process. Process defines how work is actually executed in a specific context. It includes the methodologies, ceremonies, tooling choices, cadences, and documentation formats that teams use day to day. Process is intentionally variable — it can and should differ by product type, regulatory context, organisational maturity, and team composition. The only constraint on process is that it must respect the framework and honour the philosophy. + +```mermaid +flowchart TD + P["Philosophy<br>What matters and why<br>Non-negotiable<br>Stable"] + F["Framework<br>Decision logic and structure<br>Stable but evolvable<br>Context-aware"] + PR["Process<br>Contextual execution<br>Intentionally variable<br>Adapted to need"] + + P -->|Constrains| F + F -->|Constrains| PR + PR -->|Informs| F + F -->|Informs| P + + style P fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style F fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style PR fill:#EEEDFE,stroke:#534AB7,color:#3C3489 +``` + +The ordering is non-negotiable. Philosophy constrains framework. Framework constrains process. Process that violates the framework is not Firmitas. Framework elements that contradict the philosophy are not Firmitas. The most common failure mode in framework adoption — applying the process layer without engaging with the philosophy — produces compliance theatre rather than changed outcomes. Teams follow the artefact requirements and the ceremony schedules while the underlying system remains unchanged. The risk register exists but is still a reporting tool. The gate reviews happen but still do not generate decisions. The estimation process produces three-point estimates that are still compressed toward the committed date. + +This is Firmitas applied as process rather than understood as philosophy. It will not produce the outcomes the framework is designed for. The chapters that follow are written to prevent that misapplication — by making the philosophy explicit before the framework is presented, and by grounding every framework element in the specific failure mechanism it is designed to address. + +--- + +### The twelve principles + +The twelve principles below are the philosophy layer of Firmitas. They are not values or aspirations. They are structural positions — explicit statements about how engineering organisations work and what leadership must do in response. + +Each principle has been earned by what Part One described. Each connects directly to one or more of the failure mechanisms, systemic causes, or compounding consequences identified in the preceding five chapters. They are presented in the sequence in which they build on each other — from the foundational lens to the specific structural positions to the human reality that the framework ultimately exists to protect. + +--- + +#### Principle 1: Engineering is a systems discipline + +Engineering outcomes are produced by interacting systems, not by isolated individuals, teams, or tools. The behaviour of an engineering organisation emerges from the relationships between its human, organisational, socio-technical, delivery, technical, and economic systems. Optimising any one of these in isolation degrades the whole. + +This principle is not a statement about methodology. It is a statement about where to look when things go wrong — and where to intervene when you want them to go differently. Local efficiency that damages system health is failure, even if short-term output increases. + +--- + +#### Principle 2: Engineering exists to deliver outcomes, not outputs — and the difference is always a leadership decision + +The engineering function exists to deliver value to the customer and, in doing so, create value for the business — sustainably and repeatedly. Not activity. Not output. Not utilisation. Value is realised only when engineering work solves a real problem, is usable and reliable, and survives contact with the real world. + +The difference between outputs and outcomes is not a technical distinction. It is a leadership decision. An organisation that measures velocity, throughput, and milestone completion has chosen to measure outputs. That choice determines what teams optimise for — and what they sacrifice to hit the measures they are evaluated against. + +--- + +#### Principle 3: A commitment not made by those who must keep it is a target with someone else's name on it + +Presenting a plan to an engineering team and asking whether it is achievable is not making a commitment. Working backwards from a desired date to construct a plan is not making a commitment. A genuine commitment requires that the people responsible for delivery have set the date, had the risks accepted, had the implications acknowledged, and had the plan built with realistic foundations, honest estimates, and adequate slack. + +Everything else is a date with someone else's name on it. When that date is missed, the accountability for missing it belongs with the people who made it — not with the people who inherited it. + +--- + +#### Principle 4: Delivery knowledge lives closest to the work + +The people doing the delivery hold the most accurate information about risk, timescale, and feasibility. A system that does not route decisions through that knowledge will systematically make worse decisions than one that does. + +This principle has a direct governance implication. Teams closest to the work must own the day-to-day and week-to-week management of delivery without micromanagement, second-guessing, or gates approved by people without the operational knowledge to understand what they are approving. Decision authority must be aligned with delivery knowledge. Where it is not, the system is designed to fail. + +--- + +#### Principle 5: Leadership owns the system, not the intention + +Leadership responsibility cannot be delegated. Leaders are accountable for the design of the system, the incentives it creates, the behaviours it rewards, and the outcomes it produces. When teams struggle, the system is misdesigned, constraints are unclear or contradictory, or incentives are misaligned. Blaming individuals for the outputs of a system they did not design and cannot change is a failure of leadership, not diagnosis. + +Good intent does not compensate for poor outcomes. Effort does not excuse systemic failure. Activity does not equal progress. Leadership is accountable for what the system produces over time, not for what it intended to produce. + +--- + +#### Principle 6: Decisions are first-class engineering artefacts + +Decisions shape systems more than code, hardware, or processes. The decisions made in the first programme review of Chapter 1 — to note the risk register, confirm the delivery date, and move to task status — determined the outcome of the programme more completely than any subsequent technical work. + +Significant decisions must be made deliberately, with trade-offs explicit and rationale preserved. This is not documentation for its own sake. It is memory for the system. Undocumented decisions become invisible constraints that decay system coherence over time, that cannot be challenged because they cannot be found, and that must be rediscovered through failure rather than through record. + +--- + +#### Principle 7: Flow matters more than utilisation + +Busy systems fail. Full systems cannot adapt. The rate at which value progresses through the entire system — from intent to delivered outcome — is more important than the utilisation rate of any individual team or resource within it. + +Queues are risk. Waiting is cost. Work in progress is a liability. Optimising for utilisation at the expense of flow produces systems that appear productive until they are stressed, at which point they fail catastrophically because they have no capacity to absorb variation or respond to the unexpected. + +--- + +#### Principle 8: Slack is not waste — it is the capacity to improve + +Slack is not idleness. It is capacity reserved by design to absorb variation, respond to the unexpected, and improve the system without collapsing under the addition of unplanned work. A system with no slack is a system operating at the edge of failure. + +Organisations that eliminate slack in pursuit of short-term efficiency trade apparent productivity for long-term fragility. The cost of eliminating slack is not visible immediately. It is paid in the inability to respond to incidents, the deferral of learning, the accumulation of technical debt, and the burnout of the people who compensate with heroics for the capacity the system should have provided structurally. + +--- + +#### Principle 9: Quality is a system property, not an inspection activity + +Quality does not emerge from testing at the end of a delivery process. It emerges from the interaction of architecture, feedback loops, testing strategy, incentives, and decision timing throughout the process. Quality engineering is therefore a leadership concern — the conditions that determine whether quality is possible are set by how the system is designed, not by how thoroughly it is inspected afterward. + +Late discovery of defects is a signal of systemic failure, not testing weakness. The defect was introduced upstream. The test found it downstream. The relevant question is not whether the test found it, but why the system allowed it to travel so far before it was visible. + +--- + +#### Principle 10: Short-term optimisation is a leadership choice — the cost is real, delayed, and paid by people who had no say in the decision + +Every decision to compress scope, reduce resource, defer risk, or sacrifice quality has a cost. That cost does not disappear. It moves — to the people doing the work, to the quality of what is delivered, to the next programme that inherits the consequences, to the operational infrastructure that must be maintained and evolved by teams who were not there when the shortcuts were taken. + +Leadership owns that cost whether it acknowledges it or not. The decision to compress the timeline was made in a programme review. The cost was paid in six weeks of firmware rework, four weeks of architecture redesign, two months of delayed integration testing, and eight months of overrun. The people who made the decision and the people who paid the cost were not the same people. + +--- + +#### Principle 11: Failure is information — misattributed failure is poison + +When a programme fails and the correct cause is identified, the organisation becomes more capable of preventing the next failure. When a programme fails and the wrong cause is identified, the organisation becomes more likely to fail next time — because it has used the failure to reinforce the conditions that produced it. + +Blame is not a learning mechanism. It is the active destruction of the conditions that would have prevented the failure. Post-failure attribution that lands on individuals rather than systems, on execution rather than governance, on the people who held accurate information rather than the people who did not act on it, produces organisations that repeat their failures with increasing efficiency. + +--- + +#### Principle 12: Your people are your most significant investment — trust and respect their judgement or destroy what you spent years building + +Engineers are not resources to be allocated or headcount to be managed. They are the carriers of delivery knowledge, technical judgement, and institutional memory that no process, framework, or tool can replicate. The knowledge of why the architecture was designed as it was, what the previous programme taught about integration timing, which risks are real and which are theoretical — this knowledge lives in people. When those people are overridden, ignored, or treated as replaceable, they do not simply disengage. They take with them everything the organisation spent years building. + +An organisation that consistently fails to trust and act on the judgement of its engineers does not just waste that investment. It destroys it. The capability to deliver complex programmes well is not a property of the organisation's processes or tools. It is a property of the people who carry it. Treat them accordingly. + +--- + +### How the principles connect to Part One + +Each of the twelve principles is a direct structural response to something Part One described. + +Principles 1 and 2 establish the lens and the purpose — the systems view and the outcomes orientation — without which the failure in Chapter 1 is illegible. It looks like an execution failure rather than a systems failure. It looks like a delivery problem rather than a value problem. + +Principles 3 and 4 address the commitment mechanism and the knowledge gap — the two structural causes identified in Chapters 2 and 3. The commitment that was not theirs. The delivery knowledge that lived with the engineers and could not reach the governance layer. + +Principle 5 addresses the governance failure — the programme board that noted the risk register and moved on, that received accurate information and did not act on it, that owned the system that produced the failure without owning the failure. + +Principle 6 addresses the specific mechanism by which the failure was perpetuated — the decision to note the risk register rather than act on it was a decision, but it was not treated as one. It had no owner, no rationale, no recorded consequence. It was invisible as a decision and therefore invisible as a cause when the failure arrived. + +Principles 7 and 8 address the delivery system failures identified in Chapter 4 — the long feedback loops, the late integration, the full utilisation that left no capacity to absorb the consequences of the risks that materialised. + +Principle 9 addresses the quality consequence of the iron triangle being ignored — the silent compression of quality that occurs when time and cost are held fixed against a scope the system cannot deliver to the required standard within the required constraints. + +Principle 10 addresses the cost attribution failure identified in Chapter 5 — the displacement of costs onto people who had no say in the decisions that produced them. + +Principle 11 addresses the learning failure at the heart of Chapter 5 — the misattribution of failure that produces the wrong lesson and makes the next failure more likely. + +Principle 12 addresses the human cost that runs beneath all of the above — the progressive erosion of the conditions that make good engineering possible when the people who carry delivery knowledge are systematically overridden, blamed, and treated as interchangeable. + +--- + +## Part One is complete + +The diagnosis is made. The cause has been named. The structural conditions that produce the failure have been described in enough detail that the framework can be presented not as a set of best practices but as a set of structural responses — each connected to a specific failure mechanism, each designed to address not just the symptom but the interaction between systems that produced it. + +Part Two begins with the framework's structural logic — the layered model that connects philosophy to framework to process — and then works through each element of the framework in the sequence that corresponds to the failure mechanisms Part One described. + +The reader who has followed Part One should arrive at Part Two with a specific question in mind: what would this programme have needed in order to produce a different outcome at week three? Every chapter in Part Two is an answer to some part of that question. diff --git a/content/firmitas/part-1-why-programmes-fail/index.md b/content/firmitas/part-1-why-programmes-fail/index.md new file mode 100644 index 0000000..afbb981 --- /dev/null +++ b/content/firmitas/part-1-why-programmes-fail/index.md @@ -0,0 +1,15 @@ +--- +title: "Part One: Why Programmes Fail" +--- + +# Part One: Why Programmes Fail + +Most engineering programmes that fail do not fail suddenly. They fail slowly, visibly, and predictably — in ways that the people closest to the work could see coming and frequently said so. + +The chapters in this section describe how that happens. Not in the abstract, but through the specific structural mechanisms that convert accurate information into ignored information, honest estimates into compressed commitments, and avoidable failures into repeated ones. + +Part One does not offer solutions. That is Part Two's job. What Part One offers is an honest account of the cause — precise enough that the framework which follows can be understood as a structural response rather than a collection of best practices. + +Read Part One with your last difficult programme in mind. If the patterns described here feel familiar, they are. They are not the result of bad people or insufficient process. They are the result of systems designed — usually without anyone noticing — to produce exactly the outcomes they produce. + +That is the starting point. Systems that produce predictable failures can be redesigned. The redesign is what this book is for. diff --git a/content/firmitas/part-2-the-framework/07-leadership-owns-the-system.md b/content/firmitas/part-2-the-framework/07-leadership-owns-the-system.md new file mode 100644 index 0000000..f229857 --- /dev/null +++ b/content/firmitas/part-2-the-framework/07-leadership-owns-the-system.md @@ -0,0 +1,151 @@ +--- +title: "Chapter 7: Leadership Owns the System" +--- + +# Chapter 7: Leadership Owns the System + +The programme board in Chapter 1 did not design the system that produced the failure. It inherited it. The governance culture that treated risk registers as reporting artefacts rather than decision instruments had been established over years, across many programmes, by many decisions that individually seemed reasonable and collectively produced a structure incapable of acting on the information it received. + +But the programme board owned that system. Not in the sense of having created it. In the sense that they had the authority to change it and did not. That distinction — between creating a system and owning it — is the foundation of what leadership means in Firmitas. + +Leadership responsibility does not begin when a framework is installed or a plan is agreed. It does not end when a programme is handed to a delivery team. It persists for as long as the system operates and as long as the leader has the authority to shape the conditions within which the system functions. Frameworks, processes, and tools may assist. They do not replace leadership judgement. They cannot absorb leadership accountability. + +--- + +## What leadership actually means + +In most organisations, engineering leadership is described in terms of influence, vision, technical credibility, and stakeholder management. These attributes may be valuable. They are insufficient. + +Firmitas defines engineering leadership more narrowly and more rigorously: engineering leadership is the non-delegable responsibility for the health, alignment, and evolution of the systems that produce engineering outcomes. + +That definition has three components worth unpacking. + +**Non-delegable.** Leaders may delegate execution. They cannot delegate accountability for the systems within which execution occurs. The programme director who delegates programme management to a programme manager has not delegated responsibility for whether the programme management system is capable of surfacing risks and generating decisions. If the programme management system fails — if risk registers are filed rather than acted on, if escalations are received and not responded to, if commitments are made without honest assessment — that failure belongs to the leader who owns the system, not to the programme manager operating within it. + +**Health, alignment, and evolution.** System health means the system is capable of producing the outcomes it is designed to produce — that feedback loops are functioning, that information is reaching the people who need to act on it, that the conditions for good work exist. Alignment means the human, organisational, socio-technical, delivery, technical, and economic systems are reinforcing rather than undermining each other. Evolution means the system can adapt as conditions change without losing coherence — that it learns from what it produces and adjusts accordingly. + +**The systems that produce engineering outcomes.** Not the engineering outputs. The systems. A leader accountable for outputs can declare success when features are delivered on a milestone. A leader accountable for the systems that produce outcomes must ask whether the right things are being built, whether the conditions for building them well exist, and whether the organisation will be more capable of building the next thing than it was of building the last one. + +--- + +## Stewardship over control + +There is a model of leadership that feels like accountability but is not. It is control — the belief that the closer a leader monitors execution, the more they can ensure the right outcomes. It manifests as frequent status reviews, detailed progress tracking, direct involvement in operational decisions, and the expectation that leaders will have answers to questions about specific tasks at specific points in a programme. + +Control-based leadership assumes predictability. It relies on detailed plans, centralised decision-making, and compliance enforcement. In complex engineering environments, this approach creates the illusion of certainty while suppressing the learning and adaptation that complex environments require. + +Firmitas replaces control with stewardship. + +Stewardship means setting clear purpose and constraints, designing decision boundaries, enabling autonomy within those boundaries, monitoring system health, and intervening when systemic risk emerges. A steward does not dictate how every decision is made. A steward ensures that decisions can be made well — by the right people, at the right time, with the right information. + +The difference is not subtle. A control-oriented leader in the programme of Chapter 1 would have increased the frequency of status reviews after the programme board noted the three red risk items, would have asked the programme manager for daily updates on the firmware specification resolution, and would have added resource to the integration phase when it became clear that the timeline was slipping. None of these interventions would have addressed the cause. The cause was that the decisions required to resolve the three red items — a client conversation, an independent component evaluation, a procurement approval — had not been made and were not being made because no clear owner with authority and urgency had been assigned to them. + +A stewardship-oriented leader would have looked at the three red items and asked a different question. Not: is the team doing enough to manage these risks? But: what decisions are required to resolve these risks, who has the authority to make them, and what is preventing those decisions from being made this week rather than next month? + +That is the question the programme board in Chapter 1 never asked. It is the question that leadership stewardship requires. + +--- + +## The non-delegable responsibilities + +While execution can and should be decentralised, certain responsibilities cannot be delegated without undermining the integrity of the system. These are the responsibilities that define the leadership role in Firmitas — the things that frameworks and processes and programme managers cannot substitute for. + +**Purpose and direction.** Ensuring that the engineering function has a clear, stable understanding of why it exists and what outcomes matter. Not as a slogan. As a reference point that teams can use to make coherent decisions under uncertainty, to prioritise when priorities conflict, and to evaluate trade-offs without needing to escalate every decision upward. When purpose is implicit or inconsistent, teams optimise for local signals — closing tickets, hitting deadlines, avoiding scrutiny — rather than for the outcomes the engineering function exists to produce. + +**System design.** Shaping the organisational, delivery, socio-technical, and architectural systems so that desired behaviours emerge naturally rather than requiring heroic individual effort to sustain. The risk register that generates decisions rather than notes. The estimation process that produces honest three-point estimates rather than committed-date rationalisations. The governance culture that treats gap between commitment and honest assessment as a risk to be managed rather than a fact to be suppressed. These are not the product of good intentions. They are the product of deliberately designed systems — and system design is a leadership responsibility. + +**Trade-off resolution.** Making explicit decisions when goals conflict, rather than pushing unresolved tension downwards. The engineering team that is told to deliver to the committed date, to the contracted scope, at the contracted quality, with no acknowledgement that these three constraints cannot simultaneously be met given the current conditions, has had a leadership failure delegated to them. They will resolve the tension — they have no choice — but they will resolve it invisibly, in ways that compound later. Leadership stewardship means owning the trade-off, making it explicit, and bearing the consequence of the decision rather than distributing that consequence silently through the programme. + +**Risk ownership.** Ensuring that risk is surfaced, understood, and managed at the appropriate level rather than hidden or deferred. This is not the same as reviewing risk registers. It is ensuring that the governance structure treats risk information as a decision request rather than a reporting item — that when a risk is escalated, the response is a decision, not a note. It is also ensuring that the conditions for honest risk surfacing exist — that the human system makes honesty safe rather than expensive. + +**Sustainability.** Protecting long-term system health even when short-term pressure incentivises degradation. Defending slack when it is politically uncomfortable to do so. Resisting the compression of timelines that destroys the capacity to absorb variation and learn. Treating technical debt as a programme risk rather than a background condition. Refusing to accept that burnout is commitment and permanent urgency is effectiveness. These protections are not natural products of commercial pressure. They require active, ongoing leadership decisions. + +--- + +## The illusion of empowerment + +One of the most consistent failure patterns in engineering programme governance is the combination of empowerment language with centralised control behaviour. Teams are told they own the delivery. They are given autonomy over their ways of working. They are empowered to make decisions. + +And then the priorities are changed without consultation. The architecture decision they made is overridden in a review they were not invited to. The estimate they produced is replaced by the number that makes the business case work. The risk they escalated is noted and the conversation moves on. + +This pattern is not deliberate manipulation. It is the predictable product of a leadership layer that has genuinely adopted the language of empowerment without changing the governance behaviours that make empowerment real. The language says: you own this. The behaviour says: you own the execution of what we have decided. + +Firmitas is explicit about what genuine empowerment requires. Teams must understand what they can decide independently, what requires consultation, and what is explicitly reserved for a different level of authority. Those boundaries must be clear, coherent, and stable enough to trust. Autonomy without clarity is not empowerment. It is ambiguity — which teams respond to by becoming risk-averse, by seeking permission for decisions they should be making themselves, and by deferring upward until the governance layer is overwhelmed by decisions it should never have been asked to make. + +The leader who claims to empower teams while retaining centralised control over the decisions that matter has not empowered anyone. They have created the appearance of autonomy while preserving the substance of control — and in doing so have produced a team that manages upward rather than managing delivery. + +--- + +## Leadership as system integrator + +Engineering leaders operate at the intersection of multiple systems. The human system and the organisational system. The delivery system and the economic system. The technical architecture and the team structure. Their role is not to optimise each system independently — local optimisation consistently degrades whole-system performance, as Chapter 4 demonstrated. Their role is to align the systems so that they reinforce rather than undermine each other. + +Examples of misalignment that leadership stewardship must detect and correct: + +A delivery system optimised for speed — short iterations, continuous deployment, minimal process — operating inside an organisational system that requires multi-week approval cycles for any change to a production system. The delivery system is doing everything right. The organisational system is making those things impossible. The misalignment produces frustration, workarounds, and the eventual abandonment of the delivery system's practices because they cannot function within the organisational system's constraints. + +An architectural decision that assumes stable, long-lived teams in an organisation designed for high team turnover. The architecture distributes knowledge across team members in ways that make sense when those members remain together for years. When the organisation restructures annually, the architecture's assumptions are violated, and the cognitive load on each new team configuration is unsustainable. + +Performance incentives that reward individual output — features delivered, tasks completed, milestones hit — in an organisation that claims to value team outcomes, quality, and long-term system health. The incentive system is louder than the stated values. Teams optimise for what they are measured on. The stated values are aspirational. The incentive system is real. + +Each of these misalignments is a leadership problem. Not because the leader designed the dysfunction — they may have inherited every one of these conditions — but because the leader has the authority to identify the misalignment and change it, and that authority is the definition of ownership. + +--- + +## When leadership abdicates stewardship + +The cost of abdicating system stewardship is not immediate. It accumulates. + +The programme board that noted the risk register and moved to task status did not feel the cost of that decision at the programme board. It felt the cost at month six, when the programme missed its delivery date by eight months. By that point the connection between the week-three governance decision and the month-six outcome had been obscured by the intervening programme narrative — the firmware rework, the architecture redesign, the delayed test environment — and the accountability had been distributed across the execution layer rather than located in the governance decision where it belonged. + +This is the specific dynamic that makes leadership abdication so persistent. The cost is deferred. The cause is obscured. The people who pay the cost are not the people who made the decision. The people who made the decision draw the conclusion that the execution layer failed, not that the governance layer abdicated. + +When leadership abdicates system stewardship, organisations compensate through three mechanisms, none of which address the cause. + +Bureaucracy. More process, more governance layers, more reporting requirements — the organisational equivalent of adding more instruments to measure a problem rather than fixing it. The new process is operated by the same governance culture that produced the failure. It produces more information that is not acted on. + +Heroics. Reliance on individual strong performers who compensate, through exceptional effort and personal commitment, for the structural conditions that make normal performance insufficient. This works until the hero leaves, burns out, or is promoted away from the function that depended on them. It does not build organisational capability. It builds organisational dependency on individuals. + +Blame. Attribution of failure to specific people rather than to the system that shaped their behaviour. Blame provides emotional resolution without improving system behaviour. It discourages the reporting of future problems, because the lesson of blame is that surfaces problems receive consequences while hidden problems remain hidden until they become unavoidable. + +None of these are solutions. They are the symptoms of a system in which leadership accountability has been separated from system outcomes — in which the people with the authority to change the conditions that produce failure have convinced themselves, or been allowed to convince themselves, that the failure belongs to someone else. + +--- + +## Leadership beyond roles and titles + +Engineering leadership is not confined to formal management positions. Architects, technical leads, principal engineers, and senior practitioners all exert systemic influence. They shape architectural decisions that determine how easily the system can evolve. They model the estimation behaviour that others observe and calibrate against. They create or destroy the conditions for honest risk escalation through how they respond when problems are surfaced. + +Firmitas is explicit that formal authority and informal influence are both forms of leadership — and that both carry responsibility for the systems they shape. + +It is equally explicit that ultimate accountability sits with those who hold formal authority. Influence without authority cannot correct systemic misalignment. The principal engineer who has been saying for six months that the architecture will not support the required scale has done their job. The engineering director who has heard that and not acted has not done theirs. Leadership roles exist to absorb accountability, not to deflect it. + +--- + +## What this means for the programme in Chapter 1 + +The programme board in Chapter 1 did not fail because its members were incompetent. They failed because the governance system they operated within had not been designed to generate decisions from risk escalations — and they did not recognise that as their responsibility to change. + +A programme board operating on Firmitas principles would have looked at the three red items at week three differently. Not as risks to be noted and mitigated — as decision requests, each requiring a named owner with authority and a deadline by which the decision would be made. The hardware-software interface ambiguity: who will have a resolution conversation with the client, and by when? The third-party component validation: who will commission and fund the independent evaluation, and by when? The test environment procurement: who will approve the early procurement budget, and by when? + +Three questions. Three owners. Three deadlines. All of them answerable in the week-three programme board if the governance system had been designed to ask them. + +The outcome would not have been guaranteed. The client might not have resolved the interface ambiguity quickly. The third-party component might still have proved unsuitable. The test environment procurement might still have taken longer than hoped. But each of those outcomes would have been known earlier, when they were still cheap to respond to. And the programme would have been run against reality rather than against a plan that had already diverged from it. + +That is what leadership stewardship produces. Not certainty. Not the elimination of risk. The conditions under which accurate information reaches the people who can act on it, and those people act on it — while the cost of acting is still manageable. + +--- + +## The principle restated + +Leadership owns the system, not the intention. Good intent does not compensate for poor outcomes. Effort does not excuse systemic failure. Activity does not equal progress. + +The leader who says "we had good governance processes" while a programme fails in the way described in Chapter 1 has confused the existence of a process with the function of a system. Governance processes that do not generate decisions are not governance. They are the appearance of governance — and the appearance is more dangerous than the absence, because it prevents the acknowledgement that something needs to change. + +System stewardship is active, not passive. It requires leaders to observe how systems behave in practice — not how they are intended to behave on paper — and to intervene when the practice diverges from the intent in ways that matter. + +The programme board's risk register review was intended to surface programme risk and generate management response. In practice it surfaced programme risk and generated mitigation plan requests followed by task status reviews. That divergence between intent and practice was visible. It was correctable. It required someone with authority to notice it and change the governance behaviour. + +That someone was the leadership. That responsibility was theirs. + +It still is. diff --git a/content/firmitas/part-2-the-framework/08-delivery-knowledge-lives-closest-to-the-work.md b/content/firmitas/part-2-the-framework/08-delivery-knowledge-lives-closest-to-the-work.md new file mode 100644 index 0000000..bd2c995 --- /dev/null +++ b/content/firmitas/part-2-the-framework/08-delivery-knowledge-lives-closest-to-the-work.md @@ -0,0 +1,129 @@ +--- +title: "Chapter 8: Delivery Knowledge Lives Closest to the Work" +--- + +# Chapter 8: Delivery Knowledge Lives Closest to the Work + +The engineers on the programme in Chapter 1 knew what was wrong. They knew it at week three. They knew which risks were real, which assumptions were unsafe, and which decisions were required to change the outcome. They put that knowledge into a risk register and presented it to the people with the authority to act on it. + +The people with the authority did not have the knowledge. The people with the knowledge did not have the authority. The gap between them is where the programme failed. + +This is not an unusual configuration. It is the default configuration of most engineering programme governance. Authority flows upward through the organisational hierarchy. Knowledge stays down, with the people doing the work. The further up the hierarchy a decision travels, the further it moves from the operational reality that should inform it — and the closer it moves to the commercial, political, and organisational pressures that consistently distort it. + +The principle that delivery knowledge lives closest to the work is not a statement about who is more capable or more intelligent. It is a structural observation about where accurate information about risk, timescale, and technical feasibility actually resides — and what the consequences are when governance systems route decisions away from that information rather than through it. + +--- + +## Where delivery knowledge actually lives + +Delivery knowledge is the accumulated understanding of what a specific programme requires — the technical constraints, the integration dependencies, the risk landscape, the realistic timescales, the architectural assumptions that will hold and the ones that will not. It is not generic engineering knowledge, though that is part of it. It is contextual knowledge about this programme, this system, these components, and these conditions. + +This knowledge is held primarily by the engineers doing the work. Not because engineers are uniquely gifted, but because proximity to the work is what produces it. The firmware engineer who has been reading the hardware specification for three weeks knows where the ambiguities are. The systems integrator who has been evaluating the third-party component knows whether the supplier's claims are credible. The test engineer who has been designing the validation approach knows how long the test environment qualification will actually take. + +This knowledge is specific, current, and perishable. It is specific to the programme and cannot be reliably transferred by summary or abstraction — a risk register entry that says "hardware-software interface ambiguity" contains a fraction of the knowledge held by the engineer who identified it. It is current in the sense that it changes as the programme develops, as new information arrives, as assumptions are tested and found wanting. It is perishable in the sense that it degrades rapidly when the person who holds it leaves the programme, is moved to another team, or stops volunteering it because the system has taught them that volunteering it is not worth the effort. + +The governance layers above the engineering team hold different knowledge — commercial context, portfolio priorities, client relationship intelligence, regulatory and contractual obligations. This knowledge matters. It is not the same as delivery knowledge and it does not substitute for it. A governance layer that makes delivery decisions on the basis of commercial context and portfolio priorities without adequate delivery knowledge is not making informed decisions. It is making commercially-informed decisions in a delivery vacuum — which is precisely what produced the committed date that the engineering team was handed in Chapter 1. + +--- + +## The decision authority problem + +The gap between delivery knowledge and decision authority is structural. It is produced by the way engineering organisations are designed — with authority concentrated at the top of a hierarchy and knowledge concentrated at the bottom — and it is reinforced by a set of assumptions about what governance is for that make the gap feel normal rather than dysfunctional. + +The assumption that governance should reside at the top of the hierarchy rests on the idea that governance is primarily about commercial accountability, strategic alignment, and risk ownership at an organisational level. These are legitimate governance concerns. But they are not the only governance concerns on an engineering programme. Operational governance — the decisions about how delivery proceeds, which risks require urgent action, whether the plan reflects reality — requires operational knowledge that senior governance layers frequently do not have. + +When operational governance decisions are made by people without operational knowledge, the results are predictable. The risk register is reviewed without the ability to assess whether the mitigation plans are credible. The programme status is evaluated without the ability to assess whether the green task completion accurately reflects programme health. The delivery date is confirmed without the ability to assess whether the conditions required to achieve it are in place. + +This is not incompetence. It is a structural mismatch — the wrong type of knowledge being applied to a class of decision that requires a different type of knowledge. And the mismatch is invisible to the people making the decisions, because they do not know what they do not know. The programme board in Chapter 1 confirmed the delivery date because nothing in the information presented to them told them clearly that confirming it was the wrong decision. The information was present. The operational understanding to interpret it was not. + +--- + +## What genuine autonomy requires + +The response to the decision authority problem is not to move all decisions downward. Some decisions — commercial commitments, portfolio priorities, strategic trade-offs, regulatory obligations — genuinely belong at a governance level that has the commercial and organisational context to make them well. The goal is not to dismantle governance. It is to align decision authority with the knowledge required to exercise it well. + +This alignment looks different at different levels. + +At the team level, it means that engineers and architects own the day-to-day and week-to-week decisions about how delivery proceeds. How the work is sequenced. Which technical approaches are pursued. How risks within the team's scope are managed. These decisions require the contextual knowledge that the team holds. They do not require commercial or strategic input on a routine basis. Requiring teams to seek approval for decisions that are within their technical competence and their scope of ownership adds latency without improving decision quality. + +At the programme level, it means that programme governance focuses on the decisions that genuinely require programme-level authority — escalated risks that require resource or client engagement, trade-off decisions that span multiple teams or systems, gate decisions that represent genuine commitment points. It does not mean reviewing task completion status or second-guessing the technical decisions that the delivery team is better placed to make. + +At the senior governance level, it means that executive and portfolio governance focuses on the decisions that require strategic, commercial, and regulatory authority — portfolio prioritisation, contractual commitments, major investment decisions, regulatory strategy. It maintains visibility of programme health through the risk and issues register rather than through task status reporting. And it creates the conditions for honest information to flow upward by ensuring that the governance culture treats bad news as useful information rather than as a performance failure. + +The boundary between these levels is not rigid and will vary by programme type, scale, and regulatory context. What is rigid is the principle: decision authority should sit with the people who have the knowledge to exercise it well. Where that alignment does not exist, the governance system must be designed to create the information bridges that compensate — to ensure that the people making decisions have access to the knowledge they need, and that the people with the knowledge have access to the decision-making process. + +--- + +## No micromanagement — no second-guessing + +Genuine autonomy at the team level requires two specific commitments from the governance layers above it. + +The first is the absence of micromanagement. Micromanagement is the substitution of a governance layer's operational preferences for the team's operational judgement on matters within the team's authority. It takes many forms: requiring teams to seek approval for decisions they have the authority and knowledge to make independently, questioning specific task sequencing in programme reviews, requesting justification for technical approaches that have been made through legitimate engineering process, adding resource to a team without that team's input because the programme appears to be running late. + +Micromanagement is not always recognisable to the person doing it. It can feel like appropriate oversight, like quality control, like the exercise of legitimate programme authority. What distinguishes it from those things is that it substitutes the judgement of a person without operational knowledge for the judgement of a person with it — and in doing so, degrades rather than improves the quality of the decision while signalling to the team that their judgement is not trusted. + +The cost of that signal compounds over time. A team that has learned that its operational decisions will be second-guessed stops making them with confidence. It starts seeking approval rather than exercising judgement. It starts managing upward rather than managing delivery. The autonomy that was supposed to produce faster, better-informed decisions produces slower, more conservative ones — because the team has adapted to a governance system that does not actually trust them to decide. + +The second commitment is the absence of uninformed override. When a governance layer overrides a technical decision made by the delivery team, that override must be based on information or authority that the team did not have access to — a commercial constraint, a regulatory obligation, a portfolio-level trade-off. It must not be based on the preference of someone who has not read the technical assessment, does not have the contextual knowledge to evaluate it, and is substituting their seniority for the team's expertise. + +Uninformed override is one of the most damaging governance behaviours in engineering programmes. It is damaging not just because it produces worse technical decisions — though it often does — but because of what it teaches the people it is applied to. The engineer whose technically-grounded recommendation is overridden by a governance decision based on commercial preference, without engagement with the technical reasoning, has learned that technical reasoning is not what governs decisions in this organisation. They will still produce technical assessments. But the assessments will be calibrated to what the governance layer wants to hear, not to what the technical reality requires. + +--- + +## Gates approved without knowledge + +There is a specific governance failure that deserves direct treatment because it is so common and so consequential: the gate review approved by people who do not have the knowledge to evaluate what they are approving. + +Gate reviews are intended to be decision points — moments at which the programme's readiness to proceed to the next phase is assessed against defined criteria, and a genuine decision is made about whether to proceed, pause, or change direction. This is governance at its most valuable: a structured moment to verify that the programme is on a sound footing before making the commitments that the next phase requires. + +In practice, gate reviews in most organisations are approval ceremonies. The programme team prepares a gate pack. The gate pack is reviewed by a governance body. The governance body approves the gate. The programme proceeds. + +The approval is meaningful only if the governance body has the knowledge to evaluate whether the gate criteria have genuinely been met — whether the requirements are adequately defined, whether the architecture is sound, whether the risks are understood and managed, whether the estimates are honest and achievable. If the governance body lacks that knowledge — if it is evaluating the gate pack on the basis of whether it is complete and well-presented rather than on whether the programme is genuinely ready — the gate review is not a decision point. It is a formality that creates the appearance of governance assurance while providing none. + +The gate approval without knowledge transfers risk downward without transferring understanding. The programme proceeds into the next phase carrying the risks that the gate was supposed to surface. When those risks materialise, the gate review provides the governance layer with a defence — we approved this, the criteria were met — while the delivery team carries the consequences. + +Firmitas requires that gate reviews are genuine decision points — that the people approving gates have either the operational knowledge to evaluate readiness themselves or have been provided with the direct engineering input necessary to form an informed view. A gate approved on the basis of a well-prepared document pack by people who have not engaged with the engineering substance of what the gate is assessing is not governance. It is compliance theatre with consequences. + +--- + +## The authority-experience gap in this context + +Chapter 2 introduced the authority-experience gap — the structural condition in which the people with the most authority over delivery decisions have the least direct experience of what delivery requires. That gap is relevant here in a specific way. + +The alignment of decision authority with delivery knowledge is harder to achieve in organisations where the governance layer has not experienced the conditions under which delivery happens. Not because people without direct engineering experience cannot govern engineering programmes — they can, and many do so effectively. But because the ability to recognise when a decision requires operational knowledge that you do not have, and to create the mechanisms to access that knowledge before deciding, requires a degree of self-awareness that is only available to people who know what they do not know. + +The governance layer member who has personally written firmware against an ambiguous specification knows, viscerally, what the week-three risk register item means. They know how quickly ambiguity becomes rework, how the cost compounds when assumptions prove wrong at integration, how the window for a cheap fix closes faster than the programme schedule suggests. They do not need the risk register to tell them this is urgent. They know it is urgent. + +The governance layer member who has not had that experience may read the same risk register entry with appropriate concern but without the operational reference point that converts concern into urgency. The ambiguity is noted. It is on the register. There is a mitigation plan. The governance process has been followed. The programme proceeds. + +This is not a reason to exclude people without direct engineering experience from governance roles. It is a reason to design governance systems that do not depend on individual experiential knowledge to function correctly. The governance system should be designed so that the urgency of a risk is visible in the system — through the structure of the risk register, through the escalation mechanism, through the explicit connection between a risk and its impact on the programme plan — rather than dependent on the individual knowledge of the people reviewing it. + +That is a system design requirement. And system design, as Chapter 7 established, is a leadership responsibility. + +--- + +## The principle in practice + +The principle that delivery knowledge lives closest to the work has a set of practical implications for how programmes are governed. + +Teams own the day-to-day and week-to-week management of delivery. They sequence their own work, make their own technical decisions within defined architectural boundaries, manage their own risks within their authority level, and escalate only the items that require authority or resource they do not have. They are not required to justify routine operational decisions to governance layers that are not better placed to make them. + +Programme governance focuses on escalated risks and issues, not on task status. The primary governance instrument is the risk and issues register, reviewed with the intent of generating decisions on escalated items. Task completion is visible to the programme management layer and is used to identify issues that should be escalated — it is not the primary instrument of senior governance. + +Gate reviews are genuine decision points with defined criteria and genuine evaluation against those criteria. The governance body either has the operational knowledge to evaluate readiness or has direct engineering input that allows it to form an informed view. Gates are not approval ceremonies. They are decisions. + +Decision boundaries are explicit. Teams know what they can decide independently, what requires consultation, and what is explicitly reserved for a higher authority level. Those boundaries are stable enough to trust and clear enough to apply without seeking clarification on routine matters. Where a decision genuinely spans boundaries — where its implications are broader than a single team's scope — there is a defined mechanism for bringing the relevant parties together rather than escalating upward through a hierarchy that may lack the knowledge to resolve it well. + +--- + +## What changes when this is done well + +The change is not dramatic. It does not involve new ceremonies or complex process redesign. It involves a shift in where attention is focused and what governance systems are designed to do. + +Programme boards that focus on escalated risks and decisions rather than task status spend their time on the things that only they can resolve. The conversations are harder. They require decisions rather than acknowledgements. But the decisions are the right decisions — made by the right people, at the right time, with the right information — rather than the rubber-stamped approvals and noted risk items that the wrong governance model produces. + +Engineering teams that are genuinely trusted to manage their own delivery behave differently. Not because they have been motivated by empowerment rhetoric — but because the system now makes honest behaviour rational. The estimate that will be accepted rather than compressed can be honest. The risk that will generate a decision rather than a note is worth escalating. The architectural concern that will be heard rather than overridden is worth raising clearly. + +This is the connection between the governance principle in this chapter and the human system principle in Chapter 5's compounding cycle. Delivery knowledge reaches decisions when the governance system is designed to receive it. When it reaches decisions, the decisions improve. When the decisions improve, the programme outcomes improve. And when programme outcomes improve in ways that are attributable to the honest information the engineering team provided, the conditions for the next programme's honest information are better than they were for this one. + +That virtuous cycle is available. It requires a governance system designed to produce it. diff --git a/content/firmitas/part-2-the-framework/09-commitments-estimates-and-honest-plans.md b/content/firmitas/part-2-the-framework/09-commitments-estimates-and-honest-plans.md new file mode 100644 index 0000000..9b2d403 --- /dev/null +++ b/content/firmitas/part-2-the-framework/09-commitments-estimates-and-honest-plans.md @@ -0,0 +1,187 @@ +--- +title: "Chapter 9: Commitments, Estimates, and Honest Plans" +--- + +# Chapter 9: Commitments, Estimates, and Honest Plans + +The programme in Chapter 1 did not fail at month six when the delay was announced. It failed at the point when a commitment was made that was not based on an honest assessment of what the work required — and then compounded at the point when the honest assessment arrived and was not used to revise the commitment. + +Every element of this chapter is a structural response to that specific failure sequence. How commitments should be made. How estimates should be produced. What a genuine plan looks like. How the gap between a commercial commitment and an honest estimate should be managed. And what accepting a commitment actually means — not just accepting the number, but accepting the conditions and risks that the number depends on. + +These are not process preferences. They are the structural mechanisms by which the broken accountability loop described in Part One is closed. + +--- + +## What a commitment actually is + +A commitment is a specific promise, made by a specific person or group, to deliver a specific outcome by a specific point in time, based on a genuine assessment of what that outcome requires. + +Most things that are called commitments in engineering programme governance are not commitments by this definition. They are commercial gestures — dates and numbers agreed in service of a business need before the evidence required to support them exists. This is not inherently dishonest. Commercial contexts require decisions to be made under uncertainty. A client needs to know when a product will be available. A board needs a date for a strategic milestone. An investment case needs a cost and timeline to make sense. + +The problem is not the existence of commercial gestures. The problem is when they are treated as commitments — when the date set before the scope was understood is treated as a binding delivery commitment that the engineering team must honour, rather than as a commercial expectation that must be tested against an honest assessment of what is actually achievable. + +The distinction matters because it changes what the organisation should do when the honest assessment diverges from the commercial gesture. + +When the divergence is treated as a problem with the estimate — when the engineering team is asked to find a way to close the gap rather than the gap being treated as information about the realism of the commercial gesture — the programme proceeds on a plan that everyone involved knows is not based on reality. This is the sequence that Chapter 1 described. It produces failures that are predictable, preventable, and attributed to the wrong cause. + +When the divergence is treated as information — when the gap between the commercial gesture and the honest estimate is treated as a risk to be managed, a conversation to be had, a trade-off to be owned — the programme has the opportunity to proceed on a plan that reflects what is actually achievable. This is harder commercially. It is substantially cheaper operationally. + +A commitment, in the Firmitas sense, cannot be made until an honest assessment exists. Everything before that is a working hypothesis that should be clearly labelled as such — a target, an expectation, a commercial position — not a commitment. + +--- + +## Who makes the commitment + +Principle 3 states that a commitment not made by those who must keep it is a target with someone else's name on it. This principle requires a specific and practical answer to the question: who should make the commitment? + +The commitment should be made by the people who have both the delivery knowledge to assess what is achievable and the authority to be held accountable for the outcome. In most engineering programmes, this means the commitment requires genuine input from the engineering team and genuine ownership by the programme leadership. + +It does not mean that engineers set the date unilaterally. The commercial context matters. The client's constraints matter. The portfolio priorities matter. A date that is technically achievable but commercially unacceptable is not useful. The goal is a commitment that is both honest and commercially viable — which sometimes requires difficult conversations about what is achievable within commercial constraints, what trade-offs are available, and what the cost of maintaining a commercially convenient but technically unrealistic date will be. + +What it specifically excludes is the sequence that produced the failure in Chapter 1: commercial commitment made, engineering assessment produced, gap identified, gap suppressed in favour of the commercial commitment. That sequence transfers the commercial gesture to the engineering team as if it were a genuine commitment while retaining none of the conditions that would make it achievable. + +Presenting a reverse-engineered plan to an engineering team and asking if it is achievable is not making a commitment. It is asking the engineering team to validate a number that was not produced by honest assessment. The answer, in most cases, will be a qualified yes — achievable if these risks are mitigated, if these decisions are made, if these conditions are met — which will be recorded as a yes and used as the basis for holding the team to a number they never genuinely committed to. + +Making a commitment means the risks are accepted, not just noted. It means the implications are acknowledged, not just recorded. It means the plan has slack, realistic foundations, and honest assumptions — not the best-case scenario dressed as a plan. + +--- + +## How honest estimates are produced + +An honest estimate is produced by the people who will do the work, from a genuine understanding of the scope, the system, the context, and the risks. It is not produced by a project manager working from a high-level work breakdown structure. It is not produced by extrapolation from a previous programme without adequate consideration of what is different about this one. It is not produced under time pressure to hit a governance deadline before the scope has been adequately understood. + +Three conditions are necessary for honest estimation. + +**The right people.** Estimation by proxy is not estimation. The people estimating must be close enough to the work to understand what it actually requires — the specific technical challenges, the integration dependencies, the unknowns that will need to be resolved before certain elements can be estimated with confidence. Where that proximity does not yet exist — where the scope is not yet understood well enough to estimate reliably — the honest answer is a range estimate with explicit uncertainty bounds, not a false precision that obscures the uncertainty. + +**Full context.** Estimates produced in isolation — without understanding of the wider system, the client's real requirements, the regulatory obligations, the architectural constraints, the dependencies on other teams and external parties — are not estimates of what the programme requires. They are estimates of a simplified version of it. The divergence between the simplified version and the real programme is the estimation gap that reliably produces overruns. Estimates must be produced with full visibility of the scope, the constraints, and the risks that will shape the actual delivery. + +**A safe environment.** Estimates produced under pressure to reach a specific number are not honest estimates. They are numbers calibrated to what the estimating environment rewards. When the governance culture treats estimates as negotiating positions, estimates become negotiating positions. When the governance culture treats estimates as honest assessments that will be used to make real decisions, estimates become honest assessments. The environment is the variable. The people doing the estimating are responding to it. + +--- + +## The three-point estimate + +A single-point estimate is not an estimate. It is a commitment masquerading as an estimate — a specific number that contains no information about the conditions it depends on or the risks that could invalidate it. + +Firmitas requires three-point estimates. Not as a bureaucratic requirement, but because a three-point estimate contains the information that a single-point estimate withholds — information that the governance layer needs to make real decisions rather than to approve numbers. + +**Best case** is not the optimistic scenario. It is the scenario in which everything that can go well does go well — in which the specific conditions required for the fastest achievable outcome are all met. The best case estimate must state explicitly what those conditions are. Best case is twelve months if: the hardware-software interface specification is confirmed with the client by week four, the third-party component is validated against programme requirements by week six, and the test environment is procured and qualified by month three. Without those conditions, the best case is not achievable. The number without the conditions is not a best case. It is a wish. + +**Most likely** is the genuine professional judgement of what the programme will take under realistic conditions — not optimistic conditions, not worst-case conditions, but the conditions that actually tend to prevail on programmes of this type. It incorporates the most probable outcomes for the identified risks, the realistic rather than ideal availability of dependencies, and the friction that complex programmes reliably accumulate in ways that individual task estimates do not capture. Most likely is the number the team would bet on if they were betting with their own money. + +**Worst case** is what happens if the named risks materialise. Not all the risks — the specific ones that, if they occur, would have the most significant impact on the timeline. The worst case estimate must state explicitly which risks would produce it. Worst case is twenty-two months if: the hardware-software interface ambiguity requires a formal specification revision process with the client taking six to eight weeks, the third-party component proves unsuitable and requires replacement with a four-week architecture redesign, and the test environment qualification takes three months due to hardware availability constraints. The worst case is not a contingency buffer. It is the priced consequence of specific, named risks. + +The three numbers together tell the governance layer something a single number cannot: here is the range of realistic outcomes, here are the conditions that produce each outcome, and here are the decisions you need to make to keep the programme in the best-case range rather than the worst-case one. + +--- + +## The risk register as part of the estimate + +The risk register attached to a three-point estimate is not a separate document. It is the same document. The risks in the register are the conditions attached to the estimate — the specific items that connect the most likely case to the worst case and that define what actions are required to keep the programme at the better end of the range. + +This is the relationship that was broken in Chapter 1. The estimate and the risk register were presented together but treated separately. The governance layer accepted the committed date — a number below the most likely estimate — while noting the risk register without generating the decisions the risks required. The two documents were decoupled. The number was used. The conditions were filed. + +When the estimate and the risk register are the same document, decoupling them is not possible. Accepting the most likely estimate means accepting the conditions that produce it and the risks that could push it toward worst case. A governance layer that accepts a three-point estimate but does not act on the risks attached to it has not accepted the estimate. It has extracted the number while declining the information that gives the number its meaning. + +The governance principle follows directly: accepting an estimate requires accepting the risk register. The risk register is not reviewed and noted alongside the estimate. It is the basis on which the estimate is evaluated. The decisions that the risk register requires are the decisions that the governance layer must make in response to the estimate. If those decisions are not made, the estimate that has been accepted is not the estimate that was presented — it is the best case, stripped of its conditions, presented as the plan. + +--- + +## What a genuine plan looks like + +A genuine plan is built forward from an honest estimate, not backward from a desired date. + +This sounds obvious. It is almost universally violated. + +Building forward from an honest estimate means starting with the most likely case — the professional judgement of what the programme will take under realistic conditions — and constructing a plan that reflects that assessment. The plan identifies the specific decisions and actions that are required to achieve the best case, treats those as programme priorities, and explicitly shows the impact on the timeline if those actions are not taken by defined trigger dates. + +Building backward from a desired date means starting with the date and constructing a narrative that connects it to the current state. Every task is estimated at best case. Dependencies are shown in whatever arrangement minimises the apparent duration. Risks are listed on a register and assumed to be mitigated. The plan, when complete, shows the desired date achievable — because it was constructed to do so, not because it reflects reality. + +The distinction is visible in how the plan handles uncertainty. A plan built forward from an honest estimate contains explicit uncertainty — ranges rather than point estimates where confidence is low, identified decision points where the timeline will be revised based on new information, acknowledged contingency for the risks that have been priced into the worst-case estimate. A plan built backward from a desired date treats uncertainty as something to be removed from the presentation rather than managed in the execution. The uncertainty exists. The plan simply does not show it. + +A genuine plan in the Firmitas sense has four properties. + +It is built on honest most-likely estimates rather than best-case assumptions. Every task estimate reflects genuine professional judgement, not the number required to make the plan fit the date. + +It contains explicit slack. Slack in a plan is not waste or padding. It is the capacity to absorb the variation that complex programmes reliably produce — the integration issue that takes a day longer than expected, the client response that takes a week rather than two days, the test failure that requires a design investigation. A plan with no slack is a plan that assumes everything will go right. Complex programmes do not work that way. A plan that cannot absorb routine variation is not a plan. It is a schedule waiting to overrun. + +It is connected to the risk register. The plan shows what changes if each significant risk materialises — not as a separate scenario document, but as an integral part of how the plan represents the programme's future. When risk R-007 materialises, the timeline moves by this much, for these reasons, and this is the decision point at which the revised timeline should be formally acknowledged rather than absorbed silently into an extended delay. + +It is revisable without drama. A plan is a model of the future, not a contract with it. The future will diverge from the plan. When it does, the plan should be revised to reflect what is now known rather than maintained as a fiction to avoid the discomfort of acknowledging the divergence. A programme that cannot revise its plan without triggering a governance crisis has not built a plan. It has built a commitment structure that converts every unexpected event into a failure narrative. + +--- + +## The contract reality + +Fixed-price contracts are a commercial fact of life. They require commitment before understanding and create financial incentives to maintain dates even when honest assessment suggests they are not achievable. This is the commercial context within which most engineering programme commitments are made, and the framework must address it directly rather than pretending it does not exist. + +The honest position is this: a fixed-price contract date is a commercial commitment, not a delivery commitment. It represents the date the client needs, or the date the market requires, or the date the business case depends on. It does not represent the date that a genuine assessment of the work would produce — because the assessment does not exist at the point the contract is signed. + +When the honest estimate arrives — when the engineering team produces their three-point assessment — it will often diverge from the contract date. That divergence is information. It is not a performance failure by the engineering team and it is not a reason to compress the estimate. It is the first honest view of the gap between the commercial commitment and the delivery reality. + +What the organisation does with that information determines what happens next. + +Suppressing the divergence — compressing the estimate toward the contract date, treating the risk register as a compliance item rather than a set of decision requests — produces the sequence of Chapter 1. The programme proceeds on the contract date. The risks materialise on schedule. The delivery date is missed by a margin that reflects the original divergence between the contract date and the honest estimate. The organisation has paid the cost of the delay plus the cost of operating in denial about it. + +Managing the divergence — treating the gap as a programme risk, making the decisions required to close it where possible, having the commercial conversation with the client where necessary, and maintaining an honest plan that reflects what is achievable — is harder in the short term and substantially cheaper in the long term. + +The commercial conversation at month one, when the honest estimate first reveals the gap, is a fraction of the cost of the commercial conversation at month six, when the delay is announced. It also produces better outcomes — because at month one, there is time to make trade-off decisions, to restructure scope, to agree milestone revisions, to find the additional resource that could close the gap. At month six, the options have narrowed considerably. + +A plan that will breach the contract is not an engineering problem. It is commercial information that belongs at the level where the contract was made. The engineering team's responsibility is to produce the honest estimate. The programme leadership's responsibility is to treat that estimate as the decision input it is — and to have the commercial conversation it requires while there is still time to change the outcome. + +--- + +## Accepting a commitment + +Accepting a commitment in the Firmitas sense is a specific act with specific requirements. It is not the passive acknowledgement of a number presented in a programme governance meeting. It is the active acceptance of a set of conditions — the risks that have been identified, the decisions that are required to manage them, the implications for the programme if those decisions are not made. + +Accepting the most likely estimate without accepting the risk register is not commitment acceptance. It is number extraction — taking the number the programme will be governed against while declining the information that gives the number its meaning. + +Acceptance, in practice, means three things. + +The governance layer has read and understood the risk register — not as a list of items to be noted but as a set of conditions that the programme depends on and a set of decisions that the governance layer must make. Each red item is understood as a decision request. Each decision request has an owner with authority and a deadline by which the decision will be made. + +The governance layer has confirmed what it is providing — the client conversations it will have, the resource approvals it will make, the decisions it will take — and by when. Acceptance is not passive receipt of an estimate. It is active commitment to the governance actions required to make the estimate achievable. + +The governance layer has acknowledged the consequences of inaction — that if the decisions required to maintain the most likely case are not made by their trigger dates, the programme will trend toward the worst case, and that the worst case has been understood and accepted as the consequence of that inaction rather than as an unexpected failure. + +When acceptance looks like this, the programme board meeting in Chapter 1 ends differently. The three red items do not leave the table until the decisions they require have owners, authority, and deadlines. The programme does not proceed on the contract date as if the honest estimate had not been produced. The gap between the two is visible, owned, and actively managed rather than noted and filed. + +--- + +## A worked example + +The following shows the difference between a rubber-stamped commitment and a genuine one, using the programme from Chapter 1 as the basis. + +**The rubber-stamped commitment:** + +Programme board, week three. The lead systems engineer presents a three-point estimate: best case twelve months, most likely seventeen months, worst case twenty-two months. The risk register identifies fourteen items, three rated red. The programme board notes the estimate, confirms the contract delivery date of fourteen months, requests mitigation plans for the three red items, and moves to task status. + +The commitment that has been made: deliver in fourteen months. The conditions attached to that commitment: none that have been formally accepted. The decisions the risk register requires: unassigned. The programme proceeds. + +**The genuine commitment:** + +Programme board, week three. The lead systems engineer presents the same estimate and risk register. The programme board engages with the three red items as decision requests. + +Red item one — hardware-software interface ambiguity. Decision required: who will have the resolution conversation with the client, and by when? Owner assigned: the programme director. Deadline: end of week five. If resolution is not achieved by week five, the programme timeline moves to sixteen months minimum and a formal rebaseline is triggered. + +Red item two — third-party component validation. Decision required: who will commission and fund the independent validation, and by when? Owner assigned: the technical director. Budget approved: specific amount for independent evaluation. Timeline: validation complete by end of month two. If validation fails, the architecture team is authorised to begin assessment of alternative approaches immediately. + +Red item three — test environment procurement. Decision required: who will approve early procurement and what is the lead time? Owner assigned: the programme director. Procurement approved at the current meeting. Lead time confirmed: twelve weeks from approval. This puts the test environment available at month four. If delayed beyond month four, integration testing is impacted and the programme timeline adjusts accordingly. + +The commitment that has been made: deliver to the most likely estimate of seventeen months, contingent on the three decisions being made and actioned within their defined windows. The contract date of fourteen months is noted as the commercial target. The programme will be managed against the honest estimate while active steps are taken to close the gap where possible. If the three decisions are made and the risks are resolved, the best case of twelve months remains achievable. If the risks materialise despite action, the programme will rebaseline to the worst case immediately rather than absorbing the delay silently. + +The difference between these two scenarios is not the estimate. The estimate is identical. The difference is whether the governance layer treated the estimate as a decision input or as a number to be noted. + +--- + +## The principle restated + +A commitment is only genuine when the people responsible for delivery have set the date, had the risks accepted, had the implications acknowledged, and had the plan built with slack, honest foundations, and realistic assumptions. + +Presenting a plan to an engineering team and asking if it is achievable is not making a commitment. Working backwards from a desired date is not making a commitment. Noting a risk register and moving to task status is not accepting the conditions of an estimate. + +A commitment made without those conditions is a target with someone else's name on it. When it is missed — and it will be missed, because targets without conditions are not plans, they are aspirations — the accountability for missing it belongs with the people who made the target, not with the people who were handed it. + +Closing the gap between the people who make commitments and the people who keep them is the central governance challenge of engineering programme management. This chapter has described the structural mechanisms that close it. The chapters that follow build on them. diff --git a/content/firmitas/part-2-the-framework/10-risk-issues-and-the-register-that-actually-works.md b/content/firmitas/part-2-the-framework/10-risk-issues-and-the-register-that-actually-works.md new file mode 100644 index 0000000..2e3578c --- /dev/null +++ b/content/firmitas/part-2-the-framework/10-risk-issues-and-the-register-that-actually-works.md @@ -0,0 +1,155 @@ +--- +title: "Chapter 10: Risk, Issues, and the Register That Actually Works" +--- + +# Chapter 10: Risk, Issues, and the Register That Actually Works + +The risk register in Chapter 1 was not deficient. It contained the right information. The three red items were correctly identified, correctly rated, and correctly described. The engineering team had done exactly what a risk management process asks of them. + +The register failed not because it was wrong but because the system it fed into was not designed to act on it. The information entered the governance process and stopped. It was noted, acknowledged, filed, and superseded by the task status review. The register had been designed to produce a document. It had not been designed to produce decisions. + +This is the most common form of risk management failure in engineering programmes. Not the absence of a risk register. Not inaccurate risk identification. The presence of an accurate risk register that the governance system does not know how to use. + +This chapter describes what a risk register that actually works looks like — how it is structured, how it is reviewed, how it generates decisions rather than acknowledgements, and what the governance model around it must look like for it to function as the management instrument it is supposed to be. + +--- + +## The distinction between risks and issues + +Before describing what a functional risk register looks like, it is necessary to establish a distinction that most risk registers blur and that most risk management processes treat as optional: the difference between a risk and an issue. + +A risk is a condition that might occur and, if it does, would affect the programme's ability to deliver to its committed outcome. It has a probability — it may or may not materialise. It has a defined impact — if it materialises, this is what changes. And it has a window — a period within which action can prevent it from materialising or limit its impact if it does. + +An issue is a condition that has already occurred and is already affecting the programme. It has no probability — it is not a future event but a present reality. It has a current impact that needs to be managed, not a potential impact that needs to be mitigated. And it has a resolution path — a set of actions that will return the programme to its intended trajectory or, if that is no longer achievable, a revised trajectory that reflects the current reality. + +Confusing risks and issues produces a register that is difficult to act on. A risk that has already materialised is no longer a risk. Treating it as one — rating its probability, listing mitigation plans, keeping it in the risk register — obscures the fact that the programme is already carrying the consequence and needs a resolution response rather than a mitigation response. It also produces the specific failure pattern from Chapter 1, where risks that were never resolved gradually transitioned from potential problems to actual ones without the governance system ever formally acknowledging the transition. + +Firmitas requires risks and issues to be managed as separate categories, with separate governance responses. + +--- + +## What a risk entry must contain + +A risk entry in a Firmitas risk register is not a description of something that might go wrong. It is a decision request — a structured statement of a condition that requires a governance response before a defined trigger date. + +Each risk entry must contain six elements. + +**A specific, named description.** Not "integration risk" or "third-party dependency risk." Those are categories, not risks. A risk entry must describe a specific condition with enough precision that someone who did not write it can understand exactly what might happen, why it would happen, and what the consequence would be. "The third-party firmware library has not been validated against the programme's NB-IoT profile requirements. If validation reveals non-compliance, the architecture will require a redesign estimated at four weeks, pushing the programme from most likely to worst case." That is a risk entry. It is specific. It is actionable. It connects directly to the programme estimate. + +**A probability assessment.** Not a precise probability — engineering programmes do not admit that level of precision — but an honest characterisation of likelihood. High, medium, or low, with a brief rationale for the assessment. The rationale prevents probability ratings from drifting toward whatever makes the register look most comfortable. A risk rated low because it is genuinely unlikely is different from a risk rated low because rating it high would make a governance conversation difficult. The rationale makes the difference visible. + +**An impact assessment.** What changes to the programme if this risk materialises. Specifically: what changes to the timeline, to the cost, to the scope, or to the quality of what is delivered. The impact assessment must be connected to the programme estimate — if this risk materialises, the most likely case becomes the worst case, or the best case becomes unachievable, or the delivery date moves by this many weeks for these reasons. Impact assessments that are not connected to the programme estimate are not useful for governance decisions. + +**A named owner.** A specific person, not a team or a function, who is accountable for managing the risk. The owner is responsible for the mitigation actions, for monitoring the trigger conditions, and for escalating the risk when it transitions from manageable to critical. Risks without named owners are risks that nobody is managing. They persist on registers indefinitely, rated amber, until they materialise. + +**A mitigation or response plan.** What actions are being taken to prevent the risk from materialising or to limit its impact if it does. The mitigation plan must be specific — named actions, named owners, defined timelines — not aspirational. "We will work closely with the supplier to ensure the component is suitable" is not a mitigation plan. "The technical lead will commission and complete an independent validation of the component against the programme's requirements by end of month two. If validation reveals non-compliance, the architecture team will begin assessment of alternative approaches immediately and present options to the programme board by end of week two of month three" is a mitigation plan. + +**A trigger date or trigger condition.** The specific date or condition that will cause the risk to be escalated if it has not been resolved. The trigger date is the point at which the window for cheap resolution closes — after which either the risk has materialised or the actions required to prevent it are no longer available at the same cost. The trigger date must be connected to the programme timeline. If the third-party component validation is not complete by end of month two, the programme's ability to remain on the most likely timeline is compromised. That is the trigger date. At end of month two, if validation is not complete, the risk escalates automatically — not because the owner chooses to escalate it, but because the governance system is designed to escalate it. + +--- + +## What an issue entry must contain + +An issue entry has a different structure from a risk entry because it requires a different governance response. + +An issue is a present reality, not a future possibility. It does not require mitigation — the condition has already occurred. It requires active management toward resolution. + +Each issue entry must contain five elements. + +**A specific description of what has occurred.** Not what might happen — what has happened, when it happened, and what its current impact on the programme is. "Software component integration testing commenced on day one of month five. Test failures have been identified in three interface behaviours that were assumed to be correctly implemented based on supplier documentation. The failures are preventing completion of integration testing and are blocking the firmware validation phase. Current impact: two weeks of delay to the integration testing schedule, with potential downstream impact on the system validation phase start date." + +**The reason it occurred.** Not for blame — for learning and for informing the resolution approach. The reason the issue occurred usually reveals a gap in the programme's risk management, requirements definition, or governance that should be addressed to prevent similar issues. An issue that occurs because a risk was identified and not acted on is a different kind of issue from one that could not have been anticipated. Both require resolution. Only the first requires a governance reflection. + +**A named owner with authority.** The person responsible for driving the issue to resolution. The owner must have the authority to take the actions the resolution requires — or must have an explicit escalation path to someone who does if resolution requires authority beyond their own. + +**A resolution plan with actions and dates.** Specific actions, named owners for each action, and dates by which each action will be completed. The resolution plan is not a list of things that will be tried. It is a committed sequence of activities that will return the programme to its intended trajectory — or, if that is no longer achievable, will establish the revised trajectory and the governance decisions required to accept it. + +**An impact statement on the programme plan.** What the issue means for the programme's delivery date, cost, and scope. The impact must be stated honestly — if the issue has moved the programme from the most likely case toward the worst case, that should be visible in the issue entry and should be reflected in a formal revision of the programme plan. Issues whose programme impact is not formally acknowledged produce the silent accumulation of delay that the programme in Chapter 1 experienced — where the formal plan remained unchanged while the real programme drifted progressively further from it. + +--- + +## The register as a decision instrument + +A risk and issues register designed as described above is a fundamentally different tool from the risk register in Chapter 1. It is not a document that records what might go wrong. It is a live management instrument that tracks the specific conditions the programme depends on and surfaces the decisions required to keep those conditions met. + +For the register to function as a decision instrument, the governance process around it must be designed accordingly. + +The register must be reviewed at a frequency matched to the pace of change in the programme. A monthly review of a register whose trigger dates are measured in weeks is not risk management. It is post-hoc documentation of things that have already happened. On active delivery programmes, the register should be reviewed weekly by the programme management layer and at every programme governance meeting by the governance layer. Items approaching their trigger dates should be flagged automatically — not at the next scheduled review, but when the trigger window opens. + +The review must generate decisions. A risk register review that ends without decisions having been made on escalated items is not a review. It is an acknowledgement ceremony. The purpose of reviewing the register is not to confirm that the register has been updated. It is to make the decisions that the register is asking for. If a red item is in front of the programme board, the programme board meeting does not end until either a decision has been made or the decision has been explicitly deferred with a named owner, a rationale, and a date by which it will be made. + +The register must be connected to the programme plan. When a risk materialises or an issue is opened, the programme plan must be updated to reflect the impact. The register and the plan are not separate documents. They are the same programme, represented at different levels of detail. A risk that has materialised and is not reflected in a revised plan is a programme that is being managed against a fiction. + +--- + +## Management by exception — how senior management should use the register + +The risk and issues register is the primary instrument through which senior management should govern a programme. Not task status. Not RAG reports. The register. + +This is the governance principle that Chapter 1's programme board did not follow — and the one that would have produced a different outcome if they had. + +Senior management's job is not to track task completion. Task completion is the delivery team's concern — the operational management of day-to-day and week-to-week delivery that belongs with the people who have the knowledge to manage it. Senior management's job is to manage the conditions under which tasks can be completed — to make the decisions that the delivery team cannot make within their own authority, to provide the resources and the client engagement and the cross-programme trade-offs that only the governance layer can resolve. + +The risk and issues register surfaces exactly those things. Not the tasks being completed — the conditions being threatened. Not the activities in progress — the decisions required. The governance layer that reads the register correctly knows what it is being asked to do: make this decision, provide this resource, have this client conversation, resolve this dependency. The governance layer that ignores the register and reads the task status report instead knows what has been done. It has no view of what is at risk. + +Management by exception means the governance layer acts on escalated items — items that have breached their trigger date without resolution, items whose impact has changed, items that require authority or resource the delivery team does not have — and trusts the delivery team to manage everything within their authority without interference. + +For this to work, three things must be in place. + +The exception threshold must be explicit. Teams must know which items automatically escalate to senior governance and which they own. An ambiguous threshold produces two failure modes: teams escalating everything to avoid accountability, or teams holding everything to avoid appearing incapable. Both undermine the governance model. + +The governance response to escalated items must be timely. An item that escalates and sits without a governance response for two weeks is worse than one that was never escalated — it has consumed the team's attention, revealed the governance layer's inability to respond, and lost the time that a prompt decision would have preserved. + +The governance layer must distinguish between escalated items and status reports. A programme review that begins with escalated risks and issues and generates decisions on each of them before moving to anything else is a functional governance meeting. A programme review that begins with a RAG dashboard and a milestone completion review and reaches the risk register as the final agenda item — if it reaches it at all — is a programme review that has been designed around the wrong information. + +The principle, stated plainly: senior management governs programmes through risks and issues, not through task status. A programme review that spends more time on task status than on risks and issues has inverted its governance model. The question is not whether tasks are being done. The question is whether the conditions for doing them exist — and that question is answered by the risk register, not the Gantt chart. + +--- + +## The software component running late + +One of the most common issue types on engineering programmes is a software or firmware component that is running behind schedule. It is worth addressing directly because it illustrates how the risk and issues distinction operates in practice and what the governance response should look like. + +A component running late is not a risk. By the time it is visible as a delay, it is an issue — something that has already occurred and is already affecting the programme. It must be entered as an issue, not as a risk, and it must be managed with the governance response that issues require. + +What the governance layer needs to know is not that the component is late. It needs to know why it is late, what the resolution plan is, what the impact on the programme plan is, and what decisions are required from the governance layer to support the resolution. + +Why it is late matters because the reason determines the response. A component running late because the requirements were ambiguous is a requirements management failure that may have implications for other components being developed against the same requirements. A component running late because an external dependency has not been resolved is a dependency management failure that requires escalation to whoever owns the dependency. A component running late because the original estimate did not adequately account for the scope requires the governance layer to review other estimates developed under similar conditions. A component running late because the engineer responsible is overloaded is a resourcing failure that requires an immediate governance decision about capacity. + +Each of these causes requires a different response. The governance layer that receives "the component is two weeks late" without the reason, the resolution plan, and the impact statement cannot make the decision the situation requires. It can only express concern and request updates — which is what the programme board in Chapter 1 was reduced to, repeatedly, by a governance system that gave it insufficient information to act. + +The resolution plan must be specific. Named actions, named owners, defined dates. If additional resource is needed, the request must be explicit and the governance decision to provide or decline it must be made at the governance level, not deferred back to the delivery team to resolve within their own means. + +The impact on the programme plan must be stated honestly and immediately. If the component running late will push the integration phase back by three weeks, the programme plan shows that now — not at the next monthly review, not when other slippage can no longer be used to absorb the delay, now. A programme plan that does not reflect known delays is a fiction. A governance layer managing against a fiction cannot make real decisions. + +--- + +## The register in a regulated environment + +In programmes operating under formal safety or quality standards — IEC 62304, IEC 61508, ISO 9001, DO-178C, and their sector-specific equivalents — the risk register carries formal evidence obligations. Evidence of risk identification, assessment, and management is required by most safety-critical and quality management standards, and that evidence must be demonstrable to auditors and certifying bodies. + +The functional risk register described in this chapter satisfies these obligations more thoroughly than a compliance-driven one, because it generates evidence naturally through its operation rather than assembling it retrospectively for audit purposes. + +A risk entry with a specific description, probability and impact assessment, named owner, mitigation plan with dates, and trigger date is evidence-rich by design. When it is reviewed, the review generates a record. When a decision is made, the decision is documented. When a risk transitions to an issue, the transition and the reason are recorded. When an issue is resolved, the resolution actions and their outcomes are recorded. The register's operational lifecycle produces the audit trail that compliance requires as a natural by-product of its function. + +This is the Firmitas position on compliance documentation generally: evidence should be generated by a system that is functioning correctly, not assembled by a system that is functioning incorrectly and needs to appear compliant. A risk register that generates decisions and manages programme risk will produce better compliance evidence than one that produces formatted documents for governance meetings. The compliance follows the function. It does not substitute for it. + +--- + +## What the week-three meeting looks like with a functional register + +Return to the programme board at week three in Chapter 1. The same estimate has been presented. The same three red items are on the register. The difference is that the register has been designed as a decision instrument and the governance process has been designed to use it as one. + +The programme board does not note the three red items. It addresses them. + +Red item one — hardware-software interface ambiguity — has a trigger date of end of week five. If the ambiguity is not resolved by then, the firmware development that has been proceeding against current assumptions will require rework, and the programme will move from the most likely timeline toward the worst case. The decision required: who will have the resolution conversation with the client, and by when? The programme director accepts ownership. The conversation will be initiated within five working days. The trigger date stands at end of week five. If resolution is not achieved, the programme board will be convened for a rebaseline review. + +Red item two — third-party component validation — has a trigger date of end of month two. Independent validation must be completed by then to allow the architecture to proceed on its current assumptions. The decision required: who will commission the validation and what is the budget? The technical director accepts ownership. The budget is approved at the current meeting. The validation will be commissioned within the week. The trigger date stands at end of month two. + +Red item three — test environment procurement — has a trigger date of end of week four, after which the procurement lead time will make the required availability date of month three unachievable. The decision required: who will approve early procurement and what is the budget? The procurement approval is made at the current meeting. The order will be placed within five working days. + +The meeting ends. Three red items. Three decisions. Three owners. Three deadlines. Not noted. Addressed. + +The programme that follows is not guaranteed to succeed. The client may not resolve the interface ambiguity quickly. The component may still prove unsuitable. But each of those outcomes will be known earlier, when the options for responding are still available at a manageable cost. And the programme will be managed against reality rather than against a plan that has already diverged from it. + +That is what a risk register that actually works produces. Not certainty. The conditions under which programme governance can do its job. diff --git a/content/firmitas/part-2-the-framework/11-requirements-from-customer-need-to-testable-specification.md b/content/firmitas/part-2-the-framework/11-requirements-from-customer-need-to-testable-specification.md new file mode 100644 index 0000000..36c2bb3 --- /dev/null +++ b/content/firmitas/part-2-the-framework/11-requirements-from-customer-need-to-testable-specification.md @@ -0,0 +1,183 @@ +--- +title: "Chapter 11: Requirements: From Customer Need to Testable Specification" +--- + +# Chapter 11: Requirements: From Customer Need to Testable Specification + +Of all the ways an engineering programme can accumulate hidden risk, none is more consistent or more costly than the gap between what the customer needs and what the engineering team believes they have been asked to build. + +That gap is a requirements gap. It is produced by requirements that are too vague to be implemented unambiguously, too broad to be tested specifically, too high-level to be actionable without assumptions, or too poorly connected to their source to survive the changes that every programme experiences. It is compounded by the common organisational pattern of separating the people who write requirements from the people who understand the customer need and the people who will implement the requirement — so that by the time a requirement reaches an engineer, it may have passed through several layers of interpretation, each of which introduced a small divergence from the original intent. + +Requirements failures do not announce themselves immediately. They accumulate silently, in the assumptions engineers make when a requirement is ambiguous, in the interpretations that differ between the team that wrote the test and the team that wrote the code, in the integration failures that reveal incompatible assumptions about what an interface was supposed to do. By the time the gap is visible, it is expensive. Requirements gaps discovered during integration cost an order of magnitude more to resolve than requirements gaps discovered during specification. Requirements gaps discovered in the field cost an order of magnitude more again. + +The investment in requirements engineering — done properly, at the right level of rigour for the product and the programme — is the highest-return risk reduction activity available in engineering delivery. It is also among the most consistently under-resourced, undervalued, and prematurely curtailed activities in the programmes that fail. + +--- + +## The customer and the business + +Before addressing how requirements should be written, it is necessary to address where they come from — and specifically, to make explicit a distinction that many programmes obscure. + +The customer is not the business. + +In most engineering programmes, the requirements are provided by the business — the internal stakeholder, the product manager, the procurement team, or the commercial function that has engaged with the client. The requirements represent the business's understanding of what the customer needs, filtered through commercial priorities, contractual constraints, and the business's own interpretation of the customer's stated requirements. + +This filtering is not inherently wrong. The business has context the engineering team does not have — commercial constraints, regulatory obligations, portfolio priorities, client relationship history. But the filtering introduces a risk: the engineering team may be building what the business has specified rather than what the customer actually needs. These are not always the same thing. + +When they diverge, the consequences appear at the point of customer contact — in acceptance testing, in field deployment, in the support calls that reveal the gap between what was delivered and what the customer's operation required. At that point, the engineering team has built exactly what the requirements specified. The requirements specified something that did not fully reflect the customer's need. The cost of the gap is real. The attribution is unclear. And the engineering team has no way to advocate for a different outcome because they never had visibility of the customer reality that would have informed a different specification. + +Firmitas requires that engineering teams maintain a direct line of sight to the end customer — not mediated entirely by the business. This does not mean that engineers bypass the business relationship or have unsupervised access to clients. It means that the requirements process creates mechanisms for customer reality to inform the specification — through customer involvement in requirements reviews, through user research that precedes requirements writing, through validation activities that test requirements against real customer scenarios before they are baselined. + +The practical consequence for requirements quality is significant. An engineer who has seen how the customer actually uses the product — who has watched them interact with the interface, who understands the operational environment, who knows what failure looks like from the customer's perspective — writes different requirements from one who has only read a specification prepared by a commercial team. The requirements are more specific, more operationally grounded, and more likely to result in something the customer can actually use. + +Trade-offs made with customer context produce different outcomes from trade-offs made without it. When a feature must be descoped or a quality compromise must be accepted, the engineer who understands customer priorities can make that trade-off in the customer's interest. The engineer who does not know the customer can only make it in the interest of the programme schedule or budget. Those are not the same trade-off, and they do not produce the same result. + +--- + +## Requirements are not one size fits all + +The right level of requirements rigour depends on what is being built. This is a principle that is frequently violated in both directions — programmes that apply heavyweight requirements processes to products that do not need them, and programmes that apply user-story-level specification to products that absolutely require more. + +For a small increment to an existing software product — a new feature in an established application with well-understood architecture and a stable team — a well-written user story with clear acceptance criteria can satisfy most of the requirements properties that matter. The context is shared. The architecture is known. The customer feedback loop is short. The cost of discovering a requirements gap during development is low. + +For a full product — hardware, mechanical, firmware, software, communications, cloud infrastructure, operating in a regulated environment with safety obligations, delivering to a customer with operational dependencies on what is built — user stories are not adequate specification. The system is too complex, the integration dependencies too numerous, the regulatory obligations too specific, and the cost of discovering requirements gaps too high for the informal, iterative approach that works well for simpler products. + +The distinction is not about being agile versus being traditional. It is about being appropriate to the complexity and consequence of what is being built. A requirements process that produces ambiguous, catch-all specifications for a complex regulated product is not agile. It is a shortcut with deferred costs. + +--- + +## Properties of a well-written requirement + +A requirement is the basic unit of specification. Whether it appears in a formal requirements document, a user story, a system design specification, or an interface control document, the same properties distinguish a requirement that can be built and tested from one that will generate assumptions and ambiguity. + +**Unambiguous.** A well-written requirement has one interpretation. Two engineers reading it independently should build the same thing. If two engineers could build materially different implementations and both claim to satisfy the requirement, the requirement is ambiguous. Ambiguous requirements are not incomplete specifications — they are invitations for the implementation to be whatever the engineer decides is reasonable. On a programme with ten engineers and twenty ambiguous requirements, the assumptions may never be aligned. The integration failure that results is a requirements failure, not an integration failure. + +**Granular enough to be testable.** Every requirement must have a corresponding pass/fail condition. If a requirement cannot be tested — if there is no observable outcome that would confirm it has been met or demonstrate that it has not — the requirement is not finished. Testability is the most reliable quality gate for requirements, because it forces the question that ambiguity makes it easy to avoid: exactly what must the system do, under exactly what conditions, producing exactly what observable output, within exactly what tolerances? + +**Singular.** One requirement, one thing. Compound requirements — "the system shall encrypt data and authenticate users and log all access attempts" — are three requirements presented as one. Compound requirements allow partial compliance. An engineer can satisfy two of the three clauses and legitimately claim the requirement is met unless the requirement has been split. Splitting compound requirements is not pedantry. It is the only way to ensure that every obligation is tracked independently and can be tested individually. + +**Attributed.** Every requirement has a source — a customer need, a regulatory obligation, a safety analysis, an architectural constraint, a business rule. Attribution serves two purposes. It enables traceability — the chain from customer need to system behaviour to test evidence that compliance and quality management frameworks require. And it enables change management — when a requirement must be changed, the attribution tells you who owns the source need, what other requirements draw from the same source, and what the impact of the change is across the specification. + +**Bounded.** A requirement should state explicitly what is in scope and, where necessary, what is explicitly out of scope. Unbounded requirements are an invitation for scope creep. They are also an invitation for catch-all language — "the system shall handle all relevant error conditions" — that defers the specification decision to the implementer. The implementer will make that decision under delivery pressure, without stakeholder input, and the decision will be invisible in the programme record. + +**Verifiable at the right level.** A system requirement is verified by a system test. A software requirement is verified by a software test. A hardware requirement is verified by a hardware qualification test. Requirements written at the wrong level of abstraction either cannot be tested at the appropriate level or generate a cascade of derived requirements that should have been written explicitly rather than implied. The level of a requirement determines how it is verified. Writing requirements at the appropriate level for their intended verification method is not a documentation preference. It is what makes verification possible. + +**Agreed by both the stakeholder and the engineer.** A requirement that has been agreed only by the stakeholder who specified it may be commercially accurate but technically undeliverable. A requirement that has been agreed only by the engineer who will implement it may be technically sound but commercially wrong. Both parties must agree before the requirement is baselined — the stakeholder confirms that the requirement captures their need, and the engineer confirms that the requirement is implementable as specified. Any requirement that cannot achieve both signatures has not been finished. + +--- + +## Catch-all requirements and their cost + +Catch-all requirements are the most common and most damaging requirements failure mode. They are requirements written at a level of abstraction that appears to specify something while actually deferring all the specification decisions to the implementer. + +They are recognisable by their language. Words and phrases that reliably indicate a catch-all requirement include: appropriate, adequate, as required, suitable, as necessary, all relevant, where applicable, to an acceptable standard, and etc. Each of these phrases converts a specification decision into an implementation decision — one that will be made by an engineer, under time pressure, without stakeholder input, in a way that is invisible to the programme record and untraceable to any agreed requirement. + +The cost of catch-all requirements is not paid when they are written. It is paid when the implementation decisions they generate turn out to differ from what the stakeholder intended — which is discovered during acceptance testing, during integration, or in the field, at a point when the cost of correction is substantially higher than the cost of writing a proper requirement in the first place. + +A simple test for catch-all language: search the requirements document for the words "appropriate," "adequate," "as required," "suitable," and "etc." Count the occurrences. Each one is a requirements gap dressed as a specification. Each one is a decision that has been deferred to someone who should not be making it alone. + +--- + +## The Requirement Breakdown Structure + +For complex products — those involving multiple disciplines, regulatory obligations, and significant integration dependencies — requirements cannot be managed as a flat list. They must be structured hierarchically, with clear parent-child relationships that enable traceability, change management, and verification planning. + +The Requirement Breakdown Structure is the hierarchy that connects the highest-level contractual or business obligations to the most granular implementable and testable requirements. It is not an optional documentation choice. For complex multi-discipline programmes, it is the mechanism that makes delivery possible — the structure within which every requirement has a home, every obligation is traceable, and every change can be impact-assessed across the full specification. + +The hierarchy operates across four levels. + +```mermaid +flowchart TD + %% Left side — definition + L1["Level 1 — Business / Contractual<br>Customer obligation or need<br>Example: Conform to SMETS2"] + L2["Level 2 — System<br>System behaviour to satisfy L1<br>Example: HAN uses DLMS/COSEM"] + L3["Level 3 — Subsystem / Component<br>Discipline-specific behaviour<br>Example: ZigBee SEP 1.2 command set"] + L4["Level 4 — Derived<br>Emergent from design decisions<br>Example: Re-associate within 30s"] + + %% Right side — verification + V1["Acceptance<br>Verified against full specification"] + V2["System Integration Test<br>End-to-end behaviour validated"] + V3["Component Test<br>Subsystem behaviour validated"] + V4["Focused Test Case<br>Specific pass/fail criteria"] + + %% Decomposition (down) + L1 -->|Decomposes| L2 -->|Decomposes| L3 -->|Decomposes| L4 + + %% Verification (up the V) + L4 -->|Verified by| V4 + L3 -->|Verified by| V3 + L2 -->|Verified by| V2 + L1 -->|Verified by| V1 + + %% Traceability (diagonal intent) + L1 -.-> V2 + L2 -.-> V3 + L3 -.-> V4 + + %% Styling + style L1 fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style L2 fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style L3 fill:#EEEDFE,stroke:#534AB7,color:#3C3489 + style L4 fill:#FAEEDA,stroke:#BA7517,color:#633806 + + style V1 fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style V2 fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style V3 fill:#EEEDFE,stroke:#534AB7,color:#3C3489 + style V4 fill:#FAEEDA,stroke:#BA7517,color:#633806 +``` + +**Level 1 — Business or contractual requirements** are the highest-level obligations — the customer needs, contractual specifications, and regulatory obligations that define what the programme must achieve. They are typically stated at a level of abstraction that is not directly implementable. "The system shall conform to the SMETS2 specification" is a Level 1 requirement. It is correct, complete, and entirely inadequate as a specification for any engineering team. It is a pointer to a body of work that must be decomposed before anything can be built. + +**Level 2 — System requirements** define the system-level behaviours and properties required to satisfy the Level 1 obligations. They are the first level at which requirements become specific enough to guide architectural decisions. Each Level 2 requirement must be traceable to a Level 1 requirement — it exists because a Level 1 obligation requires it — and must be verifiable at the system level. + +**Level 3 — Subsystem or component requirements** define the specific behaviours required of individual subsystems, components, or disciplines. They are owned by the team responsible for the relevant discipline and are verifiable at the component or subsystem test level. Each Level 3 requirement traces to a Level 2 requirement. + +**Level 4 — Derived requirements** are requirements that emerge from design decisions — constraints that are not explicitly stated by the customer or contract but that are necessary to satisfy higher-level requirements given the architectural choices that have been made. They are important and frequently overlooked. An architect who selects a specific communication protocol, a specific memory architecture, or a specific timing model has implicitly committed to a set of derived requirements. If those requirements are not written explicitly, they exist as implicit assumptions — visible to the architect who made the design decision and invisible to everyone else. + +Acceptance criteria are not a separate level. They are embedded at every level — the specific, testable conditions that will confirm a requirement has been met. At Level 1, the acceptance criteria may be expressed as acceptance test scenarios. At Level 4, they are specific pass/fail conditions in a test case. + +--- + +## The traceability spine + +The Requirement Breakdown Structure is the foundation of requirements traceability — the ability to follow the thread from a customer need all the way to the test evidence that confirms it has been met, and back again. + +Traceability serves four purposes, all of them operational rather than bureaucratic. + +**Change management.** When a requirement changes — as requirements always do — traceability makes the impact visible. Which higher-level requirement is affected? Which derived requirements need revisiting? Which tests need to be rerun? Which design decisions are invalidated? Without traceability, change impact is assessed by asking the people who remember what they built and why. That knowledge is incomplete, inconsistent, and unavailable when the people who hold it have moved on. + +**Verification planning.** Traceability enables verification to be planned systematically — to confirm that every requirement at every level has a test that covers it, that no requirement is untested, and that no test exists without a traceable requirement that justifies it. Without traceability, verification coverage is estimated rather than confirmed. The gaps in coverage are the gaps that produce field failures. + +**Compliance evidence.** Most regulated environments require demonstrable traceability from requirements to verification evidence. A programme with a complete Requirement Breakdown Structure and maintained traceability can produce that evidence as a natural output of its normal activities. A programme without it must assemble the evidence retrospectively — an expensive, unreliable, and frequently unsuccessful exercise that becomes the primary source of audit stress. + +**Organisational memory.** Requirements traceability preserves the context of why things were built the way they were. When a new team member asks why the system behaves in a way that seems arbitrary or unnecessarily constrained, the answer should be traceable to a requirement — which traces to a Level 1 obligation or a regulatory constraint or a customer need. Without traceability, the answer is often "we've always done it this way" or "I think someone decided that at some point" — which is not an answer, and which cannot be used to evaluate whether the constraint is still necessary as the programme evolves. + +--- + +## The investment argument + +Requirements engineering done properly takes time. It requires skilled people — not just technical writers, but people who understand both the customer domain and the technical implementation space well enough to write requirements that are specific, testable, and achievable. It requires stakeholder engagement — the client or business representatives who own the needs must participate in requirements review, not just sign off on a document presented to them. It requires iteration — requirements are not correct the first time, and the process of identifying ambiguities, resolving conflicts, and writing acceptance criteria for difficult requirements takes multiple cycles. + +This investment has a cost that is visible immediately and a return that is visible later — which is why it is consistently under-resourced in programmes under schedule pressure. + +The return is measured in what does not happen. The integration failure that would have occurred if the interface requirement had remained ambiguous does not occur because the ambiguity was resolved during specification. The six weeks of rework that would have been required when the firmware was written against an incorrect interpretation of the hardware specification is not required because the specification was correct. The acceptance test failure that would have occurred when the client discovered the implementation did not meet their operational need does not occur because the requirements were validated against the customer reality before they were baselined. + +These are costs that never appear on a project budget because they never occurred. They are the invisible return on the requirements engineering investment. The programmes that cut requirements engineering to save time and money find out about the return the hard way — when the costs that were avoided by doing it properly materialise as failures during integration, acceptance, or operation. + +For a simple product, the investment required is modest. For a complex multi-discipline product — hardware, firmware, software, in a regulated environment, with safety obligations — the investment is substantial. It is also the most significant single factor in whether the programme delivers what was intended, on time and at manageable cost, or spends its later phases recovering from the consequences of specifications that were not clear enough to build from. + +Requirements engineering is not overhead. It is the primary act of programme risk reduction. Treat it accordingly. + +--- + +## Requirements management is not one size fits all — but it always has to be clear + +The level of formality, the tools used, the documentation structure, and the governance process around requirements will differ by programme type, scale, and regulatory context. A startup building its first mobile application and a tier-one defence contractor building a safety-critical control system will — and should — approach requirements management differently. + +What does not differ is the obligation to be clear. + +A user story that says "as a user I want to log in so that I can access my account" is adequate for a consumer application with a short feedback loop to real users and a team that can test and iterate quickly. The same format applied to "as a system operator I want the metre to respond to a remote disconnect command" in a smart metering programme is not adequate — not because user stories are a wrong format, but because the behaviour being specified is complex, the consequences of misspecification are significant, the regulatory obligations are specific, and the feedback loop is too long to rely on iteration to surface the gaps. + +The question is never "which requirements methodology should we use." The question is "are our requirements clear enough that the engineering team can build exactly what is needed without making specification decisions that belong to the stakeholder, and precise enough that we can test unambiguously whether what has been built meets what was required?" + +If the answer is yes, the requirements process is appropriate to the programme. If the answer is no, the process needs to be more rigorous — regardless of how agile or how lean the team's preferred working style is. diff --git a/content/firmitas/part-2-the-framework/12-slack-flow-and-why-efficiency-thinking-fails.md b/content/firmitas/part-2-the-framework/12-slack-flow-and-why-efficiency-thinking-fails.md new file mode 100644 index 0000000..fd05fab --- /dev/null +++ b/content/firmitas/part-2-the-framework/12-slack-flow-and-why-efficiency-thinking-fails.md @@ -0,0 +1,181 @@ +--- +title: "Chapter 12: Slack, Flow, and Why Efficiency Thinking Fails" +--- + +# Chapter 12: Slack, Flow, and Why Efficiency Thinking Fails + +There is a belief, held as firmly as the estimation assumptions described in Chapter 2 and almost as destructive, that the most efficient engineering organisation is the one where every person is fully utilised — where nobody is idle, every hour is allocated, and the system is running at maximum capacity. + +This belief feels intuitively correct. It looks right on a resource plan. It satisfies the instinct that paying for capacity means using it. It is, in the context of complex engineering delivery, reliably wrong — and the consequences of acting on it are visible in the overrun statistics, the burnout rates, and the compounding technical debt of organisations that have optimised their people and systems to within an inch of their limits. + +This chapter is the canonical treatment of slack and flow in Firmitas. Both concepts appear throughout the rest of the book because both are relevant throughout the lifecycle of any engineering programme. What they mean, why they matter, and why efficiency thinking consistently destroys them is established here. Later chapters will apply these concepts to specific contexts — the delivery model, team design, governance, portfolio management, AI-assisted delivery — without repeating the foundational argument. + +--- + +## What flow actually is + +Flow is not speed. It is not the rate at which tasks are completed or the velocity at which a team moves through its backlog. It is the rate at which value progresses through the entire system — from intent to delivered outcome, across all the stages, handoffs, and validation steps between them. + +Good flow is characterised by predictable lead times, small batch sizes, frequent feedback, and early risk discovery. The value being created moves through the system smoothly, surfaces problems quickly, and produces learning that can be acted on while the cost of acting is still manageable. + +Poor flow is characterised by long and variable lead times, hidden queues, late integration, and surprise failures. Work appears to be progressing — tasks are being completed, milestones are being hit — while the real programme state is diverging from the plan in ways that will only become visible when it is expensive to respond. + +The programme in Chapter 1 had poor flow. Firmware was being developed against an ambiguous specification for months before the ambiguity surfaced as a problem. The third-party component was treated as a settled dependency while the validation that would have confirmed its suitability was deferred. The integration phase — the point at which the programme's real state would become visible — was scheduled for the end, after all the development work had been done, when the cost of discovering problems was at its highest. + +This is what poor flow looks like in practice: work progressing through the system while the risks embedded in that work remain invisible. The system looks efficient because people are busy. The value is not flowing. It is accumulating risk invisibly while appearing to advance. + +--- + +## Queues — the hidden cost centre + +The most important concept in flow management is one that almost never appears on a programme dashboard: the queue. + +A queue is where work waits. Waiting for a review. Waiting for a decision. Waiting for a dependency to be resolved. Waiting for a test environment to be available. Waiting for a specification ambiguity to be clarified. Waiting for the next sprint. Waiting for the next release window. + +Queues are invisible in most programme reporting. Work in a queue still appears "in progress." The task is assigned, the milestone is tracked, the team is occupied. The queue is not measured. Its cost is not calculated. Its existence is not surfaced as a risk. + +But queues are where the cost is. Every day a piece of work spends in a queue is a day that the feedback it would generate is delayed. Every week that integration is deferred is a week that the assumptions embedded in the code being written remain untested. Every month that a risk sits unresolved because the decision required to resolve it has not been made is a month in which the risk is accumulating cost. + +The specific failure in Chapter 1 was a queue failure. The three red risk items were queued — waiting for decisions that required authority the programme board had but was not applying. The ambiguity in the hardware-software specification was queued — waiting for a client conversation that required programme-level initiation. The test environment was queued — waiting for a procurement decision that had not been made. None of these were active blockers in the sense that work had visibly stopped. The firmware team was writing code. The architecture team was progressing. The programme appeared to be in flow. It was not in flow. It was accumulating hidden queues whose cost would surface in months four, five, and six. + +Queue length is a more meaningful signal of programme health than task completion rate. An organisation that measures task velocity without measuring queue depth is measuring local activity while the system accumulates global risk. + +--- + +## Why full utilisation destroys flow + +The mathematical relationship between utilisation and lead time is one of the most counterintuitive and most important facts about complex system performance. It is the reason why the efficiency instinct — maximise utilisation, eliminate idle time, keep everyone fully occupied — reliably produces worse outcomes rather than better ones. + +As a system's utilisation rate increases toward 100%, its queue length increases non-linearly. A system operating at 80% utilisation can absorb moderate variation without significant queue growth. A system operating at 95% utilisation cannot absorb variation at all — the smallest perturbation produces a queue, the queue produces delay, and the delay compounds because the system has no spare capacity to absorb it. + +In engineering programme terms: a team operating at 80% utilisation has the capacity to absorb a technical problem that takes an unexpected two days to resolve, to respond to a clarifying question from a downstream team, to investigate an integration failure before it becomes a blocker. A team operating at 100% utilisation has none of that capacity. Every unexpected event becomes a crisis. Every clarifying question becomes a delay. Every integration failure becomes a multi-week recovery. + +The highly utilised team appears more productive in a resource plan. On a dashboard, every hour is allocated and every person is occupied. In practice, the team's throughput is lower, its lead times are longer, its defect rates are higher, and its ability to respond to the unexpected — which is not unexpected in complex programmes but entirely predictable — is negligible. + +This is why full utilisation is not efficiency. It is the systematic elimination of the system's ability to absorb the variation that complex work reliably produces. Organisations that drive toward full utilisation in the name of efficiency are buying the appearance of productivity at the cost of actual throughput. + +DeMarco described this as the paradox of full employment: the fully employed organisation is not the most productive one. It is the most fragile one. McConnell quantified the cost of adding resource to an overloaded project — a system already at full utilisation cannot absorb new resource without increasing coordination overhead, reducing the productivity of existing members, and lengthening rather than shortening the timeline. Brooks described the same dynamic in 1975 and called it the mythical man-month. The mathematics have not changed. The organisational behaviour that ignores them has not changed either. + +--- + +## What slack actually is + +Slack is not idleness. It is not waste. It is not the absence of useful activity. It is capacity reserved by design to absorb the variation that complex systems produce, to enable learning that cannot happen when every moment is committed to planned work, and to maintain the system's ability to respond when the unexpected arrives — which, in complex engineering programmes, is a matter of when rather than if. + +Slack exists in several forms, each serving a different function. + +**Time slack** is capacity that is not committed to planned tasks — time available for investigation, for reflection, for the technical debt reduction that never makes it onto a sprint because there is always something more urgent. Time slack is what allows an engineer to properly investigate a performance anomaly rather than closing the ticket as "no issue found" and moving on. It is what allows an architect to evaluate an emerging technology before the programme has committed to a direction that the technology makes obsolete. It is what allows a team to conduct the retrospective that produces the process improvement rather than the retrospective that produces a list of items nobody has time to act on. + +**Cognitive slack** is mental capacity that is not occupied by context switching, interruption management, or the accumulated cognitive load of too many concurrent priorities. Cognitive slack is what allows engineers to think about hard problems rather than managing the overhead of too many partially-completed tasks. When cognitive slack is exhausted — when engineers are managing too many threads simultaneously, responding to too many demands, tracking too many dependencies — the quality of their technical work degrades. Not because they are less capable, but because deep technical work requires sustained attention that a cognitively overloaded system cannot provide. + +**Structural slack** is architectural and organisational flexibility — the capacity of systems to absorb change without cascading failure. An architecture with structural slack can accommodate a changed requirement without requiring a complete redesign. A team structure with structural slack can absorb the departure of a key person without the programme halting. A delivery system with structural slack can absorb an integration failure without pushing the programme into crisis. Structural slack is the difference between a system that bends under pressure and one that breaks. + +**Portfolio slack** is unallocated capacity at the programme or portfolio level — the buffer that allows the portfolio to absorb a priority change, a strategic opportunity, or a programme failure without every other initiative being disrupted. Portfolios with no slack are portfolios where everything is at risk the moment any single initiative encounters a problem. The slack in the portfolio is what makes the portfolio manageable rather than fragile. + +--- + +## Slack and quality + +The relationship between slack and quality is direct and underappreciated. + +Quality degrades when systems are overloaded. Not because engineers become careless under pressure, but because the activities that maintain quality — proper test coverage, thorough code review, adequate investigation of defects, proper documentation of decisions, deliberate refactoring of code that has accumulated debt — are the first casualties of a system operating at full utilisation. + +These activities are never urgent in the way that delivering a feature is urgent. They can always be deferred to the next sprint, the next release, the next programme. They accumulate as a backlog of quality debt that inflates invisibly while the programme's task completion rate stays green. When the debt surfaces — in integration failures, in production incidents, in the rework that occurs when the accumulated compromises in test coverage produce defects that should have been caught earlier — the cost is substantially higher than the cost of maintaining quality throughout. + +Slack is the condition under which quality activities can be given the time they require. A team with adequate slack can conduct a proper code review rather than a superficial one. It can investigate a test failure rather than marking it as known and deferring it. It can refactor the module that has become unmaintainable rather than writing another layer of abstraction on top of it. It can write the documentation that allows the next engineer to understand what was built and why rather than the documentation that exists to satisfy a gate review. + +The organisations that claim to value quality while operating at full utilisation are structurally dishonest. Quality requires time. Time requires slack. Eliminating slack in the name of efficiency eliminates the conditions under which quality can be maintained — and then attributes the quality failures to execution rather than to the structural decision that produced them. + +--- + +## Slack and learning + +Learning requires time and attention. It requires the capacity to reflect on what happened, to investigate why, to experiment with alternatives, and to incorporate the conclusions into future practice. None of this is possible when every unit of capacity is pre-committed to planned delivery. + +The compounding cycle described in Chapter 5 — the cycle by which each failure makes the next failure more likely — is fundamentally a learning failure. The organisation does not correctly identify the cause of the failure, does not draw the right lesson, and does not change the conditions that produced the failure. This is partly a governance failure, as Chapter 5 described. It is also a slack failure — because correctly identifying failure causes, drawing useful lessons, and implementing systemic changes all require time and attention that organisations at full utilisation do not have. + +Systems without slack repeat mistakes. They defer technical debt indefinitely. They rely on heroics to compensate for structural weakness. They respond to incidents by restoring service as quickly as possible and return to full utilisation before any systemic investigation has occurred. The next incident is therefore not prevented. It is merely deferred. + +Systems with slack detect issues earlier, because engineers have the cognitive capacity to notice and investigate anomalies rather than moving directly to the next task. They experiment safely, because the system can absorb the cost of an experiment that does not work. They adapt continuously, because the governance system has the capacity to implement improvements rather than deferring them until the next programme. + +Slack is the price of learning. Organisations that eliminate slack eliminate learning — and then wonder why the same problems recur across programmes. + +--- + +## Slack and the people who do the work + +The human cost of eliminated slack is not soft. It is operational. + +When teams are permanently at full utilisation, several things happen with predictable regularity. Burnout increases — not as a personal weakness but as the physiological and psychological consequence of sustained cognitive overload without recovery. Attrition increases — because engineers who are competent enough to have options will leave a system that chronically overloads them for one that does not. Knowledge is lost with each departure, and the cost of that knowledge loss does not appear on the capacity plan. + +The quality of technical decisions degrades under sustained pressure. Engineers working at the edge of their capacity make different decisions from engineers working within it. They cut corners that a less pressured colleague would not cut. They defer the investigation that would have identified the architectural problem before it became the technical debt that the next programme inherits. They write the code that solves the immediate problem rather than the code that addresses the underlying cause. Each individual decision is rational under the conditions. The accumulation of those decisions is the compounding technical debt that makes future programmes slower and more expensive than they should be. + +Principle 12 states that your people are your most significant investment — trust and respect their judgement or destroy what you spent years building. Permanently eliminating the slack that allows them to do good work is one of the clearest ways an organisation can signal that it does not mean what Principle 12 says. The consequences are predictable. They are real. And they are paid long after the programme that produced them has been closed. + +--- + +## Deliberate slack versus accidental waste + +A specific and important distinction must be made: slack is not the same as waste, and conflating them is one of the primary ways that slack elimination is rationalised. + +Waste is activity that does not contribute to value, learning, or resilience. Unnecessary meetings. Duplicate documentation. Processes that exist to satisfy a governance requirement that does not improve the programme. Rework caused by requirements ambiguity that should have been resolved during specification. These are waste. Eliminating them is beneficial. + +Slack is capacity intentionally reserved to enable the outcomes that pure productive activity cannot achieve — the investigation of anomalies, the refactoring of accumulated debt, the learning from incidents, the cognitive space required for deep technical work. Eliminating slack in the name of eliminating waste is not efficiency. It is the systematic destruction of the system's ability to improve itself and the people within it. + +The governance test for whether a reduction in apparent utilisation represents waste elimination or slack elimination is straightforward: does the reduction improve quality, learning, or resilience over time, or does it reduce them? Waste elimination improves outcomes over time. Slack elimination degrades them, progressively and predictably. + +--- + +## Flow in the programme in Chapter 1 + +The programme in Chapter 1 had the appearance of flow. People were working. Tasks were being completed. The programme was moving. + +It did not have the substance of flow. The real state of the programme — the ambiguous interface specification, the unvalidated third-party component, the queued procurement decision — was not surfacing in the programme's feedback mechanisms because those mechanisms were not designed to surface it. The feedback loop was task completion. The signal that mattered was risk materialisation. The programme was measuring the wrong thing and concluding that the right things were happening. + +A programme designed for flow would have looked different from month one. The firmware development against the ambiguous specification would not have proceeded at full pace — it would have been paced to the resolution of the interface ambiguity, with a small spike to understand the risk and a hold on the development that depended on the ambiguous elements until the ambiguity was resolved. The integration of the third-party component would have been attempted early, with a small vertical slice through the architecture, to confirm whether the supplier's claims were credible before the architecture was built around them. The test environment procurement would have been on the critical path from day one, not an afterthought. + +Each of these changes would have reduced the apparent utilisation of the development teams in the short term. The firmware team would not have been at full utilisation while the specification ambiguity was being resolved. The architecture team would have been doing integration work rather than full development while the component was being validated. The programme, measured by task completion rate, would have looked less productive in weeks two through six. + +In weeks eight through eighteen, it would have been significantly more productive — because the problems that cost six weeks and four weeks of rework in the actual programme would have been identified in days, at a point when the cost of correction was measured in hours rather than weeks. + +This is the fundamental trade-off that flow-based management requires: accepting lower apparent utilisation in the short term to achieve higher real throughput over the full programme. It is a trade-off that is difficult to make in organisations that measure task completion and easy to make in organisations that understand that flow efficiency matters more than resource efficiency. + +--- + +## The political difficulty of slack + +Slack is politically uncomfortable. It looks inefficient on a resource plan. It is difficult to justify to a governance layer that measures utilisation. It is the first thing removed during cost-cutting, and the last thing restored when pressure eases. + +This is not irrational. The benefits of slack are often invisible until slack is gone — and by then, the cost of its absence is paid by a different team on a different programme, attributed to execution rather than to the structural decision that produced it. The person who eliminated the slack has moved on. The programme that suffers the consequence is treated as a separate failure. + +Making slack visible — naming it explicitly, defending it as a design decision rather than an inefficiency to be managed away, connecting its preservation to specific programme health outcomes — is a leadership responsibility. Leaders who accept that slack will be consumed by "just one more request" or "just this once" are not protecting their teams from pressure. They are transferring the cost of that pressure onto the quality of what is delivered and the wellbeing of the people delivering it. + +The principle states: slack is not waste. It is the capacity to improve. An organisation that genuinely believes this will protect slack when it is uncomfortable to do so. An organisation that says it believes this but eliminates slack under pressure does not believe it. The behaviour, not the language, reveals the actual position. + +--- + +## Flow and slack as connected properties + +Flow and slack are not independent. They are connected properties of the same system, and managing one requires attention to the other. + +Good flow requires slack — because flow is the smooth progression of value through a system, and that smooth progression requires the capacity to absorb variation, investigate anomalies, and respond to problems before they become queues. A system with no slack cannot maintain good flow under the variation that complex programmes reliably produce. Every unexpected event creates a queue. Every queue degrades flow. Every degraded flow increases lead times, reduces feedback frequency, and delays risk discovery. + +Slack enables flow — by providing the capacity to address the problems that would otherwise become queues. When an engineer has cognitive and time slack, the technical anomaly gets investigated before it becomes a blocker. The integration test failure gets resolved before it delays the downstream phase. The specification ambiguity gets escalated and resolved before the code written against it needs to be rewritten. + +The virtuous cycle is available: adequate slack enables rapid response to variation, which prevents queues from forming, which maintains good flow, which enables early risk discovery, which reduces the cost of the problems the programme encounters, which means less unplanned work, which preserves the slack that enables the next round of rapid response. + +The vicious cycle is more common: insufficient slack means every variation creates a queue, queues degrade flow, degraded flow delays risk discovery, delayed risk discovery means problems surface late and expensively, expensive problems consume more unplanned work, which consumes more of the slack that was already insufficient. + +The programme in Chapter 1 was in the vicious cycle from the first day. The conditions that produced it — full utilisation, long feedback loops, deferred integration, unresolved risks — were set at programme initiation. The outcome was determined long before the delivery date was missed. + +--- + +## What this means for governance + +The governance implication of the flow and slack argument is specific and direct. + +Governance decisions that drive teams toward full utilisation in the name of efficiency are not governance decisions. They are the systematic creation of conditions that guarantee the outcomes they are trying to prevent. A governance layer that insists on full resource allocation, that treats any unallocated capacity as waste, that eliminates slack at every opportunity to reduce apparent cost, is governing against programme health in the name of financial discipline. + +The correct governance signal is not utilisation rate. It is flow efficiency — the proportion of total lead time in which work is actively progressing rather than waiting. And the correct governance question about capacity is not "is everyone fully occupied?" It is "does the team have sufficient slack to absorb the variation this programme will encounter, to learn from what it produces, and to maintain the quality that the delivered product requires?" + +Those are harder questions to answer from a governance dashboard. They require the governance layer to understand how the programme actually works — which brings the argument back to Chapter 8 and the alignment of decision authority with delivery knowledge. A governance layer that understands flow and slack will govern differently from one that understands only utilisation. And the programmes it governs will deliver differently. diff --git a/content/firmitas/part-2-the-framework/13-architecture-as-delivery-infrastructure.md b/content/firmitas/part-2-the-framework/13-architecture-as-delivery-infrastructure.md new file mode 100644 index 0000000..67ff67b --- /dev/null +++ b/content/firmitas/part-2-the-framework/13-architecture-as-delivery-infrastructure.md @@ -0,0 +1,135 @@ +--- +title: "Chapter 13: Architecture as Delivery Infrastructure" +--- + +# Chapter 13: Architecture as Delivery Infrastructure + +Architecture is one of the most consistently misunderstood concepts in engineering programme management. In many organisations it is treated as a phase — something that happens at the beginning of a programme, produces a set of documents and diagrams, and is then handed over to the development teams who get on with the real work. In others it is treated as a role — the architect who attends design reviews, approves technical decisions, and maintains the system diagrams. In others still it is treated as a constraint — a set of standards and patterns that teams must comply with, enforced through governance gates. + +None of these treatments is wrong exactly. But all of them are incomplete in a way that matters. + +Architecture is not a phase, a role, or a set of compliance requirements. It is the accumulated set of decisions that shape how a system behaves, changes, fails, and evolves over time. It is delivery infrastructure — the underlying structure that determines what is easy, what is hard, what is safe, and what is dangerous, long before any development team writes a line of code or assembles a circuit board. + +That framing has a specific implication for engineering leadership: architectural decisions are not primarily technical decisions. They are delivery decisions. They determine how teams must coordinate, where risk accumulates, how quickly changes can be made, and what the programme will cost to evolve over its lifetime. Leadership accountability for architectural quality is not a preference. It is a direct consequence of leadership accountability for system outcomes. + +--- + +## Architecture encodes behaviour + +Every significant architectural decision embeds assumptions about how the system will be used, maintained, and evolved. Those assumptions are not always stated explicitly. They are encoded in the choices themselves — in the coupling between components, in the boundaries between subsystems, in the interfaces that define how the parts interact, in the dependencies that constrain what can change independently and what must change together. + +These embedded assumptions shape the behaviour of the delivery organisation in ways that are often invisible until they cause problems. + +A tightly coupled architecture forces coordination. Teams that own different parts of a tightly coupled system cannot change their part without understanding and potentially affecting the parts it depends on. They must synchronise with each other. Their lead times are extended by the need to coordinate. Their autonomy is constrained not by governance decisions but by the technical structure of what they are building. + +A poorly defined interface creates hidden contracts. When the boundary between two components is not explicitly specified, both teams develop implicit expectations about how the interface will behave. Those expectations differ. The difference is invisible until the two components are integrated — at which point the assumptions encoded on each side collide, and the cost of resolving the collision falls on the integration phase rather than on the design phase where it should have been addressed. + +An architecture that concentrates risk amplifies failure. When failure in one component can propagate to many others, the cost of defects and incidents is multiplied by the connectivity of the system. An architecture that isolates failure — that bounds the impact of any single failure to the component in which it occurs — is not just technically cleaner. It is commercially and operationally less expensive to operate and maintain over time. + +These are not abstract properties. They are the specific conditions that determined the cost of the failures in Chapter 1. The hidden coupling between the firmware and the ambiguous hardware-software interface meant that six weeks of development work was invalidated when the ambiguity was resolved. The architectural dependency on the unvalidated third-party component meant that the redesign required when the component proved unsuitable affected elements of the system that had already been completed and tested. The architecture encoded assumptions that had not been validated. The cost of those unvalidated assumptions was paid at integration. + +--- + +## Architecture and flow + +Chapter 12 established that flow — the smooth progression of value through the entire system — is the primary indicator of programme health. Architecture is one of the primary determinants of flow. It shapes the dependencies, coupling, and integration points that determine how easily value can progress from intent to outcome. + +Architectures that support flow share common properties. Ownership boundaries are clear — each team knows what it owns and what it does not. Interfaces between components are explicit and stable — they can be relied upon by both sides without requiring constant coordination to maintain. Coupling is minimal — changes in one part of the system do not cascade unpredictably into others. Integration can happen early and frequently, because the architecture has been designed to make it safe. + +Architectures that impede flow share different properties. Dependencies are hidden — not documented, not acknowledged, discovered when one team's change breaks another's implementation. Interfaces are implicit — understood informally by the people who built them, not specified formally in a way that can be relied upon by people who did not. Coupling is pervasive — changes in one area propagate unpredictably to others, making every change an exercise in archaeological investigation. Integration is deferred — because the architecture makes early integration expensive rather than cheap. + +When architecture constrains flow, no amount of delivery process optimisation compensates. Teams that appear to be moving slowly may simply be navigating an architecture that makes progress expensive. The diagnosis that the team needs better processes, or more resource, or tighter management misses the cause entirely. The cause is structural. The response must be structural too. + +--- + +## Architecture as risk distribution + +Every architectural decision is a risk distribution decision — a choice about where risk will reside in the system, when it will become visible, and how expensive it will be to address. + +Tightly coupled architectures concentrate risk. A change in one area can affect many others in ways that are not immediately apparent, making testing expensive and incomplete, and producing the integration failures that surface late and are difficult to diagnose. Architectures that isolate concerns distribute risk. Changes are localised. Failures are contained. Testing can be targeted to the specific area of change rather than requiring system-wide regression. + +The decisions made early in a programme's architecture are the highest-leverage risk decisions the programme makes. An architectural choice that places a system's highest-uncertainty element behind a well-defined interface buys the programme significant optionality — the ability to change the implementation behind the interface without affecting the rest of the system. An architectural choice that allows a high-uncertainty element to become deeply embedded in the system's core trades that optionality for apparent short-term simplicity, and pays for it when the uncertainty resolves as a problem rather than a validation. + +This is why Firmitas treats architecture as a risk management activity at least as much as a technical one. The relevant question when evaluating an architectural decision is not only whether it is technically elegant. It is: where does this decision push risk? When will we discover whether that risk has materialised? How expensive will recovery be at the point of discovery? + +An architecture that delays risk discovery creates false confidence while accumulating actual cost. The programme appears to be progressing because tasks are being completed. The risk is not visible because the architectural decisions that embedded it have not yet been tested. When the integration phase arrives and the embedded risk surfaces, the discovery is experienced as a failure of execution rather than a consequence of an architectural choice made months earlier. + +--- + +## Evolutionary architecture + +The false dichotomy between rigorous upfront design and uncontrolled emergent architecture is one of the most persistent sources of architectural failure in engineering programmes. The rigid upfront design camp produces architectures that are theoretically complete but practically disconnected from the realities that emerge as the programme develops — realities about integration complexity, about performance characteristics, about operational requirements that were not fully understood at design time. The emergent architecture camp produces systems that accumulate structural debt as local decisions pile up without reference to a coherent whole, until the resulting architecture is too entangled to evolve safely. + +Firmitas rejects both extremes in favour of evolutionary architecture — architectures that are intentional but adaptable, that establish coherent structure early while preserving the ability to revise that structure as understanding improves. + +Evolutionary architecture requires two things that are often in tension. The first is early architectural investment — enough upfront thinking to establish the foundational decisions that will shape the programme's delivery and evolution. Ownership boundaries. Interface definitions. Coupling management. Risk distribution. These decisions are too important to be left to emerge from local choices made under delivery pressure. They need to be made deliberately, at the right time, with the right people in the room. + +The second is continuous architectural evolution — the discipline to revisit architectural decisions when the evidence suggests they need revising, rather than treating them as permanent commitments that cannot be changed. An architectural decision that made sense under the assumptions that existed at month one may not make sense under the understanding that exists at month six. A programme that cannot revise its architecture as it learns is a programme that will carry the cost of its early assumptions to completion and beyond. + +The tension between these two requirements is managed through the governance of architectural decisions — specifically, through Architecture Decision Records that make the rationale for decisions visible, the assumptions they rest on explicit, and the conditions under which they should be revisited clear. + +--- + +## Architecture Decision Records + +Architectural decisions carry long-term consequences. When the rationale behind a decision is lost — when future teams know what was decided but not why, or what alternatives were considered, or what trade-offs were accepted — the decision becomes an invisible constraint. It cannot be challenged because its reasoning cannot be reconstructed. It cannot be evolved because the understanding that would guide evolution has been lost. It can only be worked around, at accumulating cost, until the workarounds become the architecture. + +Architecture Decision Records exist to prevent this. An ADR is a lightweight, structured record of a significant architectural decision — written at the time the decision is made, preserved in version control alongside the code and documentation it relates to, and accessible to future teams who need to understand the system they have inherited. + +An ADR answers five questions. What decision was made? Why was it made — what context, what constraints, what goals drove the decision? What alternatives were considered, and why were they not chosen? What trade-offs were accepted — what does this decision make easier and what does it make harder? What are the consequences of the decision — what future choices does it constrain, what risks does it introduce, under what conditions should it be revisited? + +The value of an ADR is not primarily as a documentation artefact. It is as an organisational memory mechanism. The engineer who joins the programme in month twelve and encounters an architectural constraint that seems arbitrary has access to the reasoning behind it. The programme board that is evaluating a proposed architectural change can understand what the original decision was protecting against and whether those concerns still apply. The team that is maintaining the system five years after the original delivery can understand why it was built the way it was, and can make informed decisions about how to evolve it. + +ADRs are not heavy. They do not require architectural review boards or extensive approval processes. They require the discipline to write down the reasoning at the time it is freshest — at the moment of decision — rather than attempting to reconstruct it later from code and memory. The cost of writing an ADR is minutes. The cost of not having one is paid over years. + +--- + +## Living design + +Architectural documentation that is produced upfront and never updated is not documentation. It is archaeology. The system as it was designed and the system as it was built diverge almost immediately, and by the time the programme is operating in production, the upfront documentation typically describes a system that no longer exists. + +The alternative is not to abandon documentation. It is to treat documentation as a living system that evolves alongside the code and hardware it describes — version-controlled, reviewed through the same mechanisms as other engineering artefacts, updated when decisions change, accurate enough to be useful rather than comprehensive enough to be impressive. + +Living design in the Firmitas sense includes several types of artefact. Architecture Decision Records capturing the reasoning behind significant decisions. C4 diagrams at the appropriate level of abstraction — context, containers, components — showing the structural shape of the system and the boundaries between its parts. Interface Control Documents defining the contracts between components and subsystems, versioned and maintained as the interfaces evolve. Deployment and operational views showing how the system runs in its real environment, not just how it was designed. + +The key property of living design artefacts is that they are maintained where the work happens — in the same repositories as the code they describe, reviewed by the same people who review the code, updated as part of the same workflow rather than as a separate documentation exercise that always gets deferred. If a design artefact cannot be updated with a pull request, it will decay. The coupling between architectural design and engineering practice is not optional. It is what makes the design useful rather than historical. + +--- + +## Architecture across disciplines + +In programmes that span multiple engineering disciplines — hardware, mechanical, firmware, software — architecture is not a single-discipline concern. The architectural boundaries between disciplines are the highest-risk interfaces in the programme, because they are the points at which different design assumptions, different development cadences, and different change management approaches must be reconciled. + +The hardware-software interface in Chapter 1 was an architectural boundary. The failure at that boundary — the ambiguities in the specification that produced six weeks of firmware rework — was an architectural failure as much as a requirements failure. The interface had not been defined to the level of precision required to support independent development on both sides. The assumptions embedded in the firmware development diverged from the assumptions embedded in the hardware design. The divergence was invisible until integration. + +Managing architectural boundaries across disciplines requires explicit interface definitions — Interface Control Documents that specify, with sufficient precision, what each side of the boundary provides and depends on, and how changes on one side will be communicated to the other. It requires cross-discipline involvement in architectural decisions — hardware engineers in firmware architecture reviews, systems engineers spanning the boundaries between software and hardware, integration engineers involved from the earliest design discussions. And it requires early integration — the discipline to bring components together at the architectural boundaries earlier than feels necessary, when the cost of discovering problems is still manageable, rather than later when every identified problem is a programme-threatening crisis. + +--- + +## Architecture as a leadership responsibility + +The architectural decisions that most affect programme outcomes are made early, when the scope is least well understood, when the people making the decisions have the least information about the constraints that will matter, and when the temptation to defer hard decisions in favour of apparent progress is highest. + +These are precisely the conditions under which leadership stewardship matters most. Not because leaders should be making architectural decisions themselves — in most programmes, they should not — but because leaders are accountable for ensuring that the conditions for good architectural decision-making exist. + +That means ensuring that the people making architectural decisions have the information they need — customer requirements, operational constraints, regulatory obligations, performance characteristics, integration dependencies. It means protecting the time for proper architectural investigation rather than forcing architectural decisions under schedule pressure. It means creating the governance mechanisms — ADRs, design reviews, interface control processes — that make architectural decisions visible and accountable rather than implicit and untraceable. And it means resisting the organisational pressure to treat architecture as a phase that can be completed and closed rather than as a continuous activity that must evolve with the programme. + +The programme board in Chapter 1 that confirmed the delivery date without understanding the architectural risk embedded in the unvalidated third-party component dependency was exercising leadership on the wrong level of abstraction. The commercial decision was made. The architectural consequence was not considered. The cost of that omission appeared six months later as a programme-threatening redesign. + +Architecture is not someone else's problem. In the Firmitas sense, it is everyone's problem — but specifically, it is the problem of whoever is accountable for the programme's outcomes. Because the architecture determines the cost of the programme's problems more than any other single factor. And the cost of architectural problems, when they arrive, is always substantially higher than the cost of preventing them would have been. + +--- + +## What architecture looks like when it is done well + +The contrast between the architecture that produced Chapter 1's failures and one designed with delivery in mind is instructive. + +An architecture designed for delivery would have placed the hardware-software interface ambiguity at the centre of the early programme work rather than treating it as a background risk. The interface would have been specified explicitly — in an Interface Control Document, reviewed by both the firmware and hardware teams, confirmed with the client — before significant development began on either side. The ambiguities would have surfaced during specification, when they cost hours to resolve, rather than during development, when they cost weeks. + +The third-party component dependency would have been treated as an architectural risk from day one — a load-bearing element of the architecture that required validation before the architecture was built around it. A small integration spike — running the component against a representative subset of the programme's requirements — would have been scheduled as an early programme activity, producing the validation evidence that the architecture needed before committing to the component. If the validation revealed problems, the architecture would have been revised at the point of minimum cost. + +The integration testing would have been designed into the architecture — continuous integration across discipline boundaries, with early and frequent integration checkpoints rather than a single late integration phase. Problems at the architectural boundaries would have been discovered in days rather than months, when the cost of resolution was manageable rather than programme-threatening. + +None of this is architectural sophistication. It is architectural discipline — the application of systematic thinking to the decisions that matter most, at the time when they can still be made cheaply, with the information and the governance support to make them well. + +That is what architecture as delivery infrastructure means in practice. diff --git a/content/firmitas/part-2-the-framework/14-the-delivery-model.md b/content/firmitas/part-2-the-framework/14-the-delivery-model.md new file mode 100644 index 0000000..afc2b71 --- /dev/null +++ b/content/firmitas/part-2-the-framework/14-the-delivery-model.md @@ -0,0 +1,342 @@ +--- +title: "Chapter 14: The Delivery Model" +--- + +# Chapter 14: The Delivery Model + +Every engineering programme operates within a delivery model — a structure that shapes how work is sequenced, how disciplines interact, how decisions are made at key transition points, and how the programme as a whole moves from intent to outcome. Most programmes inherit their delivery model by default: they use the methodology the organisation has always used, or the one the client expects, or the one the team most recently worked within. + +The default delivery model is almost never the right one. It is the familiar one. + +The programme in Chapter 1 used a delivery model that concentrated integration at the end, deferred validation, and provided no mechanism for surfacing the architectural risks that were visible at week three. The firmware was developed for months against an ambiguous specification. The third-party component was assumed to be suitable without early validation. The test environment was procured too late to be useful during the critical phases. These were not execution failures. They were delivery model failures — the predictable consequences of a structure that deferred feedback and discovery to the point where their cost was highest. + +This chapter describes the Firmitas delivery model — not as a prescriptive process to be followed, but as a structural response to the specific failure patterns that delivery models most commonly produce. It is a hybrid model that combines the structure that complex programmes require with the iterative learning that complex programmes need. Neither pure sequential nor pure agile. A delivery shell that uses structured foundations to establish the conditions for delivery, an iterative development core that produces learning as its primary output, and rigorous validation that builds confidence continuously rather than testing everything at the end. + +--- + +## Why pure sequential models fail + +Sequential delivery models — waterfall, V-model, stage-gate — provide structure, traceability, and clear phase boundaries. These are real advantages, particularly in regulated and safety-critical environments where evidence of controlled development is a compliance requirement. + +Their failure mode is equally consistent. They assume that requirements can be completely specified before design begins, that design can be completed before implementation begins, and that the knowledge required to make good decisions at each phase exists at the point where those decisions must be made. In complex programmes, these assumptions are wrong. + +Requirements are never complete at the start of a programme. They are approximately correct, progressively refined, and never fully stable. Design decisions made on the basis of incomplete requirements embed assumptions that will be invalidated as understanding improves. Implementation completed against a design that rests on wrong assumptions requires rework. The rework arrives at the worst possible time — when the programme is under pressure to complete, when the cost of change is highest, and when the governance structure has least tolerance for the honest acknowledgement that earlier decisions need revisiting. + +The sequential model also defers integration. In a programme where each phase must be completed before the next begins, integration happens after all the components have been individually developed. The assumptions embedded in each component's development — about interfaces, about timing, about shared behaviour — are not tested against each other until the integration phase. The integration phase is therefore where the programme discovers what the architecture chapter described: the cost of unvalidated assumptions, accumulated across months of independent development, surfacing as integration failures that cannot be resolved quickly. + +--- + +## Why pure agile models fail in complex programmes + +Iterative and agile approaches address the sequential model's primary weakness — they incorporate learning loops, produce working software early, and adapt to changing requirements. For software-dominant products with short feedback loops and low-cost change, they work well. + +Their failure mode in complex multi-discipline programmes is different and equally consistent. Pure agile assumes that requirements can be discovered through iteration, that architectural decisions can emerge from incremental development, and that the cost of change is low enough that adapting continuously is preferable to investing in upfront understanding. + +In programmes that involve hardware, mechanical systems, firmware, and regulated compliance, these assumptions break down. Hardware cannot be iterated rapidly — the cost and lead time of physical design changes are substantially higher than the cost of software changes. Regulatory frameworks require evidence of controlled development that emergent architecture cannot easily provide. Safety-critical systems require formal verification against specified requirements that must be stable enough to verify against. + +The specific failure mode of pure agile in complex programmes is architectural drift — the accumulation of design decisions made locally, under sprint pressure, without adequate consideration of their system-level implications. Individual sprints produce working increments. The architecture that accumulates across sprints is incoherent, tightly coupled, and expensive to evolve — because no single iteration was responsible for the whole, and the whole was never deliberately designed. + +--- + +## The Firmitas delivery shell + +The Firmitas delivery model addresses the failure modes of both approaches through a delivery shell that provides the structure that complex programmes require while preserving the iterative learning that complex programmes need. + +The shell has three phases. + +```mermaid +flowchart LR + subgraph shell["The Firmitas Delivery Shell"] + direction LR + SF["Structured\nFoundations\nAlignment\nnot certainty"] + IDC["Iterative\nDevelopment\nCore\nLearning as\nprimary output"] + RV["Rigorous\nValidation\nConfidence\nnot discovery"] + + SF -->|"Gate 1\nFoundations\nComplete"| IDC + IDC -->|"Gate 2\nDevelopment\nComplete"| RV + RV -->|"Gate 3\nRelease\nReadiness"| OUT["Operational\nDeployment"] + + SF -.->|"Continuous\nfeedback"| IDC + IDC -.->|"Continuous\nV&V"| RV + RV -.->|"Risk\nlearning"| IDC + end + + style SF fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style IDC fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style RV fill:#EEEDFE,stroke:#534AB7,color:#3C3489 + style OUT fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A +``` + +The phases are sequential in the sense that each creates the conditions for the next. They are not sequential in the sense that each must be fully completed before the next begins — feedback flows continuously between phases, and activities within each phase overlap in ways that the programme's specific context will determine. + +--- + +## Phase 1: Structured foundations + +### Purpose + +The structured foundations phase exists to establish the conditions for delivery. Not certainty — certainty is unavailable at the start of a complex programme and should not be pretended. The goal is alignment: a shared, sufficiently detailed understanding of what is to be built, why it matters, what constraints apply, what risks exist, and what the programme will need in order to proceed. + +The foundations phase is not big design up front. It is the minimum investment in shared understanding that makes the iterative core productive rather than chaotic. A programme that enters the iterative core without adequate foundations will spend its early iterations discovering things that could have been known before the first sprint — and the cost of that discovery will be paid in rework rather than in the time the foundations would have required. + +### What structured foundations produce + +**Understood requirements.** Not complete requirements — complete requirements are a fiction at this stage. Requirements that are clear enough at the system level to guide architectural decisions, and broken down to the subsystem level for the components with the highest architectural impact and the highest delivery risk. The Requirement Breakdown Structure described in Chapter 11 is the instrument. The level of decomposition required depends on the programme's complexity and the cost of discovering requirements gaps during development. + +**Established architecture.** Not detailed design — detailed design belongs in the iterative core. The architectural decisions that establish the system's structural boundaries, define the interfaces between disciplines, distribute risk, and determine the framework within which the development teams will work. The decisions that, if made wrong, would require the most expensive corrections later. These decisions must be made before the iterative core begins, because the iterative core will build on them. + +**Validated key assumptions.** The architectural assumptions that carry the most risk — the unvalidated third-party dependency, the unconfirmed performance characteristic, the interface whose specification contains ambiguities — must be investigated during the foundations phase. Not exhaustively, but specifically: targeted investigations of the assumptions that would be most expensive to discover as false during the iterative core. The spike, the prototype, the early integration test — these are foundations-phase activities whose purpose is to replace dangerous assumptions with validated understanding. + +**A live risk register.** The risks that were identified during the foundations phase, structured as decision requests with owners, trigger dates, and mitigation plans as described in Chapter 10. Not the compliance risk register that exists to satisfy a governance requirement — the management instrument that the programme board will use to govern the programme through its risks and issues rather than through its task status. + +**A three-point estimate.** Produced by the people who will do the work, grounded in the scope understanding produced during the foundations phase, structured as best-case, most-likely, and worst-case with explicit conditions and risks attached. The estimate produced during the foundations phase is substantially more reliable than one produced before it — because the cone of uncertainty has narrowed as requirements have been defined and architectural assumptions have been validated. + +**Operations involvement.** The operational context established. The teams who will support and operate what is built have been involved in NFR definition, deployment strategy, monitoring requirements, and support model design. Their input has shaped what is being built, not just what will eventually be handed to them. + +### Entry criteria + +The structured foundations phase begins when the programme has sufficient scope definition to establish what needs to be understood, and sufficient resource to do the understanding work properly. It does not require a complete specification — it requires enough to begin. The specific entry threshold will vary by programme, but in general: a clear statement of what the programme is for, the key stakeholders and their primary needs, the major constraints (regulatory, commercial, technical), and a preliminary identification of the programme's highest-risk areas. + +### Exit criteria + +The structured foundations phase is complete — and the programme is ready to enter the iterative development core — when: + +The requirements are clear at the system level and decomposed to the level where the development teams can begin work on the highest-priority items without making specification decisions that belong to the stakeholder. + +The architecture establishes clear boundaries and interfaces between disciplines, the key architectural risks have been investigated, and the most significant unvalidated assumptions have been addressed. + +The risk register is live, owned, and connected to the programme plan. The three red items — or their equivalent on this programme — have named owners with authority and trigger dates that are being actively managed. + +The three-point estimate has been produced, presented to programme governance, and accepted — meaning the conditions attached to the estimate have been acknowledged and the decisions the risks require have been assigned. + +The programme governance structure has been confirmed to operate by exception through the risk and issues register, not through task status reporting. + +If these conditions are not met, the programme is not ready to enter the iterative core. Entering before they are met is not an acceleration. It is the importation of foundational uncertainty into the development phase, where it will surface as rework rather than as resolved questions. + +--- + +## Phase 2: The iterative development core + +### Purpose + +The iterative development core is where the system is built. Its primary output is not features or components — it is learning. Every iteration should produce validated understanding of something that was uncertain — that a design approach works, that a performance characteristic is achievable, that an interface functions as specified, that a risk identified during the foundations phase has been resolved or requires escalation. + +This is the phase that agile methods are best suited to within the delivery shell. The structure established during the foundations phase — the architectural boundaries, the interface definitions, the requirements decomposition — provides the framework within which iterative development can operate productively. Teams have enough shared understanding to work without constant coordination. They have clear boundaries that tell them what is within their scope and what is not. They have interface definitions that allow them to develop against stable contracts without requiring the teams on the other side of the interface to have completed their work. + +### What the iterative core produces + +Each iteration produces a small vertical slice through the system — a piece of working, integrated, tested functionality that has been demonstrated against real acceptance criteria. Not a component in isolation. Not a feature that has been unit-tested but not integrated. A slice through the architecture that has been tested across the boundaries it depends on and has produced learning that can be used to update the programme's understanding of what is still to come. + +The iterative core is also where the risk register is actively worked. Each iteration should retire at least one risk — by validating an assumption, resolving a dependency, completing an investigation, or demonstrating that a technical approach achieves what the programme needs it to achieve. Iterations that produce features without retiring risks are not producing the programme's most valuable output. + +### Cross-discipline synchronisation + +The most practically challenging aspect of the iterative development core in complex multi-discipline programmes is synchronisation across disciplines that operate on different cadences. Software teams can iterate in days or weeks. Firmware teams may have longer cycles due to hardware dependencies. Hardware teams operate on weeks-to-months cycles driven by PCB fabrication, component procurement, and physical build times. Mechanical teams may have even longer cycles for tooling and manufacturing. + +These cadences cannot be reconciled into a single sprint rhythm. Requiring hardware teams to work in two-week sprints is not agile. It is the imposition of a software development cadence on a discipline for which it is inappropriate. Requiring software teams to synchronise with hardware build cycles wastes the advantage that software's lower change cost provides. + +The resolution is not a unified cadence but a structured synchronisation model — a set of integration points at which the outputs of different discipline streams are brought together for cross-boundary validation. + +```mermaid +flowchart TD + %% Streams + subgraph hw["Hardware stream — 8–12 week cycles"] + H1["Design spike"] --> H2["PCB v0.1<br>Prototype"] + H2 --> H3["PCB v1.0<br>Engineering sample"] + H3 --> H4["PCB v2.0<br>Validation build"] + end + + subgraph fw["Firmware stream — 2–4 week cycles"] + F1["Spike on simulator"] --> F2["Core drivers<br>on dev kit"] + F2 --> F3["Feature development<br>on v0.1 HW"] + F3 --> F4["Full feature set<br>on v1.0 HW"] + end + + subgraph sw["Software stream — 1–2 week cycles"] + S1["API design<br>and stubs"] --> S2["Core services"] + S2 --> S3["Integration with<br>firmware interface"] + S3 --> S4["Full integration"] + end + + %% Enablers (not just handoffs) + H1 -.->|"Interface constraints"| F1 + H2 -->|"Early dev hardware"| F2 + F2 -.->|"Stable firmware API"| S2 + H3 -->|"Engineering sample"| F3 + F3 -->|"Integrated firmware"| S3 + H4 -->|"Production-like HW"| F4 + F4 -->|"System-ready firmware"| S4 + + %% Progressive integration (key improvement) + F2 --> INT1["Early integration<br>Dev kit + services"] + F3 --> INT2["Mid integration<br>v0.1 system slice"] + F4 --> INT3["Late integration<br>v1.0 full system"] + + S2 --> INT1 + S3 --> INT2 + S4 --> INT3 + + INT1 --> INT2 --> INT3 --> INT["System validation<br>and acceptance"] + + %% Styling + style hw fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style fw fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style sw fill:#EEEDFE,stroke:#534AB7,color:#3C3489 + + style INT1 fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A + style INT2 fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A + style INT3 fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A + style INT fill:#FAEEDA,stroke:#BA7517,color:#412402 +``` + +The synchronisation model has three properties. Each stream develops at its natural cadence — hardware on hardware timescales, firmware and software on software timescales. Integration points occur when both sides of an interface have something to integrate against — not on a calendar schedule but at the natural points where one stream's output becomes the other's input. And the integration is real — actual components, on actual hardware, testing actual interfaces — not simulated or deferred. + +The programme in Chapter 1 did not have a synchronisation model. Firmware was developed against a software model of the hardware while the hardware was being designed. When the hardware arrived, the firmware assumptions about the interface diverged from what the hardware actually provided. Early synchronisation — using the hardware specification to drive an early firmware spike, then an integration point as soon as development hardware was available — would have surfaced this divergence in weeks rather than months. + +### What done means in a cross-discipline programme + +In software-only development, done is relatively well defined — the feature works, the tests pass, the acceptance criteria are met. In a cross-discipline programme, done must be defined in terms of the integration state of the work, not just the completion state of individual components. + +A firmware component that passes all its unit tests but has never been integrated with the hardware it runs on is not done. A software service that passes all its service tests but has never been integrated with the firmware interface it depends on is not done. Done, in the Firmitas sense, means integrated and tested across the boundaries that matter — not every possible boundary, but the boundaries that carry the highest integration risk for the programme at this stage. + +This definition of done drives the early integration discipline that the delivery model requires. Teams that can claim done without integration will defer integration. Teams for whom done requires integration will integrate early — and will surface the problems that early integration is designed to find at the point where they are cheapest to address. + +### Architectural integrity during iteration + +The iterative core is where architectural drift most commonly occurs — where local decisions made under sprint pressure accumulate into structural debt that makes future development progressively more expensive. Chapter 13 described the mechanisms. The delivery model's response to this is specific. + +Architectural guardrails — the principles, patterns, and boundaries established during the structured foundations phase — are not optional within the iterative core. They are the framework within which iteration operates. A development decision that violates a guardrail is not a pragmatic adaptation to delivery pressure. It is a governance decision that requires explicit review — an Architecture Decision Record that explains why the guardrail is being revisited, what the trade-offs are, and whether the revision is accepted by the governance layer that owns the architectural framework. + +This does not slow development. It provides the visibility that allows the governance layer to maintain architectural coherence without micromanaging development decisions. The teams develop within the guardrails. When they encounter a guardrail that the new information they have suggests should be revised, they raise it through the ADR process. The revision is either accepted — with the rationale recorded — or declined — with the reasoning explained. Neither outcome requires a governance meeting. Both outcomes are visible and traceable. + +### Entry criteria + +The iterative development core begins when the structured foundations phase exit criteria have been met. The programme has understood requirements, an established architecture, a validated set of key assumptions, a live risk register, and a three-point estimate that programme governance has genuinely accepted — including the conditions and risks. + +### Exit criteria + +The iterative development core is complete when: + +All planned capability has been implemented and tested to the level required for system-level validation. + +All significant risks identified during the foundations phase have been retired or have been escalated and accepted as residual risks with defined management plans. + +The system has been integrated across all significant discipline boundaries and the integration failures that those integration points revealed have been resolved. + +The acceptance criteria for all baselined requirements have been demonstrated in a representative environment. + +The documentation — ADRs, interface control documents, operational runbooks, requirement traceability — is complete, current, and accurate. + +Operations teams have been progressively involved through the iterative core and are ready to support the transition to live operation. + +--- + +## Phase 3: Rigorous validation + +### Purpose + +Rigorous validation is not a testing phase. Testing occurs continuously throughout the iterative core. Rigorous validation is the phase in which the programme builds the evidence base that the system performs as required under real or representative operational conditions, and that the accumulated evidence meets the programme's quality, compliance, and regulatory obligations. + +The distinction matters. A programme that defers all validation to a late phase is not practicing rigorous validation. It is practicing discovery — finding out late what should have been confirmed early. Rigorous validation, in the Firmitas sense, is the culmination of a continuous validation process that has been running since the foundations phase, producing progressive confidence rather than binary pass/fail at the end. + +### What rigorous validation produces + +**System-level evidence.** Demonstration that the integrated system meets its acceptance criteria under representative conditions. Not component-level test results — system-level behaviour, across the full integration surface, under the conditions the system will experience in operation. + +**Compliance evidence.** The traceability from requirements through design decisions through test evidence that regulated programmes require. Not assembled retrospectively for the audit — generated continuously through the programme's operation and compiled here into the formal assurance record. For programmes operating under IEC 62304, IEC 61508, ISO 9001, or equivalent standards, this evidence is the deliverable that enables certification or regulatory acceptance. + +**Operational readiness evidence.** Demonstration that the system can be deployed, monitored, and supported in its operational environment. That the runbooks are accurate. That the diagnostic capabilities are sufficient. That the deployment process is reliable. That the operations team understands the system well enough to support it without depending on the delivery team. + +**A residual risk position.** A formal statement of the risks that the programme has not been able to fully retire, the basis for accepting them, and the conditions under which they would require escalation post-deployment. Every programme deploys with some residual risk. The question is whether that risk is known, bounded, and consciously accepted — or unknown and accumulating. + +### Entry and exit criteria + +The rigorous validation phase begins when the iterative development core exit criteria have been met. It does not begin because a calendar date has been reached. If the development core exit criteria have not been met, the programme is not ready for validation — and proceeding regardless is not confidence. It is the systematic generation of validation failures that will need to be resolved before the validation can complete. + +The rigorous validation phase is complete — and the programme is ready for the Gate 3 release readiness decision — when: + +All system-level acceptance criteria have been demonstrated against baselined requirements with documented evidence. + +Compliance evidence is complete, traceable, and has been reviewed by the appropriate quality or regulatory authority. + +Operational readiness has been confirmed by the operations team, not declared by the delivery team. + +The residual risk position has been reviewed, accepted by the governance layer with authority to accept it, and documented. + +No open issues exist that would prevent safe, reliable, and compliant operation. + +--- + +## Gates as decision points + +The three gates in the delivery shell are not approval ceremonies. They are decision points — moments at which the programme's readiness to proceed to the next phase is evaluated against evidence, and a genuine decision is made to proceed, pause, or change direction. + +Gate 1 — Foundations Complete — is the decision that the programme has established the conditions for productive iterative development. The evidence is the foundations phase deliverables: requirements state, architectural establishment, key assumptions validated, risk register operational, estimate accepted. + +Gate 2 — Development Complete — is the decision that the iterative development core has produced the integrated, tested capability that the validation phase will build its evidence case on. The evidence is the development core exit criteria: capability implemented and tested, risks retired, integration validated, acceptance criteria demonstrated. + +Gate 3 — Release Readiness — is the decision that the system is ready for operational deployment. The evidence is the validation phase deliverables: system-level acceptance, compliance evidence, operational readiness, residual risk position. + +Each gate is a genuine risk and commitment decision. Proceeding through a gate means accepting that the evidence presented is sufficient to take on the commitments of the next phase. A gate that is approved because the date has been reached rather than because the evidence is sufficient is not a governance decision. It is the transfer of risk to the people who will discover the consequences. + +A gate can and should stop a programme when the evidence does not support proceeding. A programme that has never had a gate stop it is a programme whose gates are not functioning as decision points. + +--- + +## The traceability spine + +The delivery model produces a body of evidence that connects the customer need to the delivered outcome. This traceability spine is not a separate documentation exercise. It is the natural output of a programme that has been run with the disciplines described in this book. + +```mermaid +flowchart TB + CN["Customer need<br>Outcome and value"] + SR["System requirements<br>Externally observable behaviour"] + CR["Component requirements<br>Allocated responsibilities"] + DI["Design intent<br>Architecture decisions (ADRs)"] + BC["Built capability<br>Executable system"] + TE["Test evidence<br>Objective proof"] + + %% Forward flow (intent) + CN -->|"Requirements engineering"| SR + SR -->|"Decomposition"| CR + CR -->|"Architecture decisions"| DI + DI -->|"Implementation"| BC + BC -->|"Verification"| TE + + %% Backward flow (evidence) + TE -->|"Verifies"| CR + TE -->|"Confirms"| SR + TE -->|"Validates"| CN + + %% Missing but critical feedback + TE -->|"Challenges assumptions"| DI + BC -->|"Reveals constraints"| DI + + %% Styling + style CN fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style SR fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style CR fill:#EEEDFE,stroke:#534AB7,color:#3C3489 + style DI fill:#FAEEDA,stroke:#BA7517,color:#412402 + style BC fill:#F1EFE8,stroke:#5F5E5A,color:#2C2C2A + style TE fill:#EAF3DE,stroke:#3B6D11,color:#173404 +``` + +Every requirement traces to the customer need that generated it. Every design decision traces to the requirements it addresses and is captured in an ADR. Every component of the built system traces to the design intent it implements. Every test traces to the acceptance criteria it demonstrates. The chain is complete, navigable in both directions, and current — because it has been maintained throughout the programme rather than assembled at the end. + +This traceability spine serves four purposes, each of which has been described in earlier chapters. Change management — when a requirement changes, the impact is visible across the full chain. Verification coverage — no requirement exists without a test that demonstrates it has been met. Compliance evidence — the chain from customer need to test evidence is the artefact that regulated programmes require. Organisational memory — the chain preserves the reasoning that connects what was built to why it was needed, available to future teams who must evolve what the programme delivered. + +--- + +## The delivery model in the programme of Chapter 1 + +The failures of Chapter 1 read differently against the delivery model. + +The unvalidated hardware-software interface assumption would have been addressed during the structured foundations phase. The interface would have been defined to the level required for independent development on both sides before the iterative core began. The ambiguities would have been resolved — or explicitly flagged as open questions that the foundations phase had not yet resolved and that the gate 1 decision would need to address. + +The unvalidated third-party component would have been treated as a key assumption requiring investigation during the foundations phase. A targeted integration spike would have been scheduled as a foundations-phase activity. The validation would have been completed before the architecture was committed to the component. If validation revealed problems, the architecture would have been revised during the foundations phase, at the lowest possible cost. + +The test environment would have been on the critical path from day one — not because a project plan said so, but because the delivery model establishes that rigorous validation requires a validated test environment, and that operational readiness is a gate 3 criterion that depends on the test environment being available in time to be used. + +The programme board would have governed through the risk register, not through task status. The three red items at week three would have been decision requests that the programme board could not leave the meeting without addressing. The window to change the outcome would not have closed unnoticed. + +This is not a guarantee of success. Complex programmes encounter problems that no delivery model can fully anticipate. What the delivery model provides is the structural conditions under which problems surface early, when they are manageable, rather than late, when they are not. diff --git a/content/firmitas/part-2-the-framework/15-quality-as-a-system-property.md b/content/firmitas/part-2-the-framework/15-quality-as-a-system-property.md new file mode 100644 index 0000000..8bb0dbd --- /dev/null +++ b/content/firmitas/part-2-the-framework/15-quality-as-a-system-property.md @@ -0,0 +1,133 @@ +--- +title: "Chapter 15: Quality as a System Property" +--- + +# Chapter 15: Quality as a System Property + +Quality failures are rarely the result of engineers who did not care about quality. In most organisations that deliver products with significant defects, the engineers knew the quality was insufficient. They knew which tests had been skipped, which requirements had been ambiguously implemented, which technical debt was accumulating in ways that would cost more to address later. They knew because they were the ones making the compromises — rationally, under pressure, in a system that made quality the most available casualty when schedule and cost were threatened. + +This is the defining characteristic of quality as a system property: it is not produced by individual commitment and destroyed by individual carelessness. It is produced by systems designed to make quality the natural outcome of the way work is done, and destroyed by systems that make quality compromises the rational response to the conditions they create. + +The implication for leadership is direct. Quality cannot be mandated. It cannot be inspected in at the end. It cannot be delegated to a quality function that operates separately from the delivery process. Quality is a consequence of how the system is designed — the architecture, the requirements, the feedback loops, the incentives, the verification practices, and the slack that makes doing things properly possible. Changing the quality outcomes requires changing those system properties, not issuing quality directives. + +--- + +## Quality and the failure in Chapter 1 + +The programme in Chapter 1 produced poor quality outcomes not because engineers were careless but because the system produced them. + +Firmware was written against an ambiguous specification. Not because the firmware engineers did not know the specification was ambiguous — they did, and they said so. But the system did not resolve the ambiguity before development began. The risk was noted, the mitigation was requested, and the development proceeded because the system had no mechanism for stopping development until the ambiguity was resolved. The quality deficit was encoded in the programme at the moment the ambiguity was left unresolved. + +The third-party component was integrated without independent validation. The quality assumption — that the component would behave as specified — was never tested before the architecture was built around it. When the assumption proved wrong, the quality deficit manifested as an architectural redesign that cost weeks rather than the days a validation spike would have required. + +The test environment was not available during the phases when its absence mattered most. Integration testing was deferred to the phase when the test environment became available. The defects that integration testing would have found were found instead during late-programme validation, when the cost of addressing them was substantially higher. + +Each of these quality failures was systemic. Each was visible in advance. None was addressed by the system in time to prevent the cost. + +--- + +## Built-in quality versus inspected quality + +The distinction between built-in quality and inspected quality is fundamental to understanding both why quality fails and how to address it. + +Inspected quality is the detection of defects after they have been created. Testing finds the bugs. Reviews find the design problems. Audits find the compliance gaps. Detection is necessary — but detection after the fact is the most expensive form of quality assurance available. The cost of a defect increases by roughly an order of magnitude for every phase boundary it crosses undetected. A requirements error found during specification costs hours to fix. Found during integration, it costs days. Found during acceptance testing, it costs weeks. Found in the field, it costs the relationship. + +Built-in quality is the design of systems so that defects are prevented rather than detected — or detected as early as possible, when the cost of correction is still low. It requires that quality considerations are present at every stage of the delivery process: in how requirements are written, in how architecture is designed, in how development is conducted, in how work transitions from one team to another. Quality is not a downstream activity. It is a property of every upstream decision. + +The shift from inspected to built-in quality is not the abandonment of testing. Testing remains essential. The shift is in where the emphasis lies — not on finding problems after they are embedded, but on preventing them from being embedded in the first place, and on finding them as early as possible when prevention has not been fully achieved. + +--- + +## Quality and requirements + +The quality of the delivered system is determined, more than by any other single factor, by the quality of the requirements it was built against. + +A requirement that is ambiguous produces an implementation that is based on an assumption. That assumption may be correct. It may not be. When it is not, the consequence is rework — discovered at integration, at acceptance testing, or in the field, depending on when the system is first tested against a customer's actual operational expectations. + +A requirement that is not testable produces an acceptance process that cannot be objective. When the acceptance criterion for a requirement is "the system shall perform adequately," the acceptance decision is a matter of interpretation rather than evidence. Interpretations differ. Disputes follow. Both parties believe they are right and both may be — because the requirement did not specify what adequate means. + +A requirement that has been decomposed correctly — tracing from the customer need through system and subsystem levels to implementable leaf requirements with defined acceptance criteria — produces a programme that can verify its quality objectively at every level. Every component of the delivered system is traceable to a requirement. Every requirement has a test. Every test has a defined pass/fail condition. The quality of the system, relative to its requirements, is not a matter of judgement. It is a matter of evidence. + +This connection between requirements engineering and quality is why Chapter 11 precedes this one. Requirements engineering is not a separate discipline that serves documentation purposes. It is the primary quality investment of the programme — the activity that determines the quality of everything that follows. + +--- + +## Quality and architecture + +Architecture determines quality outcomes as directly as it determines delivery performance. The two are inseparable. + +A well-designed architecture makes quality achievable. Components with clear boundaries and well-defined interfaces can be tested in isolation, with confidence that the isolation faithfully represents the integration context. Changes can be made to one component without requiring system-wide regression testing of everything else. Failures can be contained and diagnosed. The system can be observed in operation in ways that produce the feedback required to maintain quality over time. + +A poorly designed architecture makes quality expensive and ultimately impossible to sustain. Tightly coupled components cannot be tested meaningfully in isolation. Every change carries the risk of cascading effects that are not visible until integration. Failures propagate in ways that are difficult to diagnose. The operational system is a black box that fails in surprising ways and reveals little about why. + +The architectural properties that most directly affect quality outcomes are testability, observability, and failure isolation. + +Testability is the degree to which the system can be exercised in a controlled manner to confirm that it behaves as required. A testable architecture has components with defined inputs and outputs, interfaces that can be controlled and monitored, and a structure that allows the behaviour of individual parts to be verified without requiring the full system to be operational. Untestable architectures are not a testing problem. They are an architectural choice — usually made early in the programme, under schedule pressure, in the name of implementation simplicity — that will impose its cost on every test phase that follows. + +Observability is the degree to which the system reveals its internal state through external signals. A system that can be monitored, that produces meaningful diagnostic output, that exposes its health and its failure modes through observable telemetry is a system that can be operated responsibly and maintained effectively. A system that fails silently, that produces unhelpful error messages, that requires specialist knowledge to diagnose is not just difficult to support. It is a quality liability — because problems that cannot be observed cannot be detected early, and problems that cannot be detected early accumulate into failures that cannot be contained cheaply. + +Failure isolation is the degree to which the impact of a failure in one component is bounded — prevented from propagating to other components in ways that transform a contained problem into a system-wide incident. An architecture that isolates failures produces systems that degrade gracefully under conditions the designers did not fully anticipate. An architecture that does not isolates failures produces systems that fail catastrophically in ways that are expensive to recover from. + +--- + +## Verification and validation + +Verification and validation are frequently conflated but serve different purposes, and the distinction matters for how quality is managed throughout the delivery lifecycle. + +Verification asks: did we build the system right? It confirms that the system has been implemented in accordance with its specifications — that the requirements have been met, the design has been implemented correctly, the tests pass. Verification is an internal discipline. It confirms conformance to specification. + +Validation asks: did we build the right system? It confirms that the specified system, when operated in its real environment, achieves the outcomes the customer needs. Validation is an external discipline. It confirms fitness for purpose. + +A system can pass all its verification tests and still fail validation — because the specification was wrong, or because the customer's real operational environment differs from the conditions under which verification was conducted, or because the customer's actual need differed from the need that was expressed in the requirements. This is the requirements failure mode that Chapter 11 described: an accurate implementation of a wrong specification. + +Both verification and validation must be continuous activities, not end-stage events. Verification occurs throughout the iterative development core — every test that runs, every code review that catches a defect, every interface integration that confirms the contract is being met is an act of continuous verification. Validation begins in the foundations phase — every conversation with the customer that confirms the requirements are capturing the right need, every early prototype that tests an assumption about operational behaviour is an act of early validation. + +The programme that defers all validation to the final phase is not being rigorous. It is accumulating validation risk throughout the programme and discovering it all at once, at the point when it is most expensive to address. The programme that validates continuously — that builds in regular checkpoints at which the developing system is exposed to operational reality and the gap between specification and need is measured — arrives at the validation phase with progressively higher confidence rather than with the anxiety of a system that has never been tested against the environment it must operate in. + +--- + +## Quality in regulated and safety-critical contexts + +In regulated environments — programmes operating under IEC 62304, IEC 61508, ISO 9001, DO-178C, and their sector-specific equivalents — quality is not solely an internal engineering concern. It is an external obligation with specific evidence requirements. + +The evidence obligations of regulated standards are not bureaucratic impositions. They are the explicit articulation of what good quality engineering produces as a natural output: requirements that are clear and traceable, design decisions that are recorded and justified, testing that is systematic and evidence-rich, incidents that are investigated and learned from. A programme that practises quality engineering as described in this chapter will produce the evidence that regulated standards require as a by-product of doing the work well. + +The failure mode in regulated programmes is not the existence of the evidence obligations. It is the treatment of evidence production as a separate activity from engineering delivery. Programmes that manage their engineering process correctly and then assemble compliance evidence retrospectively are doing two jobs: engineering the system and manufacturing the appearance of engineering discipline. The retrospective assembly is more expensive, less reliable, and less credible to auditors than evidence produced continuously as a natural output of the engineering process. + +Firmitas treats compliance evidence as the output of a quality engineering system that is functioning correctly. The traceability spine described in Chapter 14 is the compliance evidence structure. The ADRs are the design justification record. The test results are the verification evidence. The risk register lifecycle is the risk management record. None of these require additional activity beyond what a well-run programme produces anyway. They require the discipline to produce them in a form that is organised, current, and navigable — which is what living documentation and docs-as-code practices, addressed in Chapter 18, provide. + +--- + +## Quality, slack, and the system under pressure + +Quality and slack are connected in the way that Chapter 12 described. A system without slack cannot maintain quality because the activities that maintain quality — proper test coverage, thorough code review, adequate investigation of defects, deliberate refactoring of accumulated debt — are the first casualties of overload. + +These activities are never urgent in the sense that delivering a feature is urgent. They can always be deferred. They accumulate as a backlog of quality debt that inflates invisibly while the programme's task completion rate stays green. When the debt surfaces — in integration failures, in acceptance test failures, in field incidents — the cost is substantially higher than the cost of maintaining quality throughout would have been. + +The connection between quality and slack is not just operational. It is also cultural. A team that has slack for proper code review produces better code and learns more from the review process. A team that reviews code under time pressure produces superficial reviews that miss the structural issues the review should catch. The review exists in both cases. The quality outcome is different. + +The leadership responsibility is explicit: protecting the slack that makes quality sustainable is not a soft preference for comfortable working conditions. It is a structural prerequisite for the quality outcomes the programme requires. An organisation that claims to value quality while systematically eliminating the slack that quality requires is not actually valuing quality. It is valuing the appearance of delivery pace over the reality of delivery quality — and deferring the cost of that trade-off to whichever phase of the programme or product lifecycle is unfortunate enough to discover it. + +--- + +## Quality metrics that do not corrupt behaviour + +Measuring quality is necessary. Measuring it badly destroys the behaviour it is intended to encourage. + +The most common quality measurement failure is the use of metrics that are easy to collect but inversely correlated with the quality they are supposed to measure. Defect counts that incentivise not finding defects. Test coverage percentages that incentivise writing tests that pass rather than tests that find problems. Story point velocity that incentivises closing tickets rather than solving problems. Each of these metrics, when used as performance measures, produces behaviour that optimises the metric at the cost of the quality it was supposed to represent. + +Firmitas favours quality metrics that measure system behaviour rather than development activity. Defect escape rate — the proportion of defects that reach a later phase than the one in which they should have been detected — reveals the effectiveness of quality practices at each phase boundary. Lead time to detect — how long after a defect is introduced it is found — reveals the effectiveness of feedback loops. Rework rate — the proportion of completed work that must be revisited due to quality failures — reveals the upstream quality of requirements, design, and review practices. These metrics are harder to collect than story points. They are more meaningful. They reveal the quality of the system rather than the activity of the teams. + +The governance principle for quality measurement follows from Chapter 8: metrics that senior management uses to evaluate quality should measure system health, not team activity. The governance layer that is reading defect discovery rates and rework percentages is asking the right question — is the quality system working? The governance layer that is reading test case counts and coverage percentages is asking the wrong question — are teams producing the artefacts we associate with quality work? + +--- + +## Quality engineering as a leadership discipline + +Quality is a system property. Systems are designed by leaders. Quality engineering is therefore a leadership discipline, not a technical one. + +This does not mean leaders write test cases or conduct code reviews. It means they shape the conditions under which quality is either possible or impossible. They set the incentives that determine whether teams optimise for apparent velocity or for sustained quality. They protect the slack that makes quality activities possible. They establish the governance mechanisms that treat quality failures as system signals rather than execution problems. They ensure that architectural decisions are made with testability, observability, and failure isolation as first-class concerns. They create the culture in which a test failure is welcomed as early learning rather than penalised as a programme delay. + +The programme board in Chapter 1 that confirmed the delivery date without addressing the unvalidated assumptions in the architecture was making a quality decision. The decision was not framed as a quality decision — it was framed as a schedule decision. But schedule decisions that require quality compromises are quality decisions. The leadership layer that makes schedule decisions without acknowledging the quality consequences they carry is not governing quality. It is generating the conditions that will produce quality failures later, at a cost that will be attributed to execution rather than to the governance decision that caused it. + +Quality engineering, in the Firmitas sense, begins at the moment the programme board first reviews the programme's plan and risks, and continues throughout the programme's life. It ends when the last field incident has been resolved and the system has been retired. That scope is not a burden. It is what responsibility for outcomes actually means. diff --git a/content/firmitas/part-2-the-framework/16-governance-guardrails-and-decision-authority.md b/content/firmitas/part-2-the-framework/16-governance-guardrails-and-decision-authority.md new file mode 100644 index 0000000..11144bc --- /dev/null +++ b/content/firmitas/part-2-the-framework/16-governance-guardrails-and-decision-authority.md @@ -0,0 +1,159 @@ +--- +title: "Chapter 16: Governance, Guardrails, and Decision Authority" +--- + +# Chapter 16: Governance, Guardrails, and Decision Authority + +Governance is one of the most consistently misunderstood concepts in engineering programme management. In most organisations it means one of two things: either the set of approval processes that work must pass through before proceeding, or the committee that reviews programme status and asks questions. Both of these are expressions of governance. Neither is governance itself. + +Governance is the system by which decision authority is exercised — the structure that determines who can decide what, under what constraints, with what evidence, and with what accountability for the consequences. It is not a set of processes. It is the infrastructure of decision-making. When governance is well designed, decisions are made at the right level with the right information and produce outcomes the system can be accountable for. When governance is poorly designed, decisions are made at the wrong level, without adequate information, by people who will not bear the consequences — which is precisely the governance failure that Chapter 1 described. + +The governance model in Firmitas is built around three principles that this chapter addresses in sequence. Guardrails rather than gates, wherever possible. Evidence-based decisions rather than opinion-based ones. And management by exception — the principle that senior governance acts on escalated risks and issues rather than on task-level status, described in Chapter 10 and elevated here to a governance structural principle. + +--- + +## What governance is for + +Governance exists to answer four questions consistently, at every level of the programme. + +Who can make which decisions? Decision authority must be explicitly allocated. Ambiguous authority produces one of two failure modes: either every decision escalates to a level that does not have the operational knowledge to make it well, or decisions are made at whatever level is most convenient without clarity about whether that level has the authority. Both failure modes are visible in Chapter 1 — the programme board that had authority but did not exercise it on the decisions the risk register required, and the firmware team that made specification decisions it should not have had to make because the governance layer had not resolved the ambiguity. + +Under what constraints? Authority is always bounded. Teams have authority over their own delivery decisions within the architectural guardrails and quality standards the programme has established. The programme board has authority over programme-level decisions within the commercial commitments and portfolio constraints that the senior governance layer has established. Senior governance has authority within the organisation's strategic, regulatory, and commercial boundaries. The constraints are not bureaucratic limitations on autonomy. They are the boundaries within which autonomous decision-making is legitimate — the guardrails that allow teams to move quickly without requiring approval for every decision. + +With what evidence? Governance decisions must be evidence-based. A gate review approved without evidence that the criteria have been met is not governance. It is the appearance of governance, which is worse than no governance at all, because it creates a false assurance that decisions have been made on sound foundations when they have not. Evidence requirements should be proportionate to the risk and irreversibility of the decision — not uniform across all decisions regardless of their significance. + +With what accountability? Every significant decision must have a named owner who is accountable for the consequences. Decisions made by committees without named owners diffuse accountability in ways that make learning from poor decisions impossible. When the firmware specification decision in Chapter 1 was effectively made by inaction — by the programme board's failure to resolve the ambiguity — there was no named owner of that inaction and therefore no named owner of its consequences. The accountability appeared later, attributed to the firmware team who had proceeded as best they could in the absence of a resolved specification. + +--- + +## Guardrails, not gates + +The fundamental governance design choice in Firmitas is the preference for guardrails over gates wherever the programme's risk profile permits. + +A gate stops work until permission is granted. It is a synchronisation point that requires the programme to pause, assemble evidence, present it to a governance body, and receive approval before proceeding. Gates are appropriate where the risk of proceeding without review is high, where the decision is irreversible, or where external regulatory or contractual requirements mandate them. They are expensive — they introduce latency into the delivery system, create administrative overhead, and concentrate decision-making at a governance level that may not have adequate operational knowledge. + +A guardrail defines where teams may safely operate without stopping. It establishes the constraints within which autonomous decision-making is legitimate, and makes the boundaries of that legitimacy clear. Teams do not need approval to operate within the guardrails. When they identify a situation that would require stepping outside the guardrails, they escalate — not because the governance system has stopped them, but because the nature of the decision requires authority or information that they do not have. + +The practical difference is significant. A gate-heavy governance system creates a programme culture in which every decision feels like it requires approval. Teams become permission-seeking rather than decision-making. Lead times increase by the latency of approval cycles. The governance layer is overwhelmed with decisions that should have been made at a lower level. And the decisions that genuinely require governance attention — the escalated risks and issues that only the governance layer can resolve — compete for attention with the routine approvals that should not have needed governance involvement at all. + +A guardrail-based governance system creates a programme culture in which teams operate with confidence within their boundaries, escalate decisions that genuinely require higher authority, and trust that the governance layer will act on those escalations rather than adding them to a review backlog. The governance layer focuses on the decisions that only it can make — and makes them quickly enough to be useful. + +--- + +## Designing guardrails + +A guardrail is only useful if it is explicit, stable, and understood by the people operating within it. Implicit guardrails — the unspoken norms about what requires approval — produce the same failure mode as ambiguous authority: either teams escalate everything because they are uncertain what they can decide, or they decide everything themselves because they are uncertain what requires escalation. + +Guardrails in Firmitas are defined across several dimensions. + +**Architectural guardrails** establish the technical boundaries within which development decisions are autonomous. The architectural principles, patterns, and interface definitions established during the structured foundations phase are the primary architectural guardrails. A development decision that operates within those principles and within the defined interfaces is a legitimate autonomous decision. A decision that would violate a principle, change a defined interface, or introduce a new external dependency requires an ADR and review — not because the team cannot make the decision, but because the decision has system-level implications that the team's local view cannot fully assess. + +**Quality guardrails** establish the minimum quality standards that every piece of work must meet before it can be considered complete. Coding standards, test coverage thresholds, review requirements, definition of done criteria at the component, integration, and system levels. Teams operate within these standards autonomously. Work that does not meet the standards is not complete, regardless of how close to the delivery date the discovery occurs. Quality guardrails that are negotiable under schedule pressure are not guardrails. They are suggestions — and suggestions do not produce quality. + +**Commercial and scope guardrails** establish the boundaries of the programme's committed scope and the commercial constraints within which trade-off decisions can be made autonomously. A team that can descope a feature within the programme's agreed change management process without escalation is operating within a commercial guardrail. A team that discovers a scope change that would affect the contractual deliverable or the commercial commitment requires escalation — because the decision has consequences beyond the team's authority to accept. + +**Regulatory guardrails** in regulated programmes establish the boundaries of what can be done without formal review against the applicable standards. Some decisions in regulated environments have no autonomous path — they require documented evidence, independent review, or formal sign-off. These are not guardrails in the sense of autonomous operation within boundaries. They are mandatory gates. The governance model must accommodate both. + +--- + +## Mandatory gates in regulated environments + +The guardrails-not-gates preference has a necessary exception: externally mandated gates that cannot be dissolved into continuous flow without violating the regulatory framework the programme operates within. + +IEC 62304, IEC 61508, DO-178C, and their sector-specific equivalents all contain review, approval, and sign-off requirements that are not discretionary. The safety case cannot be approved by the team that built the system. The verification evidence cannot be reviewed only by the people who produced it. The configuration management baseline cannot be established without formal authorisation. These requirements exist for good reasons — the independence they mandate is a genuine quality assurance mechanism, not a bureaucratic imposition. + +Firmitas accommodates mandatory gates without allowing them to dominate programme governance. The key is the distinction between the evidence required by the gate and the activity required to produce it. + +The evidence for a mandatory regulatory gate should be produced continuously throughout the programme — as the natural output of a quality engineering system that is functioning correctly — not assembled retrospectively in the weeks before the gate. A gate review that requires six weeks of evidence assembly has not been governing its evidence continuously. It has been deferring the evidence work until it became unavoidable, and paying the cost of that deferral as a compressed, stressful scramble that produces evidence of variable quality. + +A programme that produces its evidence continuously arrives at a mandatory gate with the evidence already in place. The gate review becomes a verification that the continuously produced evidence is complete, current, and coherent — not a discovery exercise in which the evidence is assembled and the gaps are found for the first time. The gate is faster, less stressful, and more credible to the regulatory body reviewing it. + +The governance principle: mandatory gates are satisfied by the quality of the engineering system, not by the effort of evidence assembly. Design the engineering system to produce compliance evidence continuously. The gate review then confirms that the system has been working correctly. + +--- + +## Evidence-based governance + +Every significant governance decision should be based on evidence rather than assertion, opinion, or optimism. + +This is a principle that sounds obvious and is systematically violated. Programme boards that approve gate reviews on the basis of well-prepared slide decks rather than engineering evidence. Senior governance layers that accept status reports at face value without the means to verify whether the status reflects operational reality. Risk registers that are reviewed as documents rather than as instruments — acknowledged for their content without generating the decisions their content requires. + +Evidence-based governance in the Firmitas sense means three things. + +The evidence presented at governance reviews is specific, verifiable, and current. Not "testing is on track" — but "23 of 31 acceptance test cases have been executed with pass results, 6 have been executed with documented failures currently under investigation, and 2 have not yet been executed. The 6 failures are detailed in the attached test failure report with their current resolution status." The first statement is an assertion. The second is evidence. The governance layer that accepts assertions without the specific evidence behind them is not governing the programme. It is accepting a narrative. + +The evidence requirements are proportionate to the decision risk. A low-risk, reversible decision requires less evidence than a high-risk, irreversible one. The gate that represents the commitment to manufacture — the point at which hardware tooling is ordered and the cost of design change increases by an order of magnitude — requires substantially more rigorous evidence of design completeness than a gate that represents the start of software prototyping. Calibrating evidence requirements to decision risk is not lowering standards. It is allocating governance attention appropriately. + +The governance layer has the capability to evaluate the evidence it receives. Evidence presented to people without the knowledge to evaluate it is not evidence-based governance. It is evidence theatre — the performance of rigour without the substance of it. Where the governance layer lacks the technical knowledge to evaluate the engineering evidence directly, the governance model must provide for independent technical review by people who do have that knowledge. The programme board that cannot evaluate whether a test failure report represents a managed risk or a programme-threatening defect needs access to someone who can — and that access must be structural, not occasional. + +--- + +## Scaling governance without centralisation + +As programmes scale — across multiple teams, multiple disciplines, multiple delivery streams — the instinct is to centralise governance. A central architecture board. A central change control board. A central quality authority. These mechanisms are introduced with good intent: consistency, coherence, risk management at scale. + +The consistent failure mode of centralised governance at scale is that it becomes a bottleneck. Decisions that should be made in days take weeks because the central board meets infrequently, has full agendas, and has insufficient operational knowledge to evaluate the specific decisions that are presented to it. Teams learn that escalating decisions to the central board costs time without adding value. They begin making decisions locally that should have governance visibility — not because they are circumventing governance, but because the governance system has made itself too slow and too distant to be useful. + +Firmitas scales governance by replicating the governance model rather than centralising the governance function. Each level of the programme has its own decision authority, its own guardrails, and its own escalation path to the level above it. The architecture at team level is governed by the architectural guardrails established during the foundations phase. The programme-level architecture is governed by the programme architecture board. The portfolio-level architecture is governed by the enterprise architecture function. Each level has authority over decisions within its scope and escalates decisions that exceed that scope. + +The consistency that centralised governance is supposed to provide comes not from a single governing body but from shared principles, shared artefacts, and shared understanding. Guardrails that are established at the programme level and communicated clearly to all teams produce more consistent decisions than a central board that reviews individual decisions inconsistently because it lacks the context to evaluate each one properly. Architectural Decision Records that are accessible to all teams produce more coherent architectural evolution than a central architecture board that is the only body allowed to make architectural decisions. + +The governance test at scale is not: does every significant decision pass through a central body? It is: are decisions being made at the right level, with the right information, within guardrails that produce coherent outcomes across the programme? + +--- + +## Management by exception as a governance structure + +Chapter 10 introduced management by exception in the context of the risk and issues register — the principle that senior programme governance acts on escalated items rather than on task-level status. The principle is restated here as a structural governance principle because its implications extend beyond the risk register to the entire governance model. + +Management by exception is not a management style preference. It is the structural consequence of two things working together: delivery knowledge residing closest to the work, as Principle 4 states, and governance focusing on decisions that require authority or information beyond the level at which delivery is managed. + +When delivery knowledge is at the team level, team-level decisions should be made at the team level. When programme-level authority is required — for escalated risks, for cross-boundary decisions, for commercial or contractual commitments — those decisions are made at the programme level. When portfolio-level authority is required — for strategic trade-offs, for major investment decisions, for decisions that affect multiple programmes — those decisions are made at the portfolio level. + +The governance layer at each level acts on the things only that level can resolve. Everything else is owned by the level below it. This is not the absence of governance. It is governance operating at the correct level of abstraction — focused on the decisions that require its specific authority, not occupied with the operational management of delivery that belongs with the people who have the knowledge to manage it. + +The programme review that is structured around escalated risks and issues, that generates decisions on each of them, and that trusts the delivery team to manage everything within their authority is a programme review operating on management-by-exception principles. The programme review that begins with a RAG dashboard and a task completion percentage is a programme review that has inverted the principle — substituting comfort for governance. + +--- + +## Governance theatre and how to recognise it + +Governance theatre is the performance of governance without its substance. It is the gate review that approves the programme because the programme presented well, not because the evidence supports approval. It is the risk register that is reviewed and acknowledged without generating decisions. It is the architecture board that approves designs without the knowledge to evaluate their quality. It is the programme board that notes concerns and moves on. + +Governance theatre is not a moral failure. It is a structural one. It occurs when the governance mechanisms have been designed around the form of governance — the meetings, the documents, the sign-offs — without the substance — the informed decisions, the accountable owners, the evidence-based evaluations. The mechanisms exist. The function does not. + +The distinguishing marks of governance theatre are recognisable. Gate reviews that always result in approval regardless of the evidence state. Risk registers that grow without any risk ever being retired through decision and action. Programme reviews that end without any decision having been made. Architecture boards that approve designs that later require significant rework. Audit preparations that require weeks of evidence assembly rather than days of evidence organisation. + +Each of these marks is an indicator that the governance mechanism has been separated from the function it is supposed to serve. The mechanism exists to satisfy the requirement to have governance. The function — the informed, accountable, evidence-based decision-making that governance is supposed to produce — has been lost. + +The corrective is not to add more governance. It is to reconnect the mechanisms to the function. Gate reviews that require the evidence to be reviewed before approval rather than after. Risk register reviews that end with named decision owners. Programme reviews that have risk and issues as the primary agenda item. Architecture decisions that are made by people with the knowledge to evaluate them. Evidence that is continuously produced rather than retrospectively assembled. + +--- + +## Governance in a regulated environment — the mapping + +For programmes operating under formal standards, the Firmitas governance model produces the evidence and the process that those standards require. The mapping is direct. + +ISO 9001 requires leadership accountability for quality, defined processes, evidence-based decision-making, and continuous improvement. The Firmitas governance model provides all four: leadership accountability through Principle 5, defined guardrails through the structured foundations phase, evidence-based decisions through the governance principles in this chapter, and continuous improvement through the learning systems described in Chapter 5. + +IEC 62304 requires software development planning, requirements management, architectural design, implementation, verification, and problem resolution. The Firmitas delivery model provides the lifecycle framework. The governance model provides the decision points, the evidence requirements, and the accountability structure that the standard requires. + +IEC 61508 and DO-178C require independence of verification, documented design rationale, configuration management, and formal review at defined lifecycle stages. The ADR practice provides design rationale. The continuous traceability spine provides the evidence base. The mandatory gates accommodate the formal review requirements. The guardrails-based governance provides the operational framework within which the formal requirements are satisfied without displacing the delivery discipline. + +The compliance arises naturally from a programme that is being run correctly. The evidence is produced continuously as a natural output of the engineering system. The governance structure provides the decision points and the accountability that the standards require. The programme that treats compliance as an engineering output rather than a documentation exercise produces better compliance evidence, with less effort, and with more credibility to the regulatory bodies evaluating it. + +--- + +## What good governance looks like + +Good governance is nearly invisible. When governance is working well, decisions are made quickly and well at the appropriate level, the people making decisions have the information they need, the people affected by decisions understand why they were made, and the accountability for outcomes is clear and accepted. + +Good governance does not feel like governance to the teams operating within it. It feels like having clear boundaries, adequate authority, and reliable escalation paths when decisions exceed those boundaries. It feels like the absence of the friction that poor governance produces — the approval queues, the uninformed overrides, the governance theatre that consumes time without producing decisions. + +The test of governance quality is not the sophistication of the governance mechanisms. It is the quality of the decisions the governance system produces. Does the programme make the right decisions, at the right time, with the right information? Are the people who make decisions accountable for their consequences? Does the governance structure make honest information flow upward — so that the programme board reviewing the risk register at week three is able to see what the engineering team sees and act on it? + +If the answer is yes, the governance is working. The mechanisms that produce those outcomes — whether they are simple or complex, whether they look like governance or not — are the right mechanisms for this programme. + +If the answer is no, the governance is not working. And the response is not to add more governance. It is to diagnose what the current governance is failing to produce, and to redesign the mechanisms that connect decision authority to delivery knowledge, risk information to governance action, and evidence to accountability. + +That is what governance design means. And that, as Chapter 7 established, is a leadership responsibility. diff --git a/content/firmitas/part-2-the-framework/17-teams-cognitive-load-and-organisation-for-flow.md b/content/firmitas/part-2-the-framework/17-teams-cognitive-load-and-organisation-for-flow.md new file mode 100644 index 0000000..01a4c25 --- /dev/null +++ b/content/firmitas/part-2-the-framework/17-teams-cognitive-load-and-organisation-for-flow.md @@ -0,0 +1,143 @@ +--- +title: "Chapter 17: Teams, Cognitive Load, and Organisation for Flow" +--- + +# Chapter 17: Teams, Cognitive Load, and Organisation for Flow + +The delivery model in Chapter 14, the governance model in Chapter 16, the architectural principles in Chapter 13 — all of these assume something that is rarely examined explicitly: that the teams responsible for delivery are organised in a way that makes delivery possible. Not just capable of delivery, but structured so that the work can flow, the decisions can be made at the right level, and the cognitive load on the people doing the work is manageable rather than overwhelming. + +In most programmes, team organisation is an afterthought. Teams are assembled from available people. Boundaries between teams follow organisational charts rather than system architecture. Responsibilities are allocated based on historical function rather than delivery flow. The team structure is inherited, not designed. + +The consequences of undesigned team structure accumulate in predictable ways. Teams that cannot make changes independently because their architecture requires coordination with other teams slow each other down. Teams with responsibilities that span too broad a scope accumulate cognitive load that degrades the quality of their decisions and increases their error rates. Teams organised around functions rather than outcomes produce handoffs that introduce delay, information loss, and accountability gaps at every boundary. + +This chapter addresses team design as a delivery discipline — the structural decisions about how teams are organised, what they own, how they interact, and what cognitive constraints apply — that determine whether the delivery model produces flow or friction. + +--- + +## Teams as socio-technical systems + +A team is not a collection of people with skills. It is a socio-technical system — a combination of people, processes, tools, responsibilities, and relationships that together produce outcomes. The behaviour of a team is not determined by the capability of its individual members. It emerges from how those members interact with each other, with the technical systems they are responsible for, and with the broader organisational context in which they operate. + +This framing has a specific implication for team design: changing the people in a team without changing the system they operate within produces the same outcomes. The new team members adapt to the same incentives, the same constraints, the same information flows, and the same coordination patterns that shaped the previous team's behaviour. The system is the determinant. The people are responding to it. + +The converse is also true. Designing the team's system well — the boundaries, the ownership, the interaction modes, the tools, the governance — produces good outcomes even from teams that are not exceptionally talented. Most of the variance in team performance between organisations is explained by team system design rather than by individual capability. The high-performing team is high-performing because its system makes good performance possible. The struggling team is struggling because its system makes good performance difficult. + +Conway's Law, observed by Mel Conway in 1967, makes this argument at the system level: organisations design systems that mirror their communication structures. The architecture of the software a team produces tends to reflect the organisational boundaries within which it was produced. This is not a tendency to be fought. It is a constraint to be used. If the desired architecture requires clear boundaries between components, design teams with clear boundaries that match those components. If the desired architecture requires loose coupling between subsystems, design teams whose interaction modes are also loosely coupled. + +Conway's Law is not a warning. It is a design tool. The team structure is the architecture. Design them together. + +--- + +## Cognitive load as a first-class design constraint + +Cognitive load — the total amount of mental effort required to understand the system a team is working in, reason about changes safely, and make decisions with confidence — is one of the most consistently underestimated constraints in engineering team design. + +When cognitive load exceeds what a team can sustainably carry, the consequences are predictable. Error rates increase because the mental overhead of managing too many concerns leaves insufficient attention for each individual one. Decisions degrade because the cognitive resources required for careful analysis are occupied by context-switching and coordination overhead. Learning stops because the capacity required for reflection and experimentation is consumed by the demands of keeping existing work moving. Burnout follows because the cost of permanent cognitive overload is physiological, not just operational. + +Cognitive load in engineering systems is not monolithic. Three types interact to determine the total load a team carries. + +**Intrinsic load** is the irreducible complexity of the problem domain. Real-time constraints in embedded systems. Regulatory interpretation in a regulated environment. Distributed system failure modes. Safety-critical reasoning. Intrinsic load cannot be removed — it is the inherent difficulty of the problem the team exists to solve. What can be done is to ensure that the team's scope is bounded to intrinsic loads that are coherent and manageable together, rather than assembling an arbitrary collection of complex concerns into a single team's responsibility. + +**Extraneous load** is the cognitive overhead imposed by the organisation rather than by the problem. Unclear responsibilities that require negotiation before work can begin. Excessive handoffs that require context reconstruction at each boundary. Inconsistent tooling that requires different mental models for different parts of the work. Contradictory governance signals that require effort to interpret before decisions can be made. Extraneous load is almost always removable. It is the most damaging form of cognitive load because it consumes capacity without producing value — every unit of cognitive resource occupied by extraneous load is a unit unavailable for the intrinsic complexity the team exists to address. + +**Germane load** is the cognitive investment required for learning and mastery — the productive challenge of developing new understanding and capability. Germane load is healthy, but only when the team has the cognitive capacity to engage with it. A team operating at the limit of its intrinsic and extraneous load cannot learn. It can only cope. Sustainable skill development requires that the total load leaves capacity for the germane — which is why Chapter 12's argument about slack applies directly here. Cognitive slack is the capacity for learning. + +--- + +## Ownership as the primary load-reduction mechanism + +The most effective mechanism for reducing cognitive load is clear, end-to-end ownership — the allocation of a coherent domain of responsibility to a team that includes both the authority to make decisions within that domain and the accountability for the outcomes those decisions produce. + +Ownership reduces cognitive load in a specific way. When a team owns a domain end-to-end, the questions they must answer to make a decision within that domain are answerable from within the team. They do not need to coordinate with other teams to understand the implications of a change within their boundary. They do not need to escalate routine decisions to a governance level that lacks the context to evaluate them. They do not need to reconstruct the reasoning of a previous team to understand why a constraint exists. The domain is theirs. The knowledge required to manage it is present within the team. + +Partial ownership is the inverse. When a team is responsible for implementing a component but not for its design, or for delivering a feature but not for its quality, or for building a system but not for operating it, the team must carry the cognitive overhead of the missing responsibilities without having the authority to resolve them. They must anticipate decisions they do not control. They must defend work they cannot fully reason about. They must negotiate responsibility after problems arise. + +The hidden cost of partial ownership is one of the most consistently underestimated costs in engineering programme delivery. Every handoff between partially-owning teams introduces a seam in the cognitive continuity of the work — a point at which knowledge is reconstructed imperfectly, assumptions are restated rather than inherited, and the information loss of the transition accumulates as misalignment that surfaces later as integration failures or quality gaps. + +Firmitas insists on end-to-end ownership wherever possible — not because it is organisationally convenient, but because it is the structural condition that makes cognitive load manageable and decision-making local. + +--- + +## Team topologies + +The Team Topologies model, developed by Skelton and Pais, provides a practical vocabulary for designing team structures that optimise for flow. Firmitas uses this vocabulary because it is the most useful currently available — the distinctions it draws between team types and interaction modes are directly relevant to the delivery challenges complex programmes face. + +Four team types serve different functions in a well-designed delivery organisation. + +**Stream-aligned teams** own a stream of value from end to end — a product, a service, or a capability that delivers directly to a customer or operator. Stream-aligned teams are the primary delivery teams. They own their domain completely: requirements, design, implementation, testing, deployment, and operational support. They minimise dependencies on other teams. Their performance is measured by the outcomes they produce for the customers or operators they serve, not by the activities they complete. + +In the programme of Chapter 1, the firmware team and the software team were not stream-aligned in this sense. Neither owned the full capability it was contributing to. The firmware team owned the firmware, not the integrated product that the firmware enabled. The software team owned the software, not the customer outcome the software was delivering. The absence of end-to-end ownership was one of the structural conditions that allowed the integration failures to accumulate without any team feeling the full cost of them. + +**Platform teams** provide capabilities that multiple stream-aligned teams need, in a form that allows those teams to use them without becoming dependent on the platform team for routine operations. A platform team's output is an internal product — a toolchain, a development environment, a test framework, a shared infrastructure service — that reduces the cognitive load and the operational overhead of the teams that use it. The platform team is accountable for the quality and usability of its platform, not for the outcomes of the stream-aligned teams that use it. + +The critical property of a platform team is self-service. A platform that requires stream-aligned teams to request help from the platform team for routine operations has not reduced cognitive load. It has created a dependency. The platform team becomes a bottleneck. The stream-aligned teams accumulate wait time. The flow that the platform was supposed to enable is degraded by the coordination overhead it has introduced. + +**Enabling teams** work with stream-aligned teams for a bounded period to develop capabilities that the stream-aligned team can then own and operate independently. They are not a permanent support function. They are a temporary accelerant — bringing expertise in testing practices, security, architectural patterns, or regulatory compliance to a stream-aligned team that needs to develop that capability, with the explicit goal of making themselves unnecessary. + +**Complicated-subsystem teams** own components whose internal complexity is high enough that it requires specialist knowledge that stream-aligned teams cannot be expected to carry. A cryptographic library. A real-time operating system kernel. A signal processing pipeline. These components are valuable to multiple stream-aligned teams. Their complexity is such that building them well requires a depth of expertise that it is not practical to replicate in each stream-aligned team. The complicated-subsystem team owns the component, defines its interface, and is accountable for its quality and fitness for purpose. + +--- + +## Interaction modes + +The interaction mode between two teams — how they work together and what each expects from the relationship — is as important as the team structure itself. Three interaction modes produce different cognitive loads and different flow properties. + +**Collaboration** is the mode in which two teams work closely together, with high bandwidth communication, to discover the right solution in a space of significant uncertainty. Collaboration is appropriate for short-duration, exploratory work where the solution cannot be defined in advance. It is expensive in terms of cognitive load — both teams must carry the full context of the shared work — and produces integration risk if sustained beyond the period where close coordination is genuinely required. When the exploration is complete and the interface is understood, the collaboration should transition to a defined interface mode. + +**X-as-a-Service** is the mode in which one team provides a capability to another through a well-defined, stable interface. The consuming team uses the capability without needing to understand its internal workings. The providing team owns the capability without needing to understand the consuming team's use of it. This is the lowest-cognitive-load interaction mode — each team operates independently within its own domain, coordinating only through the defined interface. It is only viable when the interface is clear enough to be relied upon without continuous coordination. + +**Facilitating** is the mode in which an enabling team works alongside a stream-aligned team to help it develop a capability, without owning the outcome. The facilitating team brings expertise and guidance. The stream-aligned team owns the work and develops the capability. The interaction is bounded in duration — it should end when the stream-aligned team has the capability it needs and the enabling team can disengage. + +The choice of interaction mode should match the maturity of the interface between teams and the degree of uncertainty in the work they are doing together. New interfaces require collaboration to define. Mature interfaces should operate in x-as-a-service mode. Teams that need to develop capability benefit from facilitating interaction. + +A common failure mode is sustaining collaboration past the point where it is productive — treating the exploratory mode as a permanent arrangement rather than a transition toward a defined interface. Sustained collaboration between teams produces high coordination overhead, ambiguous ownership, and cognitive load that grows with the number of teams involved. It should be time-bounded and replaced by a more independent interaction mode as soon as the interface is understood well enough to define. + +--- + +## Boundaries and cognitive load + +The boundaries between teams are where cognitive load is either contained or leaked. + +Good boundaries align with the natural seams in the architecture — the places where components are already loosely coupled, where interfaces are already well-defined, where the knowledge required to manage one side of the boundary is largely independent of the knowledge required to manage the other. Teams with boundaries aligned to architectural seams can reason about their domain locally. Changes within their boundary do not require understanding of other teams' domains. Integration with other teams occurs through defined interfaces that the team can rely on without deep knowledge of what is behind them. + +Poor boundaries slice the system horizontally by function — all the database administrators in one team, all the front-end developers in another, all the hardware engineers in a third — regardless of which systems those functions serve. Horizontal boundaries force teams to coordinate across every feature change, because every feature requires contributions from multiple functional teams. They create handoffs at every boundary, information loss at every handoff, and accountability gaps that nobody owns. They maximise coordination overhead and minimise the cognitive continuity that makes sustained delivery possible. + +The restructuring required to move from functional boundaries to system-aligned boundaries is organisationally difficult. It disrupts existing management structures, challenge established career paths, and requires people to develop broader skills and accept narrower managerial domains. These are real costs. They are substantially smaller than the ongoing cost of delivery systems that are slowed by the coordination overhead that functional boundaries produce. + +--- + +## Organising for flow + +The principle that connects team design to the delivery model is straightforward: team boundaries should be designed so that the work the delivery model requires can flow without unnecessary coordination overhead. + +The structured foundations phase requires cross-discipline collaboration on architecture, requirements, and interface definition. The teams involved in this phase need interaction modes that support close collaboration — because the interfaces are being defined, not yet relied upon. As the foundations phase produces defined interfaces and architectural guardrails, the interaction mode should transition toward x-as-a-service — because the interfaces are now clear enough to be relied upon independently. + +The iterative development core requires that teams can deliver their iterations without depending on coordination with other teams for every change. This requires that team boundaries align with the architectural boundaries established during the foundations phase — so that a team's iteration scope is within its own boundary, testable against the defined interfaces on each side, and integrated with other teams' work at the synchronisation points rather than through constant coordination. + +The validation phase requires that the evidence produced by each team's development activities is coherent and traceable across team boundaries. This requires that the traceability spine described in Chapter 14 is maintained throughout the programme — not as a documentation exercise, but as the natural output of teams working within defined boundaries with clear ownership of their contributions to the programme's evidence base. + +--- + +## Scaling — boundaries rather than hierarchies + +As programmes scale — adding teams, increasing the number of disciplines, managing larger portfolios — the temptation is to add governance hierarchy. More managers, more coordination roles, more approval layers. This is the correct response to the problem of maintaining visibility over a larger organisation. It is the incorrect response to the problem of maintaining flow. + +Firmitas scales delivery by designing boundaries rather than adding hierarchies. Well-designed boundaries allow teams to operate independently within their scope without requiring escalation to coordination roles for routine decisions. The coordination overhead that hierarchy introduces — the meetings, the status reports, the approval processes — is replaced by the operational independence that clear boundaries enable. + +The question that drives scaling decisions is not "how do we coordinate all these teams?" It is "where do we place boundaries so that coordination requirements are minimised?" When boundaries are placed at the architectural seams, at the points of natural loose coupling, coordination requirements within each team are high and coordination requirements between teams are low. The programme scales not by adding coordination infrastructure but by replicating well-designed team systems across a larger architecture. + +This approach has limits. Some decisions genuinely require cross-boundary coordination — architectural changes that affect multiple teams, commercial decisions that affect the portfolio, risk escalations that require portfolio-level authority. These decisions belong at the governance level described in Chapter 16. The principle is not that cross-boundary coordination never occurs. It is that cross-boundary coordination is reserved for the decisions that genuinely require it, rather than imposed routinely on the decisions that teams can and should make independently. + +--- + +## The human dimension + +Teams are not engineering components. They are groups of people whose performance depends on conditions that extend beyond the technical and structural. Chapter 11 addressed psychological safety and motivation as system properties — and those properties apply directly to team design. + +A team whose boundaries are clear enough that members understand what they own and what they do not produces lower anxiety and better decisions than a team whose boundaries are ambiguous. A team whose cognitive load is manageable produces higher quality work and more sustainable performance than one that is permanently overloaded. A team that is trusted to manage its own delivery within defined boundaries develops the ownership and commitment that produces the outcomes the programme needs. + +The team design decisions described in this chapter — boundaries, ownership, interaction modes, cognitive load management — are not only technical decisions. They are conditions for human performance. Getting them right produces teams that are capable of delivering well. Getting them wrong produces teams that are capable but prevented from performing — by the structure that surrounds them rather than by any deficit in themselves. + +This is Principle 12 applied to team design: your people are your most significant investment. The structural decisions that determine whether their capability can be applied effectively are leadership decisions. They deserve the same deliberate attention as the architectural decisions that determine whether the system can evolve, and the governance decisions that determine whether the programme can be managed honestly. + +Design the teams. Do not inherit them. diff --git a/content/firmitas/part-2-the-framework/18-communication-information-flow-and-documentation.md b/content/firmitas/part-2-the-framework/18-communication-information-flow-and-documentation.md new file mode 100644 index 0000000..71e31ab --- /dev/null +++ b/content/firmitas/part-2-the-framework/18-communication-information-flow-and-documentation.md @@ -0,0 +1,135 @@ +--- +title: "Chapter 18: Communication, Information Flow, and Documentation" +--- + +# Chapter 18: Communication, Information Flow, and Documentation + +The failure in Chapter 1 was, at its most fundamental level, a communication failure. Not in the interpersonal sense — the engineers communicated clearly. They produced a risk register that named three specific, actionable risks. They presented an honest three-point estimate. They described the conditions that the programme depended on. The communication was precise, timely, and accurate. + +The failure was that the information did not move. It was produced, received, acknowledged, and stopped. The risk register existed in a document. The programme board had read the document. The information that the document contained — the specific decisions required, the windows within which those decisions needed to be made, the consequences of not making them — did not travel from the document into the governance process in a form that generated action. + +This is the communication failure that engineering organisations most commonly experience: not the failure to produce information, but the failure to design systems in which information flows to the people who need to act on it, in a form they can act on, at a time when action is still possible. + +Communication in Firmitas is a designed system, not an emergent by-product of good intentions or individual effort. How information is structured, how it is stored, how it travels between teams and governance layers, what format makes it actionable, and what cultural conditions determine whether it is received honestly — all of these are design decisions. They are as consequential as architectural decisions and as much a leadership responsibility as governance design. + +--- + +## Communication is a system, not a skill + +The most persistent misattribution in engineering programme communication is the treatment of communication failures as individual skill failures. When information does not reach the people who need it, the diagnosis is frequently "poor communication" — implying that someone did not communicate clearly enough, frequently enough, or to the right person. + +This framing locates the problem in individuals and produces individual-level solutions: communication training, stakeholder management guidance, presentation skills workshops. It leaves the system that determined what information was produced, how it was structured, who it was sent to, and what happened when it arrived entirely unchanged. + +People communicate as the system allows them to. If information is fragmented across disconnected tools, people share fragments. If the governance culture treats bad news as a performance failure, people share good news. If the only formal communication channel is the programme status report, people produce status reports that reflect what the programme board wants to see rather than what the programme needs them to know. If the risk register is treated as a filing obligation rather than a decision instrument, people file risks rather than surface them as urgent decisions. + +The system determines what information is produced, how honestly it is presented, and whether it reaches the people who need to act on it. Changing communication outcomes requires changing the system, not coaching individuals to communicate differently within a system designed to prevent honest communication. + +--- + +## Information that must flow + +Not all information is equally important to programme outcomes. The information failures that most consistently damage programmes are failures in a specific set of information types — the ones that carry risk, drive decisions, and connect delivery reality to governance action. + +**Intent and purpose.** Teams must understand why the work they are doing exists — what customer problem it solves, what business outcome it contributes to, what constraints define its boundaries. Intent is not a motivational nicety. It is the reference point that allows teams to make coherent decisions under uncertainty without escalating every question that the specification does not explicitly address. A team that understands intent can make good local decisions. A team that only understands specification cannot — because every situation the specification does not cover becomes either a blocker or an assumption. + +Intent must be persistent, discoverable, and stable. Not communicated once in a project kick-off presentation that is forgotten before the second sprint. Embedded in the programme's living documentation in a form that new team members can find, existing team members can reference when priorities shift, and the programme board can use as a reference when commercial pressures create conflicting demands. + +**Decisions and their rationale.** What was decided matters. Why it was decided matters more. The decision without its rationale is a constraint that cannot be evaluated — it can only be followed or violated. The decision with its rationale is a constraint that can be maintained when it is still valid and revised when it is not. + +Architecture Decision Records, as described in Chapter 13, are the primary mechanism for decisions and their rationale in the technical domain. The same principle applies to programme-level decisions: the decision to accept a commercial risk, the decision to descope a feature, the decision to proceed through a gate despite an open issue. Every significant decision leaves a trace that includes not just what was decided but why, what alternatives were considered, and under what conditions the decision should be revisited. + +**Risk and uncertainty.** Risk does not become dangerous when it exists. It becomes dangerous when it is hidden. The information that carries the most value in programme governance — and that the system most consistently prevents from flowing — is honest information about what the programme does not know, what assumptions it is relying on, and what could go wrong if those assumptions prove false. + +The risk register, as described in Chapter 10, is the primary risk information instrument. But the risk register is only effective if the culture makes honest risk disclosure rational — if the system does not penalise the person who surfaces a risk, does not convert risks into evidence of poor performance, and does not treat the absence of visible risks as a measure of programme health. Psychological safety, addressed in Chapter 11, is the cultural condition that determines whether risk information flows honestly. Without it, the risk information that exists in engineers' heads is absent from the documents that governance relies on. + +**State and reality.** The actual current state of the programme — which components are working, which are not, which integration assumptions have been validated, which remain unresolved — must be visible to the governance layer in a form that allows honest evaluation. Status reports that present the programme state optimistically, that resolve ambiguity in the direction of good news, that omit the information that would require difficult decisions, are not state information. They are programme theatre — the performance of progress rather than its reality. + +Information that contradicts the plan is more valuable than information that confirms it. A system that makes contradictory information difficult to surface — because surfacing it threatens the programme narrative, creates governance conversations that nobody wants to have, or identifies the person who surfaces it with the problem they are surfacing — does not have a communication problem. It has a governance culture problem that presents as a communication problem. + +--- + +## The story of what has been done + +Chapter 4 identified organisational memory — the preservation of decisions, reasoning, and the story of why things were built as they were — as one of the most critical and most commonly lost forms of knowledge. + +The story is what new team members cannot reconstruct from code and documentation alone. Not the decisions — ADRs capture those. Not the current state — the system itself captures that. The story connects the decisions and the state to the context in which they were made: the constraints that existed at the time, the alternatives that were considered, the pressures that shaped the choices, the things that were tried and did not work before the approach that was adopted. + +Without the story, the next team makes decisions that appear locally rational but that repeat the reasoning of decisions already made — sometimes arriving at the same conclusion after wasted effort, sometimes arriving at a different one that violates a constraint they could not see because the story that explained it had been lost. + +The story is preserved through the practices described in this chapter — the ADRs that capture not just the decision but its context, the documentation that lives alongside the code and evolves with it, the knowledge-sharing practices that distribute understanding across the team before it becomes the property of a single person. When a key person leaves, the story they carried should already be distributed. If it has not been, the programme has a knowledge risk that will surface as a cost when the next decision depends on understanding that can no longer be accessed. + +--- + +## Asynchronous by default + +Synchronous communication — meetings, calls, real-time conversations — does not scale. It requires all participants to be available simultaneously. It produces decisions and agreements that exist only in the memories of the participants unless someone records them. It creates communication dependencies that become scheduling dependencies, adding latency to the information flows that programmes depend on. + +Firmitas strongly biases toward asynchronous communication, supported by living artefacts that carry the information in a form that is accessible to anyone who needs it, at the time they need it, without requiring a meeting to access it. + +The preference is not absolute. Some communication genuinely requires synchronous interaction — conflict resolution, high-ambiguity alignment, incident response, the relationship repair conversations that text cannot adequately support. These are legitimate uses of synchronous communication. They are not the majority of the communication that engineering programmes require. + +The majority of programme communication is informational: sharing the current state, documenting a decision, describing a technical approach, flagging a risk, explaining a constraint. All of this can be done asynchronously, in a form that is more precise, more durable, and more discoverable than the equivalent meeting. The meeting that produces a decision without a written record produces a decision that cannot be referenced, cannot be challenged, and cannot inform the next person who needs to understand the reasoning behind it. + +Architecture Decision Records instead of architecture meetings. Written risk assessments instead of verbal risk discussions. Design documents reviewed through pull requests instead of design review meetings. These are not the elimination of communication. They are the substitution of communication forms that produce durable, discoverable artefacts for communication forms that produce ephemeral shared memories. + +The cultural shift this requires is significant. Many organisations treat synchronous communication — meetings — as the primary evidence of communication happening. The meeting that produced no record produced communication that, for programme purposes, did not happen. Changing this norm requires leadership behaviour change as much as process change. Leaders who make decisions in meetings and do not record them are modelling the behaviour that makes asynchronous-first communication impossible. + +--- + +## Documentation as a delivery enabler + +Documentation fails in most organisations for systemic reasons, not because engineers are careless. The failures are predictable. + +Documentation created to satisfy a governance gate — a gate pack, a design review document, a test plan written to pass a review — is created to satisfy the gate, not to be used. It is written for the reviewer, not the reader. By the time the programme enters the phase the document was supposed to support, the document no longer reflects the decisions that have been made since it was produced. It has drifted. Nobody updates it because the gate has been passed and the document has no further formal purpose. It becomes archaeology within months. + +Large documents written upfront, assembled from the best current understanding of scope that exists before development begins, face a version of the same problem at larger scale. They reflect the programme as it was understood at the moment of their creation. The programme evolves. The documents do not evolve with it, because updating a large document requires a different workflow from the one the development team uses for everything else. It requires someone to remember that the document exists, to find it, to understand it, and to produce a revised version that does not break the references that other documents have to it. This is expensive enough that it reliably does not happen. The large document drifts from reality and eventually describes a programme that no longer exists. + +Docs-as-code addresses both failure modes by treating documentation as an engineering artefact — version-controlled, reviewed through pull requests, stored alongside the code it describes, updated as part of the same workflow that updates the code. Documentation that is changed as part of the same pull request that changes the code it describes stays current without requiring a separate documentation workflow. Documentation that is reviewed through the same mechanisms as code benefits from the same review culture. Documentation that is stored in version control is traceable, attributable, and recoverable. + +The core principle is not a technical one. It is a cultural one: documentation must live where the work lives. If the engineers and architects who know what decisions have been made, and why, and what the current state of the system is, do not own the documentation — if documentation is produced by a separate function, in a separate tool, on a separate workflow — the documentation will not reflect the engineers' knowledge. It will reflect a mediated version of it, filtered through the communication cost of the handoff, and drifting from reality at the rate at which the programme evolves faster than the documentation process can follow. + +--- + +## Documentation types that matter + +Firmitas deliberately limits the types of documentation it considers essential, focusing on the artefacts that carry decisions, reasoning, and state — rather than the artefacts that exist to satisfy compliance requirements that could be satisfied more efficiently by the former. + +**Vision and intent.** Why the product exists, who it serves, what success looks like, and what the principal constraints are. Concise, narrative, stable. The reference point for teams making local decisions under uncertainty. Updated when the programme's fundamental purpose or constraints change, not when individual features change. + +**Requirements and constraints.** The functional and non-functional requirements structured as described in Chapter 11, with traceability from customer need to acceptance criteria. Living artefacts — updated when requirements change, maintained with the same discipline as code. + +**Architecture and design.** The structural shape of the system, captured through C4 diagrams at the appropriate level of abstraction, Interface Control Documents for the boundaries between components and disciplines, and Architecture Decision Records for the significant decisions that shaped the structure. Living artefacts — diagrams updated when the structure changes, ADRs written when decisions are made and superseded when they are revised. + +**Verification and validation evidence.** Test plans, test results, traceability matrices, validation reports. Generated continuously through the iterative core, compiled into the formal evidence record during the validation phase. Not assembled for the audit — produced as the natural output of a quality engineering system that is functioning correctly. + +**Operational documentation.** Deployment procedures, monitoring runbooks, incident response guides, support documentation. Produced with the involvement of the operations team, maintained by the teams who use them, updated when the operational reality changes. Documentation that the operations team cannot rely on is more dangerous than no documentation — it creates false confidence and is likely to fail at the moment it is most needed. + +--- + +## Signals, noise, and information hygiene + +More information is not better information. The programme that produces comprehensive status reports, detailed meeting minutes, exhaustive design documents, and complete audit trails for every decision creates an information environment in which the signals that matter are buried under the volume of information that exists because someone believed that documentation equalled governance. + +Information hygiene is the practice of maintaining signal quality over volume — ensuring that the information the programme produces is meaningful, current, owned, and serves a specific purpose for a specific audience. Artefacts that no one reads should not exist. Documents that have drifted from reality should be updated or removed. Dashboards that show activity rather than health should be replaced by dashboards that show health. + +The mechanisms of information hygiene are simple. Every artefact has a named owner who is responsible for its currency. Every artefact has an intended audience and a defined purpose. Every artefact that has become stale is either updated or explicitly archived. The programme review that surfaces artefacts that nobody reads is the prompt to remove them, not to continue maintaining them. + +The cognitive load of navigating an information environment full of stale, redundant, and low-quality artefacts is not trivial. Engineers who cannot find the current interface specification because it exists in three versions in two tools and none of them is marked as the authoritative one spend time on archaeology rather than engineering. The cost is real. Information hygiene reduces it. + +--- + +## Leadership's role in information flow + +Leadership behaviour is the strongest determinant of whether information flows honestly in a programme. Not the documentation tools. Not the risk register format. Not the governance processes. The behaviour of the people at the top of the authority structure, and specifically how they respond when information arrives that is uncomfortable. + +Leaders who punish bad news create programmes in which bad news arrives late. Not because engineers are hiding it maliciously, but because the system has taught them that the cost of surfacing bad news early — the attention, the scrutiny, the implication of failure — exceeds the cost of managing the problem quietly until it becomes unavoidable. By the time unavoidable arrives, the problem is larger, the options are fewer, and the cost of resolution is substantially higher than it would have been if the information had been surfaced at the first warning. + +Leaders who reward bad news early — who treat early warning of a problem as valuable information rather than as a performance indicator, who respond to escalated risks with decisions rather than with questions about why the risk was not caught sooner — create programmes in which information flows toward the people who need to act on it. The risk register reflects the real risk landscape. The status reports describe the real programme state. The honest three-point estimate is used rather than suppressed. + +This is not a management aspiration. It is a governance requirement. A programme governed on false information is not being governed at all. The governance layer that is reading a green status report and a mostly-amber risk register while the programme is carrying unresolved red items is not making informed decisions. It is making decisions in a vacuum created by the information environment the governance culture has produced. + +The programme board in Chapter 1 did not lack information. It lacked the governance culture that would have treated the information it had as urgent rather than routine. That culture is a leadership product. It is designed, or it emerges by default from the behaviours that leadership models. Leadership that models the treatment of honest risk information as welcome rather than threatening produces organisations where honest information flows. Leadership that models the treatment of honest risk information as a problem to be managed produces organisations where the risk information that governs are the versions that have been sanitised to avoid uncomfortable conversations. + +Communication is a system. Leadership designs the system. The system produces the information environment. The information environment determines the quality of the decisions. The quality of the decisions determines the programme's outcomes. + +Design the system deliberately. diff --git a/content/firmitas/part-2-the-framework/19-technical-debt-and-change-management.md b/content/firmitas/part-2-the-framework/19-technical-debt-and-change-management.md new file mode 100644 index 0000000..de6edea --- /dev/null +++ b/content/firmitas/part-2-the-framework/19-technical-debt-and-change-management.md @@ -0,0 +1,125 @@ +--- +title: "Chapter 19: Technical Debt and Change Management" +--- + +# Chapter 19: Technical Debt and Change Management + +Every engineering programme makes compromises. Scope is reduced to meet a timeline. An implementation shortcut is taken when the proper approach would take longer than the delivery window allows. A design decision is made on incomplete information and would be made differently if the full picture were available. A dependency is accepted because the alternative would require more time than the programme has. + +These compromises are not failures of discipline or courage. They are the rational responses of engineering teams to the conditions of delivery under constraint. Complex programmes always operate under constraints. Compromises are inevitable. The question is not whether compromises will be made but whether they will be made consciously, documented honestly, and managed as the liabilities they are — or made invisibly, left undocumented, and inherited by future teams as constraints they cannot see and cannot evaluate. + +Technical debt is the accumulated liability of those compromises. Change management is the discipline that governs how the programme evolves as understanding improves and conditions change. Both are addressed in this chapter because they share a common governance principle: the decisions that produce technical debt and the decisions that govern change are consequential programme decisions that require the same deliberate ownership as any other programme risk. + +--- + +## What technical debt is and what it is not + +Technical debt is a useful metaphor, introduced by Ward Cunningham, that describes the future cost of shortcuts taken in the present. Like financial debt, technical debt has a principal — the work that was deferred or done improperly — and interest — the compounding cost of carrying the debt, paid in slower development, higher defect rates, and the accumulated complexity of working around rather than through the underlying problem. + +The metaphor has limits that are worth naming. Financial debt is taken on with explicit knowledge of its terms — the principal, the interest rate, the repayment schedule. Technical debt is most commonly taken on without explicit acknowledgement of its terms — without quantifying the principal, without estimating the interest rate, and without any commitment to a repayment schedule. This makes it substantially more dangerous than financial debt, because the organisation carrying it frequently does not know how much it is carrying or what it is costing. + +Firmitas distinguishes between two categories of technical debt that require different governance responses. + +**Deliberate debt** is a conscious, documented trade-off — a shortcut taken with full knowledge of what is being deferred, why it is being deferred, what the cost of carrying the debt will be, and when and how it will be addressed. Deliberate debt is a legitimate programme instrument. The pressure of a delivery deadline may make it rational to implement a temporary approach that will need to be replaced later. The benefit of proving a concept quickly may justify an implementation that would not survive production at scale. The cost of doing something properly before the requirements are fully understood may be higher than the cost of reworking it once the requirements are clear. In each of these cases, the deliberate acceptance of technical debt is a reasonable programme decision — but only when it is genuinely deliberate, genuinely documented, and genuinely managed. + +**Accidental debt** is the accumulation of shortcuts, suboptimal decisions, and deferred work that was never explicitly acknowledged as debt — the implementation that was done quickly under pressure without recording that it needs to be revisited, the architectural decision that was made on incomplete information without flagging the assumption it rests on, the test that was not written because the sprint was ending and the ticket was almost done. Accidental debt is the dominant form of technical debt in most programmes, and it is substantially more dangerous than deliberate debt because it is invisible. It cannot be managed because it is not known. It surfaces as unexplained slowness, as defects in areas that seem unrelated to recent changes, as the discovery during a refactoring exercise that the system has dependencies that nobody documented and several people assumed did not exist. + +The governance response to these two categories is different. Deliberate debt requires documentation, ownership, and a committed resolution plan. Accidental debt requires practices that prevent it from accumulating — architectural discipline, code review culture, definition of done criteria that include quality standards — and periodic investigations that surface what has already accumulated. + +--- + +## The four conditions for legitimate debt + +Deliberate technical debt is legitimate when four conditions are met. When any of them is absent, the debt is not being managed — it is being deferred with the hope that it will not surface as a problem before it can be addressed. + +**Conscious.** The decision to incur the debt was made explicitly, not discovered after the fact. Someone chose this approach, understood what they were deferring, and accepted the trade-off with full knowledge of its terms. Debt that was incurred because nobody noticed, because the pressure of delivery made careful consideration impossible, or because the implications of the shortcut were not visible at the time it was taken is not conscious. It is accidental. + +**Documented.** The debt exists as a named, described item in the programme's technical debt register. What was deferred. Why it was deferred. What the consequence of carrying it is. What resolving it requires. What happens if it is not resolved. Undocumented debt is invisible to anyone except the person who incurred it — and frequently to them as well after the programme has moved on. When that person leaves, the debt remains but the understanding of it is gone. The next team discovers it as a constraint they cannot explain. + +**Business-accepted.** This is the condition that most treatments of technical debt omit, and its omission is one of the primary reasons that technical debt accumulates without management. The decision to carry technical debt is a business decision, not a technical one. The business is accepting a future cost — the cost of resolving the debt — and a current risk — the possibility that the debt will cause problems before it is resolved. That acceptance must be explicit. A senior engineer who accepts technical debt on behalf of the business without business sign-off has made a commitment the business did not know about. The business's subsequent surprise when the debt surfaces as a problem is not failure of awareness. It is the predictable consequence of a governance process that allowed a significant liability to be accepted without the visibility of the people who will pay for it. + +**Resourced for resolution.** Future work to resolve the debt has been identified, estimated, and allocated. Not just acknowledged as something that will need to be done someday — specifically planned, with capacity allocated, within a timeframe that the programme accepts as appropriate for the level of risk the debt carries. Technical debt that is accepted without a credible resolution path is not managed debt. It is deferred failure with optimistic labelling. The backlog of debt items that has been accumulating for three years, revisited at each planning cycle, and deferred to the next cycle is not a managed debt register. It is a list of liabilities the organisation has chosen not to address while pretending to be managing them. + +--- + +## Technical debt taxonomy + +Not all technical debt is the same. Different types carry different risks, require different governance responses, and have different resolution costs. Understanding the type of debt the programme is carrying determines the urgency and approach of the resolution response. + +**Strategic debt** is consciously incurred to accelerate delivery of something more valuable. The temporary integration approach used to validate a market assumption before investing in the proper architecture. The simplified implementation used to demonstrate a capability proof-of-concept before the production design is committed. Strategic debt is the most legitimate form of deliberate debt — it is incurred because the cost of doing it properly before the value is confirmed exceeds the cost of the rework when the value has been confirmed. It is legitimate when the strategic purpose is real, the debt is documented, and the rework plan exists. + +**Tactical debt** is small shortcuts made under time pressure that are safe to carry for short periods. The hardcoded configuration value that needs to be externalised before scaling. The error handling that is adequate for current load but not for production volumes. The test that covers the happy path but not the edge cases that production will reveal. Tactical debt is manageable when it is tracked, when its safe carry period is understood, and when it is resolved within that period rather than deferred indefinitely. + +**Systemic debt** is the most dangerous category — accumulated architectural decisions that are now constraining the system's ability to evolve. The coupling that was introduced in small increments across many deliveries until the system can no longer be changed in any part without understanding all of it. The performance bottleneck that was acceptable at the original scale and is now the primary constraint on the programme's ability to grow. The security architecture that was adequate for the original threat model and is now inadequate for the evolved one. Systemic debt is rarely visible as debt in the moment it is accumulating. It becomes visible when the programme attempts a change that the accumulated debt makes expensive or impossible. + +**Inherited debt** is debt that exists in systems the current team did not build and did not choose. Every programme that builds on an existing system inherits its accumulated shortcuts, its undocumented decisions, and its architectural assumptions. Inherited debt must be assessed on arrival — not assumed to be manageable because it has been managed so far. Many programmes have failed because the inherited technical debt that was present at the start of the programme was treated as background noise rather than as a risk to be understood and managed. The assessment of inherited debt is not optional. It is a programme risk quantification exercise that determines how much of the programme's delivery capacity will be consumed by managing the legacy it has been handed. + +--- + +## Who makes the debt call + +Senior engineers must have the autonomy to identify, manage, and report technical debt. They have the technical understanding to assess what is safe to defer and what constitutes a systemic risk. They have the contextual knowledge to evaluate whether a shortcut is a considered trade-off or a structural liability. They are the people closest to the work, which is where the knowledge that determines the correct debt decision resides. + +But they need two things that are often absent. + +The first is the safety to surface debt honestly. In organisations where raising technical debt is treated as an admission of failure, as evidence that the team is not performing, or as a risk to the programme narrative, debt goes underground. Engineers who have raised debt concerns and watched them be ignored — or worse, used as evidence that the programme needs tighter management — stop raising them. The debt accumulates invisibly. The connection between the governance culture and the organisation's ability to manage its technical debt is direct and consequential. + +The second is the authority to ensure that the business-accepted condition is genuinely met. A senior engineer who identifies a significant piece of technical debt, raises it, gets it acknowledged, and then watches it sit unresolved through three planning cycles while new features are prioritised has learned something important about the organisation's actual commitment to debt management. The next time they identify debt of similar severity, they will make a calculation about whether raising it will produce a different outcome. The resolution commitment must be real to make the identification commitment real. + +--- + +## Change management + +Change is not a failure of planning. It is the inevitable product of building complex systems in conditions of genuine uncertainty. Requirements that are specified before the system is fully understood will be revised as understanding improves. Architectural assumptions that were valid when they were made will be invalidated as the system is built and its real behaviour becomes visible. External dependencies will change in ways that affect the programme's commitments. + +The question is not whether change will occur. It is whether the programme has a mechanism for managing change that preserves the integrity of its commitments, maintains the traceability of its requirements and design decisions, and ensures that the decisions about whether and how to accommodate a change are made at the appropriate level with the appropriate evidence. + +Firmitas distinguishes between three classes of change that require different governance responses. + +**Learning-driven change** is change that results from the programme's discovery of information that was not available when earlier decisions were made. The integration test that reveals an interface assumption was wrong. The customer feedback that reveals a requirement was misunderstood. The performance measurement that reveals an architectural assumption was invalid. Learning-driven change is expected and welcomed in a programme that is practising iterative development — it is the evidence that the feedback loops are working. The governance response is not to prevent learning-driven change but to ensure that it is incorporated systematically — updating the affected requirements, revising the affected design decisions, creating ADRs that capture the new understanding, and assessing the impact on the delivery estimate. + +**Scope change** is change to the programme's committed deliverable — a new requirement added, an existing requirement revised, a feature removed. Scope change requires formal change management because it affects the programme's commitments — the delivery date, the cost, and the quality of what will be delivered. Scope change that is accepted without assessing its impact on the programme's commitments is not change management. It is the silent absorption of additional obligation into a programme that is already committed — which is how programmes accumulate the gap between their committed scope and their delivered capacity that surfaces as the surprise at month six that Chapter 1 described. + +Every scope change must be evaluated against four questions: What is the impact on the delivery estimate? What is the impact on the risk register? What is the impact on the architectural decisions that have already been made? And what is the commercial and contractual implication? Only after those questions have been answered honestly can the decision about whether to accept the change be made with adequate information. + +**Architectural change** is change to the fundamental decisions that shape the system's structure — the boundaries between components, the interfaces between disciplines, the patterns and principles that the development teams are building within. Architectural change carries the highest programme risk of the three classes, because its consequences are systemic. An architectural change that affects a shared interface affects every team that uses it. An architectural change that revises a fundamental pattern potentially requires every team's implementation of that pattern to be revisited. + +Architectural change requires the ADR process — an explicit record of the decision, the rationale, the alternatives considered, and the assessment of impact. It requires involvement from the teams affected before the decision is finalised — because the teams building within the architecture have knowledge of its current state that the architectural decision-makers may not have. And it requires an impact assessment on the delivery estimate and the risk register before the change is accepted. + +--- + +## The cost of thrash + +Change is not free. Every change to requirements, design, or architecture carries a cost — the cost of revising the affected artefacts, updating the affected implementations, re-running the affected tests, and communicating the change to everyone who has been working on assumptions that the change invalidates. + +When change is frequent, unpredictable, and poorly governed, the cost of managing change can exceed the cost of the development work itself. Teams spend more time responding to change than delivering new capability. The instability of the requirements and the architecture makes it impossible to build on previous work without the risk of being invalidated by the next change. Cognitive load increases as teams must hold increasingly complex mental models of what is current versus what has been superseded. Quality degrades because the test coverage required to validate a changing system cannot be maintained at pace with the changes. + +This is the failure mode called thrash — a programme that is consuming its delivery capacity in the management of change rather than in the production of value. Thrash is not caused by change itself. It is caused by change that is poorly governed — accepted without impact assessment, incorporated without architectural discipline, and communicated without adequate time for the affected teams to update their work. + +The governance response to thrash is not to prevent change. It is to ensure that change goes through the process required to assess its impact, accept it deliberately, and incorporate it without destroying the work that preceded it. Programmes that cannot change without thrashing have not been designed for change. They have been designed for stability under conditions — predetermined requirements, validated architecture, stable external dependencies — that complex programmes do not reliably provide. + +--- + +## Knowing when not to change + +One of the most important and least discussed aspects of change management is the discipline of not changing — of recognising when a proposed change would consume more delivery capacity than the value it would add, and explicitly deciding not to accept it. + +Every programme receives more change requests than it can productively accommodate. The features that were not in the original scope. The requirements revisions that make previous implementations obsolete. The architectural improvements that would benefit future maintainability but at the cost of current delivery. The client requests that arise from evolving understanding of their operational needs. + +All of these changes may be individually justified. Collectively, they can consume the programme's delivery capacity and prevent any of them from being delivered well. The discipline of change management includes the discipline of saying no — of applying the governance questions to each proposed change, assessing its real cost against its real benefit, and declining the changes that do not justify their delivery impact. + +This requires a governance culture that can have honest conversations about what the programme can and cannot accommodate. A governance culture that cannot decline client change requests because client relationships are managed by people who lack the authority to say no to clients while having the authority to commit engineering teams is a governance culture that will absorb change until it destroys the programme it was supposed to protect. + +The commercial conversation about what a programme can and cannot accommodate is a governance decision that belongs at the level where both the commercial commitment and the delivery capacity are understood. That level is not the engineering team. + +--- + +## Technical debt and the long view + +The relationship between technical debt, change management, and the long-term health of the systems the programme produces is the direct expression of Principle 10 and Principle 11 — the cost of short-term optimisation is paid by people who had no say in the decision, and optimising for long-term system health is the leadership responsibility that most directly determines whether what the programme builds will be a sustainable asset or a accumulating liability. + +Programmes that manage debt deliberately — accepting it when it is genuinely justified, documenting it honestly, ensuring business acceptance, and resourcing resolution — produce systems that are sustainable. They carry known liabilities that are being managed. Future teams can understand the constraints the system operates under, evaluate which constraints are still valid, and plan their evolution accordingly. + +Programmes that allow debt to accumulate invisibly — through accidental shortcuts, through the systematic treatment of quality as optional under schedule pressure, through the acceptance of change without architectural discipline — produce systems that become progressively harder and more expensive to operate and evolve. The interest on the debt compounds. The cost of each new delivery increases as the system's complexity grows. The teams maintaining the system spend an increasing proportion of their capacity managing the consequences of the decisions made under pressure in previous delivery cycles. + +The technical debt that is created in a programme is not just the programme's problem. It is the inheritance of every team that will touch the system afterward — potentially for decades. Engineering stewardship, in the Firmitas sense, extends beyond initial delivery to include responsibility for the operability, maintainability, and evolvability of what is built. Accepting technical debt without managing it is not a programme delivery decision. It is an act of borrowing against the future of the people who will maintain what the programme has built. diff --git a/content/firmitas/part-2-the-framework/20-capability-people-and-building-for-the-long-term.md b/content/firmitas/part-2-the-framework/20-capability-people-and-building-for-the-long-term.md new file mode 100644 index 0000000..bd658f0 --- /dev/null +++ b/content/firmitas/part-2-the-framework/20-capability-people-and-building-for-the-long-term.md @@ -0,0 +1,113 @@ +--- +title: "Chapter 20: Capability, People, and Building for the Long Term" +--- + +# Chapter 20: Capability, People, and Building for the Long Term + +A programme can have sound architecture, honest estimates, a functional risk register, and a governance model that generates decisions rather than acknowledgements — and still fail to deliver if the people responsible for the delivery do not have the capability the work requires. Capability gaps are programme risks. They are among the most common programme risks and among the most consistently underestimated. + +Underestimated because they are visible before they become problems, but the visibility does not reliably produce the response. The gap between the capability a programme needs and the capability the team has is usually identifiable during the structured foundations phase — if anyone is looking for it. It is rarely managed with the same rigour as a technical risk or an integration dependency. Instead, it is absorbed silently, managed through on-the-job learning under delivery pressure, and attributed to execution failure when the limitation it produces surfaces as a programme problem. + +This chapter addresses capability as a delivery discipline — how gaps are identified, how responses are matched to the specific nature of each gap, how people are developed rather than consumed, and what the long-term stewardship of engineering capability means for organisations that want to build something sustainable rather than running their people down in service of the next deadline. + +--- + +## Capability gaps are programme risks + +The lead systems engineer on the programme in Chapter 1 identified three red risks in week three. None of them was a capability gap. That is not because capability gaps did not exist on that programme — they almost certainly did. It is because capability gaps are rarely surfaced as programme risks in the way that technical and integration risks are. + +The reasons for this are structural. Technical risks are visible in the architecture and the requirements — the unvalidated interface, the unconfirmed dependency, the ambiguous specification. Capability gaps are visible in the people — in what they have and have not done before, in what skills the programme requires that the team does not currently possess. Surfacing a capability gap requires someone to look at the team as honestly as they look at the architecture, to ask what this programme needs that the team cannot yet reliably provide, and to treat the answer as programme risk rather than as an uncomfortable observation about the people involved. + +This honesty is rare, for understandable reasons. Naming a capability gap feels like a criticism of the people whose gap it is. In a team that has been assembled and is working, the social cost of formally identifying gaps is high. The result is that capability gaps are managed informally — through close supervision, through pairing people who have the capability with people who are developing it, through tolerating slower delivery in the areas where the capability is weakest — without the explicit identification and resource allocation that would constitute proper risk management. + +The informal management is not always wrong. Some capability gaps are appropriate to manage informally — when they are small, when the work is not on the critical path, and when the person developing the capability has adequate support and slack to do so. But when the gap is in a critical capability, on a critical path, in a team that is already at full utilisation, informal management produces the same outcome as managing any other risk informally: the risk materialises on schedule, and the consequence is attributed to execution failure rather than to the risk management failure that allowed it to materialise unmanaged. + +--- + +## Identifying capability gaps + +Capability gap identification is a structured activity that belongs in the programme's foundations phase, not a conversation that happens when a problem has already surfaced. + +The identification process has two parts. The first is a clear articulation of what the programme needs — not generically, but specifically for this programme, at this point in its delivery. What technical capabilities are required? What domain knowledge? What regulatory and compliance understanding? What toolchain and process familiarity? What leadership and communication capabilities are required at the team level? + +The second is an honest assessment of what the team currently has. Not a self-assessment by the team members themselves — self-assessments of capability are reliably optimistic, for the same reason that self-assessments of risk tend toward amber rather than red. A structured assessment by someone with the relevant expertise to evaluate the team's capability against the programme's requirements. + +The gap between these two assessments is the capability risk. It should be entered in the risk register in the same form as any other programme risk — specific description, probability and impact assessment, named owner, mitigation plan, trigger date. The mitigation plan is the capability development response described in the next section. The trigger date is the point in the programme at which the capability will be needed at the level the programme requires — because the response to a capability gap must be initiated early enough to be effective before that point, not at the point itself. + +--- + +## The capability decision framework + +When a capability gap is identified, the response must be proportional to the criticality of the capability and the duration for which it is needed. The most common response — on-the-job upskilling with minimal support — is chosen because it is cheap and immediate, but it is appropriate only when the gap is small, the capability is not on the critical path, and adequate support and slack exist for genuine learning to occur. + +Four questions determine the appropriate response. + +**How critical is this capability to delivery outcomes?** If the gap is in a non-critical area, on the job learning with appropriate support is usually adequate. If the gap is in a critical path capability — the firmware architect, the safety engineer, the systems integrator, the regulatory specialist — the response must be more deliberate and better resourced. A capability gap in a critical path capability that is managed informally is not being managed. It is being deferred until it surfaces as a delivery problem. + +**How long is the capability needed?** A capability needed for a specific, bounded activity in the programme — the regulatory submission phase, the performance characterisation sprint, the security review — may justify a specialist contractor or third-party engagement for the duration of that activity. A capability needed throughout the programme's life requires a different response — either recruiting someone who has it, or developing it within the team at a pace and with a resource allocation that matches the timeline on which it will be needed. + +**Is there time for the person to develop the capability before it is needed?** If the answer is yes — if the capability will not be needed at full competence for several months, and if adequate support and slack exist for structured development — on-the-job development with mentoring and protected learning time is the right response. If the answer is no — if the capability is needed now, or within a window that does not allow adequate development time — bringing in external capability while developing internal capability in parallel is the honest response. Expecting someone to develop a complex capability in a compressed timeframe while delivering at full pace is not a development plan. It is a plan to fail more slowly. + +**What happens if the capability gap is not filled?** If the honest answer is that an important delivery milestone will be missed, that the programme will produce something that does not meet its requirements, or that a regulatory or safety obligation will not be met, the capability gap is a programme risk that must be owned at the level that made the commitment — not absorbed silently by the team that is expected to deliver against it. + +--- + +## The options and their honest assessment + +**On-the-job upskilling with structured support** is the most common response and the most commonly misapplied one. It is appropriate when the gap is modest, the work is not on the critical path, the person has adequate cognitive slack to learn while delivering, and the support — a mentor, a pairing partner, protected learning time — is genuinely provided rather than nominally promised. It is the most likely to be applied when none of these conditions are met, because it is the cheapest option and the one that requires no difficult decisions. + +The specific failure mode of on-the-job upskilling without adequate support is that the person develops the capability more slowly than the programme needs it, carries more cognitive load than is sustainable, produces work of lower quality during the development period, and ends up neither developing the capability adequately nor delivering the work effectively. The programme absorbs the cost of this through slower delivery, higher defect rates, and the programme manager's growing awareness that certain areas of the work are progressing less well than others — which is attributed to individual performance rather than to the capability response decision. + +**Specialist contractor or third-party engagement** is appropriate when the capability is needed for a bounded period, the window for development does not exist, or the specialisation required is too narrow to justify a permanent hire. It is also appropriate as a bridging response while a permanent capability is being developed or recruited — providing the capability the programme needs now while building the internal capability that will sustain it over time. + +The risk of specialist contractor engagement is dependency — the programme becomes reliant on knowledge that resides in an external party rather than in the organisation. Firmitas accepts this risk when it is the honest response to the capability gap, and manages it through structured knowledge transfer — ensuring that the specialist's knowledge is progressively distributed to the internal team rather than remaining concentrated in the specialist. + +**Training and structured development** is appropriate for capabilities where the gap is real but the development can be done through formal training, certification, or structured learning rather than through on-the-job development alone. Regulatory knowledge. Toolchain proficiency. Quality management practices. These are capabilities that can be developed systematically through structured programmes, rather than through the unguided learning that on-the-job upskilling often provides. + +**Recruitment** is appropriate when the capability is needed for the long term, when the internal development path is too slow, and when the capability is strategic enough to justify the cost and time of hiring. The lead time of recruitment — typically several months from the decision to hire to the point at which a new person is contributing at full effectiveness — must be factored into the capability timeline. Deciding to recruit when the capability is already needed does not solve the near-term problem. It creates an expectation of capability that will not be met for months, during which the programme continues to operate with the gap it has. + +--- + +## People are not interchangeable + +The governance and commercial pressure that produces the capability responses described above also produces a more fundamental misframing: the treatment of people as interchangeable units of capacity that can be swapped in and out of programmes without cost to programme outcomes or to the people themselves. + +This misframing is embedded in the language of resource management. Headcount. FTEs. Resource allocation. The language treats people as fungible — one senior engineer equivalent to another, one firmware developer interchangeable with the next. The reality is the opposite. + +The senior engineer who has been on a programme for twelve months carries knowledge that no new hire can replicate in a week, or a month, or possibly ever. They know which architectural decisions were made and why. They know which risks were debated and what the arguments for and against were. They know which constraints are fundamental to the system and which are historical artefacts. They know where the technical debt is and what it will cost to address. They know the client's real concerns as distinct from their stated requirements. This knowledge — the programme's accumulated understanding, distributed across the people who have been building it — is the most valuable asset the programme has. It cannot be put in a document quickly enough to be fully preserved. It is lost, partially and irreversibly, every time someone who carries it leaves the programme. + +The cost of this loss does not appear on the capacity plan. It appears later — in the decisions made by incoming team members without the context to make them correctly, in the technical debt accumulated because the understanding required to manage it has been lost, in the integration failures that result from assumptions that the departing team member held and never documented, in the rework required when the system's undocumented constraints are violated by people who did not know they existed. + +Principle 12 exists because this cost is real and because the treatment of people as interchangeable resources is one of the primary mechanisms by which it is incurred. Your people are your most significant investment — recruited carefully, developed over time, and irreplaceable in practice. The organisation that treats them as interchangeable will eventually have interchangeable people — not because that is who they are, but because the people with irreplaceable knowledge and the self-respect to expect to be treated accordingly have left for organisations that recognise the difference. + +--- + +## Building capability for the long term + +The argument in this chapter so far has been primarily about managing capability gaps — the immediate, programme-level responses to capability that is insufficient for current delivery needs. But capability management has a longer horizon than any single programme, and the organisations that consistently deliver well are the ones that build capability deliberately over time rather than responding to gaps reactively when programmes are already at risk. + +Building capability for the long term requires three things that most organisations underinvest in. + +**Deliberate development paths.** Engineers do not develop from competent to expert without deliberate exposure to progressively complex challenges, structured feedback, and protected time for the reflection that converts experience into expertise. The organisation that puts engineers in challenging positions without adequate support produces engineers who are anxious rather than developing. The organisation that provides support without challenge produces engineers who are comfortable but not growing. Deliberate development requires both — structured exposure to challenges that are at the edge of current capability, with the mentoring and slack that allow genuine learning to occur rather than mere survival. + +**Knowledge transfer as a programme discipline.** The knowledge that engineers develop on programmes must be distributed before it becomes the property of individuals who can take it with them when they leave. This is not a matter of documentation alone — documentation captures explicit knowledge, not the tacit understanding that comes from experience. Distributing tacit knowledge requires practices that create the conditions for it to be shared: pair programming, design reviews that involve developing engineers as full participants rather than observers, retrospectives that examine what the team learned rather than just what happened, mentoring relationships in which the knowledge holder deliberately teaches rather than simply working alongside the learner. + +**Technical career paths that do not require people to become managers.** The most technically capable engineers in most organisations face a career structure that offers two paths: become a manager, or remain in a role whose seniority and compensation plateau at a level that the organisation does not consider commensurate with real leadership. This structure consistently loses the people whose technical depth is the organisation's most valuable capability — either to management roles for which they are less suited than the technical roles they have vacated, or to other organisations that offer senior technical tracks. + +The organisations that retain deep technical capability offer technical career paths that are treated as equivalent in seniority, compensation, and organisational influence to management paths. The principal engineer who influences architectural decisions across the portfolio. The technical fellow who advises on emerging technology strategy. The domain expert whose judgement is explicitly sought in governance decisions that require their knowledge. These are real leadership roles, even though they do not involve managing people. Treating them as such — with the recognition, the compensation, and the governance influence that reflects their value — is how organisations keep the capability they have invested years in building. + +--- + +## The cost of getting this wrong + +The cost of poor capability management is not visible on a delivery dashboard. It accumulates in ways that look like execution problems rather than capability problems — slower delivery in areas where the capability is weakest, higher defect rates in components built by engineers who are developing rather than competent, integration failures at the boundaries where discipline-specific knowledge should have been applied and was not. + +By the time these costs become visible, the root cause has typically been obscured by months of programme narrative. The delivery problem is real. The connection to the capability gap that preceded it — the gap that was not entered in the risk register, not escalated as a programme risk, not addressed before it became a constraint on delivery — is lost. + +The programme management response, when these problems surface, is typically to apply more pressure to the people experiencing the capability limitation — to manage their work more closely, to set harder deadlines for the areas that are running late, to escalate the performance of the team members in whose work the gap is most visible. This response is not only ineffective. It is the structural consequence of treating a capability risk as an execution failure. + +The governance response, when a capability gap surfaces as a delivery problem, should not be pressure on the individuals affected. It should be the governance question that was not asked earlier: what capability does this programme require that the team does not have, why was this not identified and managed as a programme risk, and what needs to change to provide the programme with the capability it needs — now that the gap is visible, and in future programmes before it becomes a problem? + +That question is more uncomfortable than the pressure response. It locates the cause correctly. It requires the governance layer to examine its own risk management rather than the team's delivery performance. It produces the learning that might prevent the next programme from experiencing the same gap in the same way. + +That is what capability management, properly done, looks like. diff --git a/content/firmitas/part-2-the-framework/21-portfolio-flow-and-prioritisation.md b/content/firmitas/part-2-the-framework/21-portfolio-flow-and-prioritisation.md new file mode 100644 index 0000000..d51c95f --- /dev/null +++ b/content/firmitas/part-2-the-framework/21-portfolio-flow-and-prioritisation.md @@ -0,0 +1,121 @@ +--- +title: "Chapter 21: Portfolio Flow and Prioritisation" +--- + +# Chapter 21: Portfolio Flow and Prioritisation + +Every failure pattern described in this book exists at the programme level. A programme with a reverse-engineered plan. A programme with an unresolved risk register. A programme with a delivery team that was not trusted with its own commitments. The failures are programme-level failures, and the structural responses described in Chapters 7 through 20 are programme-level responses. + +But programmes do not exist in isolation. They are funded, prioritised, and governed as part of a portfolio — a collection of programmes, products, and investments that the organisation is pursuing simultaneously with finite people, finite capability, and finite governance attention. The decisions that determine which programmes start, which continue, which pause, and which stop are portfolio decisions. And the portfolio-level failure modes — too much work in flight, hidden dependencies between initiatives, priorities that change faster than delivery systems can absorb — produce the same class of outcomes as programme-level failures, at a larger scale and with a longer tail. + +This chapter addresses portfolio management as a flow problem — not a planning problem, not a resource allocation problem, not a prioritisation methodology problem — but a system design problem whose resolution requires the same structural thinking that programme delivery requires. + +--- + +## The portfolio as a flow system + +The conventional framing of portfolio management treats the portfolio as a collection of projects to be planned, tracked, and reported on. Projects are selected against a business case, resourced against a capacity plan, scheduled against a timeline, and tracked against a milestone plan. The portfolio review confirms that projects are progressing, identifies projects that are behind, and manages the commercial and strategic implications of portfolio performance. + +This framing is not wrong. It is insufficient. It treats the portfolio as a planning artefact rather than as a delivery system — and it produces the same failure mode as treating a programme's Gantt chart as a delivery system rather than as a representation of delivery that must be governed against operational reality. + +A portfolio is a flow system. It is a system through which work enters — as requirements, ideas, mandated obligations, commercial commitments — progresses through various stages of delivery, and exits as operational capability, value for customers, and returns for the business. The performance of the portfolio as a system is determined by how smoothly work flows through it, how much work is in progress at any given time relative to the system's capacity to process it, and how effectively the system surfaces and resolves the constraints that slow flow. + +The application of flow principles to portfolios — limited work in progress, queue management, explicit dependency visibility, constraint identification and resolution — produces substantially better outcomes than the planning-and-tracking approach because it addresses the actual dynamics of the system rather than a simplified model of it. It also produces the same governance implications at the portfolio level that flow principles produce at the programme level: the people making portfolio decisions need to understand how the portfolio system actually works, not just how the portfolio plan says it should work. + +--- + +## The cost of too much work in flight + +The most consistent portfolio-level failure is accepting more work than the system can productively deliver simultaneously. This failure mode is the portfolio equivalent of the team at full utilisation — a system that appears fully occupied and productive while actually accumulating queues, deferring integration, and degrading the quality of everything it is attempting to deliver. + +The symptoms of excessive portfolio WIP are recognisable. Programmes that are "nearly done" for months, consuming resource and governance attention without closing. Teams context-switching across multiple initiatives because each one has a sufficiently urgent reason to claim their time. Dependencies between programmes that nobody is managing because each programme is treating the dependency as someone else's problem. Integration phases that stretch indefinitely because the components being integrated are owned by teams that are each working on three other things simultaneously. Leadership confusion despite extensive reporting, because the reporting describes individual programme status without revealing the systemic constraint that is preventing the portfolio from progressing. + +The systemic constraint in portfolios with excessive WIP is almost always the same: too many commitments against finite delivery capacity, with the commitment decisions made without adequate assessment of what the capacity is and what is already committed against it. The portfolio looks manageable on a roadmap. The roadmap does not show the queues. It shows the work. The queues are invisible until they surface as delays. + +The governance principle follows from Chapter 12's utilisation argument: limiting work in progress at the portfolio level is not a management preference. It is a prerequisite for portfolio flow. A portfolio that starts less work will finish more — because the work it does start will receive adequate attention, progress without being constantly interrupted by competing priorities, and complete without the delays that excessive context-switching produces. + +--- + +## The prioritisation decision + +Portfolio prioritisation is the decision about what to start, continue, pause, or stop. It is one of the most important governance decisions an organisation makes, and one of the most poorly made — because it is typically made on the basis of incomplete information, under political pressure, against a plan rather than against current reality, and without the authority to act on what the information says. + +This last point is the one you identified precisely: organisations frequently have the information required to make good prioritisation decisions. The dependencies are known, or knowable. The estimates exist, or can be produced. The risks are on registers. The resource constraints are visible. What is missing is not the information. It is the authority and the courage to act on it — to say that this initiative should be deferred because the dependency it requires has not been resolved, or that this programme should be stopped because the risk that has materialised has moved it beyond the most likely case into the worst case with no recovery path, or that this backlog item should not be started this quarter because the team that would build it is already committed to something that will not complete in time to release capacity. + +The plan protects the people who follow it. Deviating from the plan requires authority and the willingness to defend the decision to the stakeholders who built their own plans around what the plan said would be delivered. In most organisations, that willingness is rare enough that the plan becomes the authority — work proceeds because the plan says it should, regardless of whether the conditions for successful delivery exist. + +This is plan inertia. The plan was a model of the future at the time it was produced. The future has changed. The plan has not. The organisation continues to execute the model of the future that was constructed under different conditions, rather than the model of the future that the current conditions warrant. The result is the same as the programme that proceeds against the reverse-engineered plan despite the honest estimate sitting in the risk register: a delivery system managed against a fiction, producing outcomes that the fiction could not predict. + +--- + +## What good prioritisation requires + +Portfolio prioritisation that produces good outcomes requires four things working together. + +**Accurate information.** The actual state of each initiative — its current risk register, its current estimate, its current dependency state, its current team capacity. Not the plan-versus-actual report that shows percentage completion against the committed timeline. The operational reality of where each initiative actually is, what is blocking it, and what it needs to progress. + +**Explicit decision criteria.** The principles against which prioritisation decisions are made. What matters more — strategic alignment, commercial urgency, risk reduction, regulatory obligation, customer commitment? These criteria must be established before the prioritisation decision, not inferred from the outcome of a political negotiation about which stakeholder's priorities win. Without explicit criteria, prioritisation is not a governance decision. It is a power contest dressed as a governance process. + +**Real decision authority.** Someone must have the authority to make prioritisation decisions and act on them — to defer an initiative that the dependency analysis says should be deferred, to stop work that the risk assessment says should not continue, to decline a new commitment that the capacity analysis says the organisation cannot absorb. Authority distributed across too many stakeholders produces decisions that satisfy nobody and change nothing. Authority concentrated in a single governance body that meets infrequently and lacks operational knowledge produces decisions that are slow, poorly informed, and disconnected from the reality they are supposed to govern. + +**Protection from plan inertia.** An explicit governance mechanism that treats the current plan as an input to the prioritisation decision rather than as its output. The plan represents commitments made under conditions that existed when the plan was produced. When conditions change — when a dependency does not resolve, when a risk materialises, when a higher-priority obligation emerges — the plan must be subordinate to the current reality. An organisation that cannot deviate from its plan without a governance crisis has not built a delivery system. It has built a commitment structure that converts every unexpected event into a failure narrative. + +--- + +## The four prioritisation questions + +When assessing what to start, continue, pause, or stop, four questions provide the governance structure for the decision. + +**What is the cost of delay?** Not rhetorically — specifically. What value is lost, what risk accumulates, what dependency is blocked for every week this work is deferred or this decision is avoided? Cost of delay is the most important input to prioritisation decisions and the one most frequently omitted. An initiative with a high cost of delay should be prioritised above one with a low cost of delay, regardless of which one is currently further advanced in the plan. Prioritising based on plan status rather than cost of delay consistently produces portfolios that complete the wrong work first. + +**What does starting this now prevent?** Every initiative that starts consumes capacity. That capacity is unavailable for other things. Starting initiative X now means that initiative Y, which also needs that capacity, will be delayed. This is the opportunity cost of the start decision, and it is almost never calculated explicitly. A prioritisation process that evaluates each initiative independently, without assessing what starting it prevents, systematically underestimates the cost of the portfolio's WIP level. + +**Are the conditions for success present?** Is the scope clear enough to begin? Are the dependencies resolved or actively managed? Is the capability available? Are the estimates honest? Has the risk register been produced and reviewed? If the conditions for success are not present, starting the work does not accelerate delivery. It accelerates the accumulation of rework. A programme that starts before its foundations are adequate will spend its early phases discovering what the foundations phase should have established — at higher cost, under greater pressure, with less time to resolve the problems it finds. + +**What does the honest risk picture say?** Not the amber risk register. The real risk landscape. If the risks attached to this initiative materialise, what happens to the portfolio? Is there slack to absorb the impact, or will it cascade into other initiatives? If a risk materialises and the programme moves from most likely to worst case, does the portfolio have the capacity to absorb that and continue to meet its other commitments, or does the worst case on one programme threaten the delivery commitments of the whole portfolio? + +--- + +## Dependencies as first-class portfolio risks + +Dependencies between initiatives are the most consistently undermanaged risk in portfolio management. Every portfolio has them. Most portfolios do not manage them explicitly until they become blockers. + +A dependency in a portfolio is a condition under which one initiative's progress depends on the output, decision, or completion of another. The team building the integration layer depends on the API specification from the platform team. The regulatory submission depends on the evidence produced by the validation programme. The client launch depends on the infrastructure that a separate internal programme is delivering. Each of these dependencies is a risk — if the upstream initiative is delayed, the downstream initiative is blocked. If the dependency is not managed explicitly, the downstream initiative proceeds on the assumption that the upstream output will arrive on time, and discovers the delay at the point of impact rather than at the point of maximum opportunity to mitigate it. + +Firmitas requires dependencies to be managed as first-class portfolio risks — explicitly identified, owned at the portfolio level, and actively monitored in the same way as programme-level risks. Each dependency has a trigger date — the point at which the upstream output is required for the downstream initiative to proceed without delay. If the upstream output will not be available by the trigger date, the portfolio governance layer needs to know in time to make a decision: accelerate the upstream work, defer the downstream work, or accept the delay and manage the commercial and programme consequences of that decision explicitly. + +Dependencies that are not managed explicitly are not being managed. They are being assumed away until they surface as emergencies. + +--- + +## Portfolio governance through risk and issues + +The governance principle established in Chapter 10 — that senior management governs programmes through risks and issues rather than through task status — applies at the portfolio level with the same force. + +Portfolio-level governance through risk and issues means that the portfolio review is structured around the initiatives that require governance attention — the programmes with escalated risks, the dependencies approaching their trigger dates, the initiatives whose current risk picture materially differs from the plan — rather than around a status overview of every initiative in the portfolio. + +A portfolio review that devotes equal time to every initiative regardless of its current risk state is not governing the portfolio. It is reviewing it — which is a different activity, with a different governance value. Reviewing the portfolio provides assurance. Governing it produces decisions. The governance layer that needs to make decisions to resolve escalated risks and manage critical dependencies does not have time to review every initiative in equal detail. It needs a mechanism that surfaces the initiatives requiring governance attention and provides the information required to make the decisions they need. + +The portfolio risk and issues register is that mechanism. It aggregates the escalated risks and issues from every programme in the portfolio, adds the portfolio-level dependencies and their status, and presents the governance layer with a view of where decisions are required. The portfolio review that starts with the register and ends when the required decisions have been made is governing the portfolio. Everything else is reporting on it. + +--- + +## Portfolio flow in regulated environments + +In regulated or safety-critical domains, portfolio flow is constrained by validation cycles, certification requirements, and external approval processes that do not respond to portfolio prioritisation decisions in the way that internal dependencies do. A regulatory body that requires a six-month review period for a safety case submission does not compress that period because the portfolio's commercial priorities demand it. + +Firmitas accommodates these constraints by treating regulatory dependencies as first-class portfolio risks with their own trigger dates and escalation paths. The regulatory submission is not a milestone to be managed against the programme plan. It is a dependency with a specific content requirement, a specific submission date, and a specific review period that must be factored into the portfolio's prioritisation decisions from the earliest planning stage. + +The portfolio that treats regulatory constraints as planning inputs from the start — sequencing work to produce the evidence the regulatory submission requires, allocating validation capacity with adequate lead time, and managing the dependency on the regulatory body's review period as a constraint rather than a variable — delivers against regulated commitments without the crisis that characterises the portfolio that discovers these constraints late. + +--- + +## From project lists to value streams + +The final observation in this chapter addresses the framing of the portfolio itself. A portfolio managed as a list of projects — each with a defined scope, timeline, and budget, each optimised independently — produces the local optimisation failure that Chapter 4 identified at the programme level. Projects complete. The underlying capability they were supposed to produce is delivered in fragments, by teams that have moved on, to stakeholders who must assemble the fragments themselves. The value that the investments were supposed to generate does not follow automatically from the project completions. + +A portfolio managed as a set of value streams — persistent, long-lived investments in capabilities that generate value continuously rather than delivering and closing — produces a different relationship between investment and outcome. The team responsible for the customer onboarding capability does not complete a project and disband. It owns the capability, evolves it as customer needs change, maintains it as the operational infrastructure it supports evolves, and is measured by the outcomes the capability produces rather than by the milestones it hits. + +This is a different organisational model, and it is not appropriate for every portfolio. Some programmes are genuinely one-time investments that should complete and close. But in portfolios where the same capability requires ongoing investment — where the product will evolve, the platform will be maintained, the regulatory compliance will need to be updated — the project framing creates discontinuity, knowledge loss at every programme close, and the perpetual reinvestment in reestablishing the context that a persistent team would carry. + +The portfolio that manages its long-lived investments as value streams rather than as projects retains the capability knowledge, the customer understanding, and the operational continuity that the project model discards at every handoff. The cost of this retention is the ongoing team investment. The return is the compounding improvement in delivery quality and velocity that comes from teams that know their domain deeply rather than learning it afresh each time. diff --git a/content/firmitas/part-2-the-framework/22-business-alignment-economics-and-the-customer.md b/content/firmitas/part-2-the-framework/22-business-alignment-economics-and-the-customer.md new file mode 100644 index 0000000..9e325b2 --- /dev/null +++ b/content/firmitas/part-2-the-framework/22-business-alignment-economics-and-the-customer.md @@ -0,0 +1,129 @@ +--- +title: "Chapter 22: Business Alignment, Economics, and the Customer" +--- + +# Chapter 22: Business Alignment, Economics, and the Customer + +Engineering exists within a commercial reality. Programmes are funded because they are expected to produce value — for customers, for the business, for the portfolio of investments the organisation is managing. The decisions engineers make — about architecture, about quality, about technical debt, about how much time to invest in testing and documentation — carry economic consequences whether or not those consequences are acknowledged. Engineering that ignores commercial reality does not produce better engineering. It produces engineering that is technically excellent and commercially irrelevant, or technically excellent and unaffordable, or technically excellent and delivered to no one who needed it. + +At the same time, engineering that is entirely subordinated to commercial pressure — that optimises for apparent delivery pace at the cost of quality, that defers every risk that would slow the current sprint, that accepts every commercial commitment without assessing its feasibility — does not produce commercial value either. It produces the programme in Chapter 1: on time, on budget, on a plan that was reverse-engineered from the committed date, delivering something that satisfied the contract while failing the customer. + +The relationship between engineering and the commercial context it operates in is not a balance to be struck once and maintained. It is a continuous alignment challenge — keeping the engineering function connected to the customer need it exists to serve, connected to the business outcomes it is supposed to produce, and connected to the economic reality that determines what is sustainable. + +This chapter addresses three dimensions of that alignment: the customer, the economics, and the strategic connection between engineering decisions and business outcomes. + +--- + +## The customer is not the business + +The most important alignment distinction in engineering programme management is one that most programmes blur or ignore entirely: the customer is not the business. + +The business commissions the programme. The business holds the contract. The business receives the delivery. The business manages the client relationship. In many programmes, particularly those delivered by an engineering organisation on behalf of a commercial team, the business is the only voice the engineering team hears. The requirements come from the business. The acceptance criteria are defined by the business. The feedback on what is delivered is filtered through the business's interpretation of what the customer needs. + +The customer is the person or organisation whose problem is actually being solved — whose life, operation, or process the product enters, whose trust must be earned, whose real needs may differ from the needs that have been articulated in the commercial process. + +These two can be the same. An internal team building a platform for the organisation's own use is both the engineering team's customer and its business sponsor. In that case the alignment is direct, though it comes with its own challenges — the business sponsor may optimise for commercial deliverables rather than operational outcomes, and the distinction between what was commissioned and what is needed can still blur. + +But in most externally delivered programmes, the business and the customer are different, and the difference matters. The business has optimised the requirements for commercial reasons — for what could be negotiated, for what could be fitted within the contractual scope, for what the sales team believed would win the contract. The customer has operational needs that the commercial process may have captured imperfectly. + +When engineering teams are insulated from the customer — when the only information they receive about what is needed flows through the business's interpretation of the customer's requirements — they make decisions based on a mediated version of the customer's need. Requirements gaps that the customer would immediately identify are invisible to engineers who have never seen the product being used. Trade-off decisions that would obviously affect the customer's ability to use the product are made without the customer context that would inform them correctly. Quality compromises are accepted because the engineer does not know which quality properties the customer depends on most. + +The consequences appear at acceptance, at integration, or in the field. The product meets the specification. The specification did not fully capture what the customer needed. The gap is real and expensive regardless of who is technically correct about what was contracted. + +--- + +## What happens when engineers see the customer + +The quality improvement that results from engineering teams having direct visibility of end customers is not a motivational effect, though motivation may improve. It is an informational effect. Engineers who have seen how the customer actually uses the product write different requirements, make different trade-off decisions, and accept different quality compromises from those who have not. + +The engineer who has watched a customer interact with the interface — who has seen the confusion, the workarounds, the operational context that shapes how the product is actually used — does not make the same assumptions about what is acceptable as the engineer who has only read a requirements document. The architect who understands the operational environment in which the system will run — the network conditions, the failure modes, the recovery procedures, the support capabilities — does not make the same architectural choices about resilience and diagnosability as the architect who has only seen a high-level deployment diagram. + +When trade-offs must be made — and trade-offs always must be made — the engineer with customer visibility makes them in the customer's interest. The feature that must be descoped is the one the customer can live without, not the one that is easiest to cut. The quality compromise that is accepted is the one that affects the customer least, not the one that is most convenient to the delivery schedule. The technical shortcut that is taken preserves the properties the customer depends on, rather than sacrificing them for development speed. + +This is not idealism. It is the practical consequence of having the right information at the point of decision. Engineers insulated from customer reality are not making better trade-off decisions than engineers with customer visibility. They are making worse ones, with better intentions — because the information required to make them correctly is not present. + +Firmitas requires that engineering teams maintain a direct line of sight to the end customer — not bypassing the business relationship, but ensuring that customer reality informs the specification, the design, and the trade-off decisions throughout the programme. Customer involvement in requirements reviews. User research that precedes specification writing. Validation activities that test requirements against real customer scenarios. Feedback loops from operational deployment that inform the next iteration. + +The customer line of sight does not require constant customer access. It requires that the engineering team's understanding of the customer's real need is current, direct, and not entirely mediated by a commercial function optimising for a different set of objectives. + +--- + +## Engineering decisions are economic decisions + +Every significant engineering decision carries economic consequences — in development cost, in operational cost, in the cost of future change, in the cost of managing the risk that the decision creates. These consequences exist whether or not they are explicitly calculated. + +The architectural coupling decision that makes future change expensive has an economic consequence. The quality shortcut that increases the defect rate has an economic consequence — paid in support cost, rework cost, and customer impact. The technical debt that is accepted without a resolution plan has an economic consequence — paid in the increasing cost of development that compounds as the debt accumulates. The test coverage that is insufficient has an economic consequence — paid when the defects that coverage would have found surface in the field. + +When engineers do not have visibility of these economic consequences, they cannot make engineering decisions that account for them. They make technically correct decisions in a commercial vacuum — choosing the architecturally elegant solution when the economically efficient one was the right answer, or accepting the technically convenient shortcut when the economically defensible approach required more upfront investment. + +The governance principle follows: engineering decisions that carry significant economic consequences should be evaluated against those consequences explicitly. Not to subordinate technical judgement to financial optimisation, but to ensure that the people making technical decisions understand what they are accepting economically, and that the people making commercial decisions understand what the engineering implications of their choices are. + +This requires a vocabulary for expressing economic consequences in terms that governance layers can evaluate. Cost of delay — what the programme loses for every week a decision is deferred. Cost of change — what it will cost to reverse a decision once it has been implemented and built upon. Cost of quality failure — what it will cost when a quality shortcut surfaces as a defect, an incident, or a customer relationship problem. These are not precise calculations. They are order-of-magnitude assessments that allow the governance layer to compare the cost of doing something properly with the cost of not doing so — which is almost always the comparison that reveals the real economics of a shortcut. + +--- + +## The cost of quality is lower than the cost of poor quality + +The argument that investing in quality is expensive is consistently true in the short term and consistently wrong over any meaningful time horizon. Quality investment costs time and capacity upfront — in requirements work, in architectural discipline, in test coverage, in documentation discipline, in the governance that ensures quality standards are maintained under delivery pressure. These costs are visible immediately and attributable directly to the quality investment. + +The cost of poor quality is deferred and distributed. It appears in defects found late, in rework that consumes the capacity that should have been available for new delivery, in field incidents that damage customer relationships, in the technical debt that makes future delivery progressively more expensive. These costs are rarely attributed to the quality decisions that produced them. They are attributed to execution failure, to engineering incompetence, to bad luck — anything except the governance decision months or years earlier that traded quality for apparent delivery pace. + +The return on quality investment is therefore invisible in the short term in the same way that slack's value is invisible — it appears as the absence of costs that would otherwise have been incurred. The organisation that invests properly in requirements engineering, architectural discipline, and continuous verification does not experience the integration failures, the acceptance test crises, and the field incidents that the organisation that cuts these activities does. The absence of those costs does not appear on a budget. The savings are not attributed to the quality investment that produced them. + +This invisibility is the structural reason why quality investment is consistently underfunded. The cost is visible. The return is not. The governance response — protecting quality investment from schedule pressure, treating quality shortcuts as economic decisions with deferred costs rather than as delivery accelerations — requires the understanding and the discipline to manage the economy of the programme across its full lifecycle rather than within the current sprint. + +--- + +## Business strategy as an upstream constraint + +Engineering programmes do not exist in a strategic vacuum. They are investments made in service of a business strategy — a set of choices about where the organisation will compete, what capabilities it will build, what customers it will serve, and what it will be in the market over the next several years. + +When engineering programmes are aligned to business strategy — when the capabilities being built are the ones the strategy requires, when the architectural decisions that the programme makes are compatible with the direction the business intends to evolve — the engineering investment compounds. Each programme builds on the previous ones. The technical capabilities developed in one programme become the foundation for the next. The architectural decisions that the strategy required, made consistently across programmes, produce a system that can support the strategy's evolution rather than constraining it. + +When engineering programmes are misaligned with business strategy — when they build capabilities that the strategy has moved past, when their architectural decisions embed assumptions that the business's direction has invalidated, when the investment is made in things that will not matter in three years — the engineering investment does not compound. It is written off. The capability that was delivered is not used in the way its specification intended. The architecture that was built does not support the direction the business actually took. + +This misalignment is usually not visible until it is expensive. The programme that was funded against a strategy that has since changed does not announce itself as misaligned. It continues to deliver against its specification while the specification becomes progressively less relevant to the business's actual direction. The cost surfaces when the programme closes and the capability it has delivered requires significant rework before it is usable in the context the business now needs. + +The governance response is alignment — ensuring that engineering programmes are funded, scoped, and governed in the context of the business strategy they are supposed to serve, and that changes in strategy are reflected in the portfolio's prioritisation before they are reflected in the programme's consequences. The portfolio review that asks not only "is this programme on track?" but "is this programme still building the right thing?" is connecting engineering investment to business strategy in the way that produces compounding returns rather than compounding misalignment. + +--- + +## Metrics that reveal alignment + +The measurement of engineering alignment with business outcomes is one of the most consistently mishandled governance activities in engineering programme management. The default metrics — delivery velocity, feature count, milestone completion, defect rates — measure engineering activity. They do not measure whether the engineering activity is producing the business outcomes it was intended to produce. + +The metrics that reveal alignment are outcome metrics — measures of whether what engineering has delivered is producing the value the investment was supposed to generate. Customer adoption rates. Operational efficiency improvements. Reduction in support burden. Revenue enabled by the delivered capability. These metrics are harder to collect than activity metrics, noisier, and more ambiguous in their attribution. They are also the only metrics that answer the question that justifies the investment. + +Activity metrics are not useless. They reveal whether engineering is performing internally — whether teams are productive, whether quality standards are being maintained, whether delivery commitments are being met. But they do not reveal whether the internal performance is producing external value. A programme that delivers reliably against its specification is not necessarily delivering value — if the specification does not reflect the customer need, the reliable delivery is reliably wrong. + +The governance principle: outcome metrics and activity metrics serve different governance purposes and must not substitute for each other. Activity metrics govern engineering performance. Outcome metrics govern engineering alignment. Both are necessary. Treating activity metrics as if they reveal alignment, or treating outcome metrics as if they reveal engineering performance, produces the wrong governance conversation at every level of the programme and portfolio. + +--- + +## Funding models and flow + +The relationship between how engineering programmes are funded and how effectively they deliver is rarely discussed explicitly. It is consequential. + +Project-based funding — the allocation of a fixed budget to deliver a defined scope by a defined date — creates the incentive structures that produce many of the failure patterns described in this book. The fixed budget creates pressure to underestimate costs to win funding. The defined scope creates pressure to lock requirements before they are adequately understood, because the scope must be fixed to determine the budget. The defined date creates the committed date problem that Chapters 3 and 9 described in detail. + +Product-based or value-stream funding — the allocation of ongoing capacity to a team responsible for a long-lived capability — creates different incentive structures. The team is funded to produce outcomes, not to deliver a scope. The investment decision is made based on the value the capability is expected to generate, not on the cost of a defined scope. The scope can evolve as understanding improves without requiring a change request process that treats evolution as a failure of upfront planning. + +Not all programmes benefit from product-based funding. Some genuine one-time investments — building a specific regulated product for a defined customer, delivering a specific contracted scope under a fixed-price agreement — are appropriately funded as projects. The governance question is not "should everything be funded as a product?" but "which investments are genuinely one-time and which are ongoing commitments to capability that would benefit from funding continuity?" + +The organisation that defaults to project funding for everything, including the capabilities it will maintain and evolve indefinitely, is creating the discontinuity and knowledge loss at every project close that Chapter 20's value stream argument described. The organisation that defaults to product funding for everything, including genuinely one-time investments, is creating the accountability ambiguity of open-ended commitments without defined completion criteria. The governance skill is in matching the funding model to the nature of the investment. + +--- + +## The alignment principle + +Business alignment, economics, and customer visibility are not three separate concerns. They are three dimensions of the same alignment — the alignment between what engineering is doing, why it is doing it, and whether it is producing the outcomes that justify the investment. + +Engineering aligned with its customers produces work that meets real needs rather than specified requirements. Engineering aligned with its business economics makes decisions with full awareness of their commercial consequences. Engineering aligned with business strategy builds capabilities that compound in value rather than depreciate through misalignment. + +The failure to align in any one of these dimensions produces a specific and predictable consequence. Engineering insulated from customers produces specification failures that are invisible until acceptance or the field. Engineering insulated from economics produces technical decisions that are commercially unsustainable. Engineering insulated from strategy produces investment that is strategically irrelevant before it is operationally mature. + +The governance response to each is the same: bring the relevant context into the engineering decision-making process, at the right level of detail, through the appropriate channels. Not as a substitute for engineering judgement — but as the information that engineering judgement requires to be exercised well. + +Engineering that knows its customers, understands its economics, and is connected to its strategy is not constrained by commercial reality. It is enabled by it. The constraints are the guardrails within which the engineering work produces outcomes rather than just outputs. The context is the information that makes trade-offs intelligible rather than arbitrary. + +That is what alignment means. And that is what it produces. diff --git a/content/firmitas/part-2-the-framework/23-ai-automation-and-human-accountability.md b/content/firmitas/part-2-the-framework/23-ai-automation-and-human-accountability.md new file mode 100644 index 0000000..cd8adf0 --- /dev/null +++ b/content/firmitas/part-2-the-framework/23-ai-automation-and-human-accountability.md @@ -0,0 +1,139 @@ +--- +title: "Chapter 23: AI, Automation, and Human Accountability" +--- + +# Chapter 23: AI, Automation, and Human Accountability + +Engineering work is changing. The tools available to engineers — for generating code, for producing documentation, for analysing requirements, for designing tests, for reviewing architecture — are developing at a pace that is genuinely transforming the cost of certain activities that previously consumed substantial human effort. A requirements document that took a senior engineer two weeks to produce can now be drafted in hours. Test cases that required careful manual construction can be generated against a specification automatically. Code that implemented a well-specified behaviour can be produced by a tool that did not exist three years ago. + +This change is real. Its consequences for engineering delivery are significant. They are not, however, the consequences that the most enthusiastic accounts suggest. The collapse of implementation cost does not eliminate the cost of engineering. It redistributes it — from the cost of producing artefacts to the cost of specifying them correctly, verifying them rigorously, and maintaining accountability for the outcomes they produce. The engineering discipline that matters most in an AI-assisted environment is not the discipline of implementation. It is the discipline of intent, specification, and verification. + +This chapter addresses AI and automation in the context of the Firmitas framework — not as a special case requiring a separate framework, but as a set of capabilities that amplify the properties of the engineering systems they operate within. AI accelerates what already works. It magnifies what is already broken. The governance implications are specific, practical, and grounded in the same principles that govern every other aspect of engineering delivery. + +--- + +## AI as an accelerator, not a replacement + +The starting position is simple and important: AI tooling is an accelerator, not a replacement for engineering judgement, engineering accountability, or the human understanding that gives engineering work its meaning and its consequences. + +An AI tool that generates a requirements document has produced a hypothesis about what the system should do. The hypothesis may be well-formed. It may be coherent. It may even be largely correct. It is not a specification until a human being with the knowledge to evaluate it has reviewed it, challenged it where it is wrong, confirmed it where it is right, and accepted accountability for the claim it makes about what the system should do. + +An AI tool that generates test cases has produced a set of test hypotheses against the specification it was given. Those tests may be correctly derived from the specification. They may also reflect the specification's ambiguities — testing what was written rather than what was meant. They may share the blind spots of the implementation they are testing, because both were generated from the same specification by processes that make the same assumptions. A human engineer reviewing those tests must ask not only whether they pass but whether they are asking the right questions. + +An AI tool that generates code has produced an implementation of a specification. The code may be functionally correct. It may be architecturally inappropriate for the system it is being integrated into. It may violate the quality standards, the security requirements, or the performance constraints that the specification did not explicitly state but that anyone with operational knowledge of the context would have applied. A human engineer reviewing that code must bring the contextual knowledge that the tool does not have. + +In each case, the AI produces the first draft. The human provides the judgement that converts the first draft into a specification, a test suite, or an implementation that the programme can build on with confidence. The AI saves the time to first draft. It does not save the time or the cognitive investment required to produce something that is genuinely fit for purpose. + +--- + +## Where AI accelerates across the delivery lifecycle + +AI tooling provides genuine acceleration at specific points in the delivery lifecycle. Understanding where the acceleration is real allows organisations to capture it. Understanding where it is illusory prevents them from accepting false confidence in its place. + +**In the structured foundations phase**, AI tooling can accelerate first-draft requirements production from specification documents, standards, or customer statements of need. It can identify gaps and ambiguities in existing requirements by cross-referencing against known patterns and standards. It can generate first-draft Interface Control Documents from architectural descriptions. It can produce initial risk assessments from system descriptions. + +The verification obligation at this stage is high. AI-generated requirements are hypotheses about what the system should do — and they are as good as the information they were generated from. Requirements generated from an ambiguous customer statement will be ambiguous. Requirements generated from an incomplete specification will be incomplete. The review process for AI-generated requirements must be as rigorous as the process for human-generated ones — arguably more so, because the speed of generation can create false confidence in the quality of the output. + +**In the iterative development core**, AI tooling can accelerate code generation for well-specified behaviours, documentation of decisions and interfaces, generation of test cases from requirements, and analysis of change impact across requirements and architecture. + +The specific risk in this phase is the AI-generated test problem. Tests generated automatically from a specification may correctly reflect the specification as written while missing the intent behind it. They may produce high code coverage with low confidence — because they test the implementation's behaviour rather than the customer's need. A test suite generated by AI should not be reviewed only for correctness against the specification. It should be reviewed for whether it is asking the questions that matter — whether the test cases that would catch the failures the customer would care about most are present and adequately specified. + +**In the validation phase**, AI tooling can accelerate traceability matrix generation, test result analysis, and compliance evidence compilation. The verification obligation here is the highest in the programme — because the evidence produced in this phase is the basis for the programme's quality and compliance claims. AI-generated traceability matrices must be verified for correctness and completeness before they are used as compliance evidence. A traceability matrix that looks complete but contains incorrect or missing links is not compliance evidence. It is the appearance of compliance evidence, which is more dangerous than an acknowledged gap. + +**In operations**, AI tooling can accelerate incident analysis, knowledge base generation from operational data, and documentation maintenance. The governance principle here is that operational decisions — decisions about what to do in response to an operational incident, decisions about whether a system behaviour is acceptable, decisions about whether to deploy a change — must remain in human hands. AI can surface patterns, suggest diagnoses, and recommend responses. The decision to act on those recommendations must be made by a human who accepts accountability for the outcome. + +--- + +## Specification as the primary engineering discipline + +The collapse of implementation cost has a specific implication that the Firmitas framework addresses directly: as implementation becomes cheaper and faster, specification becomes more important, not less. + +When implementation is expensive, poorly specified requirements are caught relatively quickly by the cost of implementing them wrong — the developer who cannot make progress against an ambiguous requirement surfaces the ambiguity early. When implementation is cheap, poorly specified requirements are implemented rapidly and incorrectly — the AI tool fills the ambiguity with a plausible interpretation, produces an implementation, and delivers it faster than the human review process can evaluate it. The ambiguity has not been resolved. It has been embedded in a fast-delivered implementation that appears complete. + +The engineering discipline that scales with AI-accelerated delivery is specification — the ability to express intent with sufficient precision, completeness, and verifiability that the tools operating on it produce what was actually needed rather than what could plausibly be inferred. Chapter 11's requirements engineering discipline becomes more critical, not less, in an AI-assisted environment. The Requirement Breakdown Structure, the properties of a well-written requirement, the testability criteria, the acceptance condition precision — these are the properties that determine whether AI-assisted delivery produces the right system or a rapidly produced version of a misunderstood specification. + +The organisation that responds to AI-assisted delivery by relaxing its requirements discipline — by treating the AI's ability to fill gaps as a substitute for specifying what should be in those gaps — will discover the consequences at validation and in the field. The gaps are not filled by the AI's judgement. They are filled by the AI's most plausible inference from incomplete information. The result is a system that is implemented consistently with the inference rather than with the intent. The inference may be close. It is not the same thing. + +--- + +## Verification must tighten as generation accelerates + +The relationship between the speed of generation and the rigour of verification is inverse, not proportional. When generation is slow, verification happens continuously — the human producing the artefact is also verifying it as they produce it, catching errors as they introduce them. When generation is fast, the artefact arrives before the human reviewer is ready for it, and the social pressure to accept the output — because it looks complete and was delivered quickly — creates the conditions for superficial review. + +Firmitas requires that verification depth is calibrated to consequence, not to generation speed. An AI-generated artefact that forms part of a safety case requires the same verification rigour as a human-generated one — the safety consequence is the same regardless of how the artefact was produced. An AI-generated test suite requires independent evaluation of whether it is asking the right questions, not just whether it passes. An AI-generated traceability matrix requires verification of every link, not confirmation that links exist. + +The specific risk to manage is automation bias — the human tendency to trust outputs from automated systems more readily than outputs from other humans. An AI-generated requirements document that is well-formatted, comprehensive in appearance, and internally consistent will tend to pass review more quickly than a human-generated document with the same quality of underlying content, because the formatting and apparent completeness create a cognitive signal of rigour that the reviewer responds to. The review process must be designed to counteract this bias — to evaluate the substance rather than the presentation, to test the requirements against customer reality rather than against internal consistency, and to ask explicitly whether the gaps that the AI may have filled plausibly have been filled correctly. + +--- + +## Provenance and traceability in AI-assisted delivery + +When AI tools contribute to engineering artefacts — generating requirements, producing test cases, drafting design rationale, creating documentation — the provenance of those contributions must be visible. Not for blame purposes. For review purposes. + +An artefact produced by a human engineer with deep domain knowledge carries different confidence than an artefact generated by an AI tool from a specification prompt. Both may be correct. The review process that is appropriate for the former may be insufficient for the latter. If the provenance is not recorded, the reviewer cannot calibrate the review depth appropriately. + +In regulated programmes, the provenance question has compliance implications. The regulatory frameworks that govern safety-critical and quality-managed engineering — IEC 62304, IEC 61508, ISO 9001, DO-178C — were written before AI-generated artefacts were a practical reality. Their requirements for controlled development, documented rationale, and independent review must be interpreted in the context of AI assistance. The interpretation that best serves the standards' intent is conservative: AI-generated artefacts must be reviewed with at least the same rigour as human-generated ones, the AI's contribution must be traceable, and the human who accepts the artefact into the programme's evidence base is accountable for its correctness. + +The traceability spine described in Chapter 14 applies with full force to AI-assisted delivery. Every requirement traces to its source — including whether it was generated by AI and reviewed by whom. Every test case traces to the acceptance criteria it was derived from and the method by which it was produced. Every design decision traces to the architectural reasoning that justified it and the review process that validated it. The AI acceleration does not remove the traceability obligation. It adds the requirement to record the AI's contribution as part of the traceability record. + +--- + +## Accountability cannot be automated away + +The most fundamental governance principle in AI-assisted engineering delivery is stated in Principle 5 and applies with particular force here: leadership owns the system, not the intention. + +No automated system is accountable. A model does not accept responsibility for the consequences of the outputs it produces. A pipeline does not bear the cost of the defect it failed to catch. A tool does not absorb the commercial and reputational impact of the compliance failure it contributed to. All of these consequences belong to the humans who designed the system, who operated it, and who accepted the outputs it produced. + +This is not a limitation of current AI capability that future AI will overcome. It is a structural property of what accountability means. Accountability is the acceptance of responsibility for outcomes — including the outcomes of decisions made under uncertainty, with incomplete information, in conditions that could not be fully anticipated. It requires a subject who can learn from failure, who can be held to the consequences of decisions, and whose future behaviour is shaped by those consequences. Accountability belongs to humans. It cannot be delegated to tools, regardless of their capability. + +The practical implications are specific. + +Every AI-assisted decision has a named human owner who is accountable for the outcome. Not the team that used the tool. Not the organisation that deployed it. A specific person who reviewed the AI's output, exercised judgement about its adequacy, and accepted it into the programme on behalf of the organisation. That person's name is attached to the decision in the programme's decision record. + +Every automated pipeline that makes decisions without human review — that deploys code, that modifies requirements, that updates a traceability matrix, that accepts test results — has a named human owner who is accountable for the pipeline's behaviour. When the pipeline produces an incorrect output, the accountability belongs to the person who designed the pipeline, not to the pipeline itself. + +Human override mechanisms are present and functional. The ability to stop an automated process, to reverse an AI-assisted decision, and to require human review at any point must be designed into the system and must be genuinely available — not technically present but practically impossible to exercise under production conditions or delivery pressure. + +--- + +## AI governance in regulated programmes + +For programmes operating in regulated environments, AI governance carries specific obligations that the framework must address. + +The first is the question of qualification. Most regulated frameworks require that the tools used in safety-critical or quality-managed development are qualified — demonstrated to be fit for their intended use within the programme's defined scope. AI tools introduce new qualification challenges because their behaviour is probabilistic, context-dependent, and not fully deterministic. A tool that generates requirements correctly 95% of the time is not qualified for safety-critical requirements engineering without a rigorous process for identifying and correcting the 5% of cases where it does not. + +The qualification obligation does not mean that AI tools cannot be used in regulated programmes. It means that the programmes using them must define the scope of their use, establish the verification processes that compensate for their probabilistic behaviour, and demonstrate that the combination of AI assistance and human verification produces outputs that meet the quality and safety obligations of the applicable standard. + +The second is the question of training data and knowledge sources. AI tools encode their training data. A tool trained on publicly available engineering documentation may have absorbed incorrect, outdated, or context-inappropriate information that it applies to the programme's specific domain. The governance obligation is to understand what the tool knows, where its knowledge comes from, and whether the domain-specific context of the programme is adequately represented in its training — or whether the tool's outputs require supplementary review by domain experts who can identify where the tool's general knowledge does not apply correctly to the specific context. + +The third is the question of auditability. Regulatory bodies increasingly expect to understand how engineering artefacts were produced. An audit trail that records what AI tools were used, what inputs they were given, what outputs they produced, and who reviewed and accepted those outputs is the minimum required for programmes in regulated environments. The audit trail for AI-assisted development is more complex than the trail for purely human development — because the trail must record not just what was done but how the AI's contribution was supervised and verified. + +--- + +## The human skills that become more valuable + +The shift in engineering work produced by AI acceleration does not diminish the value of engineering expertise. It changes which expertise matters most. + +The skills that AI tools replicate most effectively are the skills of execution — producing well-formed outputs from well-specified inputs, following established patterns, generating consistent implementations of understood requirements. The skills that AI tools do not replicate are the skills of judgement — understanding which requirements matter most and why, recognising when a specification is missing something critical, evaluating whether an architecture will hold up under operational conditions the specification did not anticipate, deciding when a trade-off is acceptable and when it is not. + +Specification quality — the ability to express intent precisely, completely, and verifiably — becomes more valuable as implementation becomes cheaper. The engineer who can write requirements that an AI tool can implement correctly the first time is more productive in an AI-assisted environment than the engineer who writes requirements that the AI must interpret charitably. + +Systems thinking — the ability to understand how engineering decisions interact with each other and with the broader context of the programme — becomes more valuable as the volume of engineering artefacts increases. The engineer who can evaluate whether an AI-generated component fits coherently into the system architecture, whether its quality properties are consistent with the programme's requirements, and whether its assumptions are compatible with the operational environment it will enter is the engineer whose judgement the programme depends on. + +Risk reasoning — the ability to identify what could go wrong, why, and what the consequences would be — becomes more valuable as the rate of change increases. The engineer who can look at an AI-generated system and identify the failure modes that the AI did not model, the edge cases that the specification did not cover, and the operational conditions that the design did not anticipate is the engineer who protects the programme from the confidence that AI-assisted delivery can inadvertently create. + +These are not new skills. They are the skills that have always distinguished good engineering from mere implementation. AI assistance makes them relatively more important — because the implementation work that previously demanded a substantial portion of engineering time becomes faster, and the time freed by that acceleration is most productively invested in the specification, verification, and systems reasoning that the tools cannot replicate. + +--- + +## What does not change + +The Firmitas principles do not change in an AI-assisted environment. They become more relevant. + +Principle 3 — a commitment not made by those who must keep it is a target with someone else's name on it — applies to AI-assisted programmes as directly as to any other. The AI does not make commitments. It produces artefacts. The humans who accept those artefacts and build programmes on them make the commitments and carry the accountability. + +Principle 4 — delivery knowledge lives closest to the work — applies with greater force when AI tools can produce artefacts rapidly from high-level inputs. The knowledge that determines whether the AI's output is adequate, whether the specification is complete, whether the test suite is rigorous enough — that knowledge lives in the engineers who understand the domain, the operational context, and the customer need. It cannot be replaced by the tool that generates first drafts against those inputs. + +Principle 9 — decisions are first-class engineering artefacts — applies to AI-assisted decisions as fully as to human ones. A decision to accept an AI-generated requirements document without substantive review is a decision with consequences. A decision to deploy AI-generated code without adequate testing is a decision with consequences. These decisions must be visible, owned, and traceable — not hidden inside the automation. + +Principle 12 — your people are your most significant investment — applies with particular urgency in a period when AI-assisted delivery creates pressure to reduce headcount in the belief that tools have replaced the need for people. The tools have not replaced the need for people. They have changed which people and which skills matter most. The organisation that responds to AI acceleration by reducing the engineering investment that produces specification quality, systems thinking, and risk reasoning capability will discover, when its AI-assisted programmes fail in the ways this chapter has described, that the tools did not replace the judgement that the people they let go had been providing. diff --git a/content/firmitas/part-2-the-framework/index.md b/content/firmitas/part-2-the-framework/index.md new file mode 100644 index 0000000..7e38c14 --- /dev/null +++ b/content/firmitas/part-2-the-framework/index.md @@ -0,0 +1,77 @@ +--- +title: "Part Two: The Framework" +--- + +# Part Two: The Framework + +Part One described a failure. Specifically, the failure of a system to act on accurate information - to convert what engineers knew into decisions that could have changed the outcome. + +Part Two describes the structural response to that failure. + +Every chapter that follows is connected to something Part One named. The governance model addresses the programme board that noted the risk register and moved on. The estimation approach addresses the committed date that arrived before the scope was understood. The risk and issues treatment addresses the three red items that required decisions and did not receive them. The delivery model addresses the feedback loops that were too long and the integration that came too late. The people and capability chapters address the engineers whose judgement was overridden and whose knowledge was never fully used. + +None of these elements exist in isolation. They are components of a coherent framework built on an explicit philosophy. Understanding the relationship between the philosophy, the framework, and the process choices within it is the prerequisite for using what follows correctly - and for recognising when it is being misused. + +That relationship is described below, before the framework chapters begin. + +--- + +**The Layered Model** + +Firmitas operates in three distinct but interdependent layers. The ordering of these layers is not a stylistic choice. It is the structural logic that separates Firmitas from every framework that has been adopted, applied mechanically, and produced compliance theatre rather than changed outcomes. + +**The philosophy layer** + +The philosophy layer defines what matters and why. It contains the non-negotiables - the structural positions about how engineering organisations work, where knowledge lives, what leadership means, and what commitments are. These positions do not change with context. They are the stable foundation on which everything else rests. + +The twelve principles in Chapter 6 are the philosophy layer. They constrain everything that follows. Any framework element or process choice that violates them is not Firmitas - regardless of how it is labelled or how closely it resembles the framework in structure. + +**The framework layer** + +The framework layer translates philosophy into repeatable structure. It defines the decision logic, the governance mechanisms, the lifecycle shape, the artefact types, and the role boundaries through which the philosophy is applied in practice. + +The chapters in Part Two describe the framework layer. The framework is stable - it does not change under delivery pressure or commercial convenience. It can evolve deliberately as understanding improves. It does not change casually. + +**The process layer** + +The process layer defines how work is actually executed in a specific context. It includes methodologies, tooling choices, ceremonies, cadences, and documentation formats. Process is intentionally variable - it should differ by product type, regulatory context, organisational maturity, and team composition. + +The only constraint on process is that it must respect the framework and honour the philosophy. A team that uses Kanban rather than Scrum, that writes requirements in structured natural language rather than user stories, that holds weekly risk reviews rather than fortnightly ones - all of this is valid process variation within the framework. A team that reverse-engineers its plan from a committed date, that treats the risk register as a reporting artefact, that defines done as code complete rather than customer-validated outcome - this is process that violates the philosophy, regardless of what methodology label it carries. + +**Why the ordering matters** + +The most consistent failure mode in framework adoption is inversion of the layers. Organisations install process - the artefacts, the ceremonies, the tooling - without engaging with the philosophy that should constrain it. The result is that the new process is absorbed by the existing system. The risk register template changes. The risk register behaviour does not. The estimation ceremony is introduced. The estimates are still compressed toward committed dates. The gate review is scheduled. The gate still does not generate decisions. + +This is not failure of the framework. It is failure to apply the framework - specifically, failure to start with the philosophy layer before building the framework layer, and failure to build the framework layer before varying the process layer. + +The chapters that follow assume the philosophy has been engaged with. Chapter 6 established the twelve principles. Every Part Two chapter builds on them. If a Part Two chapter appears to contradict a principle, the chapter is wrong. The principles do not bend. + +```mermaid +flowchart TD + PH["Philosophy layer\nThe twelve principles\nWhat matters and why\nNon-negotiable - does not vary"] + FR["Framework layer\nPart Two chapters\nDecision logic and structure\nStable - evolves deliberately"] + PR["Process layer\nContext-specific execution\nMethodologies, tooling, cadences\nIntentionally variable"] + + PH -->|"Constrains - sets\nnon-negotiables"| FR + FR -->|"Constrains - sets\ndecision boundaries"| PR + PR -.->|"Informs - surfaces\npractical reality"| FR + FR -.->|"Informs - tests\nphilosophical positions"| PH + + style PH fill:#E1F5EE,stroke:#0F6E56,color:#085041 + style FR fill:#E6F1FB,stroke:#185FA5,color:#0C447C + style PR fill:#EEEDFE,stroke:#534AB7,color:#3C3489 +``` + +The solid arrows show constraint - what must be honoured. The dotted arrows show information flow - the practical reality of delivery should inform how the framework evolves, and the experience of applying the framework should test and refine the philosophy over time. But information flows upward. Constraint flows downward. Process that violates the framework is not a signal to change the framework. It is a signal to change the process. + +--- + +## How to read Part Two + +Part Two is the majority of the book. It covers eighteen chapters across the full scope of engineering programme delivery - from leadership and governance through estimation, requirements, risk, quality, architecture, teams, and portfolio management to AI and automation. + +Each chapter is self-contained enough to be read independently if a specific element is most urgent. The connections back to Part One are present in each chapter as context, not as a prerequisite for understanding. + +The sequence of chapters is deliberate. The first six chapters - leadership, delivery knowledge, commitments, risk, requirements, and slack - establish the foundational elements. Everything that follows builds on them. A reader who is working through Part Two in sequence will find that each chapter assumes the preceding ones. + +For readers who are using Part Two as a reference rather than reading sequentially, Appendix A contains the twelve principles as a one-page orientation, and Appendix B contains the diagnostic, which will indicate which chapters are most relevant to a specific programme's current condition. diff --git a/content/firmitas/part-3-making-it-stick/24-the-leader-who-has-to-change.md b/content/firmitas/part-3-making-it-stick/24-the-leader-who-has-to-change.md new file mode 100644 index 0000000..7915f1e --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/24-the-leader-who-has-to-change.md @@ -0,0 +1,131 @@ +--- +title: "Chapter 24: The Leader Who Has to Change" +--- + +# Chapter 24: The Leader Who Has to Change + +This chapter is not addressed to the engineers who have been overridden. They know the problem. + +It is addressed to the leader who made the programme decisions described in Chapter 1. The one who confirmed the delivery date at the first programme board despite the risk register. The one who noted the three red items and moved to task status. The one who accepted the three-point estimate by extracting the best-case number and filing the conditions. The one who, when the programme failed eight months late, concluded that the engineering team had not delivered what they committed to. + +This leader is not the villain of the story. Part One was clear about that. They were behaving rationally in the systems they were in — responding to the commercial pressures, the governance cultures, the incentive structures, and the organisational norms that shaped what good leadership looked like in their context. The intent was never to produce a programme that failed. The outcome was produced anyway, by a system that the leader owned without recognising that ownership as the source of their accountability. + +This chapter is about what changing that system requires. Not from the system perspective — Part Two addressed that in full. From the personal leadership perspective. What specifically must change in how a leader thinks, decides, and behaves to produce different outcomes. What it costs. And what makes it so consistently difficult even for leaders who genuinely want to do better. + +--- + +## The diagnostic before the prescription + +Before addressing what needs to change, it is necessary to be honest about how the leader who most needs this chapter will encounter it. + +The leader who rubber-stamps gate reviews because the programme needs to proceed on schedule will not read this chapter and think "that is me." They will read it and think "that is the kind of leader I am trying to avoid being." The leader who applies pressure to estimates because they believe engineers are sandbagging will not recognise the sandbag assumption as their own belief. They will recognise it as a belief held by other people in their organisation. + +This is not hypocrisy. It is the natural human response to diagnostic material that describes a pattern. We see the pattern in others before we see it in ourselves. The specific failure behaviours described in this chapter are behaviours that feel, from the inside, like good leadership — commercial discipline, governance rigour, appropriate challenge of engineering conservatism, maintenance of accountability. They are experienced as their opposites only from the outside — as commercial pressure that overrides honest assessment, as governance theatre that protects the governance layer from accountability, as the undermining of the honest estimation that produced the estimate being challenged, and as accountability displaced onto people who held the accurate information and were overridden. + +The diagnostic exercise this chapter asks of the reader is therefore specific: not "do I recognise these patterns in other leaders I have worked with?" but "when I search for these patterns in my own behaviour, under the specific conditions of commercial and delivery pressure, what do I find?" + +That is a harder question. It requires accessing memories of specific decisions — not the principles the leader operates by but the decisions they made when the principles were under pressure. The programme board that confirmed the date. The estimate that was challenged. The risk that was noted. The status report that was accepted without the operational knowledge to evaluate it. + +--- + +## The specific behaviours that produce Chapter 1 + +The failure in Chapter 1 was not produced by a single dramatic leadership failure. It was produced by a sequence of governance decisions, each of which seemed reasonable in isolation and collectively produced a programme that failed predictably. + +**Accepting commitments made without adequate engineering input.** The date was set before the engineering assessment existed. The leader who accepted that date — or who allowed it to be set without requiring the engineering assessment first — made a commitment on behalf of people who had not been consulted. This is the commitment problem Chapter 3 described. The specific leadership behaviour that produces it is the acceptance of commercial commitments without the discipline to require that those commitments be grounded in honest engineering assessment before they are formalised. + +This behaviour feels like commercial pragmatism from the inside. The client needs a date. The business needs to win the contract. The delay required to produce an honest assessment would cost the opportunity. These are real pressures. They do not change what the behaviour produces: a programme that begins with a committed date that nobody has verified is achievable, handed to an engineering team as a fixed constraint within which they must find a way. + +**Treating the risk register as a reporting artefact.** The programme board reviewed the risk register, noted the three red items, requested mitigation plans, and moved to task status. The specific leadership behaviour that produced this outcome was the treatment of the risk review as an information receipt rather than a decision requirement — the implicit message that the appropriate response to a red item is to acknowledge it and ensure it is being managed, rather than to ask what decision is required from the governance layer and make that decision. + +This behaviour feels like appropriate delegation from the inside. The risk register has been reviewed. The risks are known. The programme management layer is managing them. The board's role is to be informed. But the risks in Chapter 1 were not manageable at the programme management level. They required decisions — client conversations, procurement approvals, independent validations — that were above the programme management layer's authority. The governance layer's failure was not ignorance of the risks. It was the failure to recognise that knowing about a risk is not the same as managing it, and that managing it required a decision that only the governance layer could make. + +**Applying pressure to estimates rather than accepting conditions.** When the engineering team produced a most likely estimate of seventeen months against a committed date of fourteen months, the governance response was to challenge the estimate rather than to accept the conditions attached to it. The specific leadership behaviour is the treatment of the gap between commitment and honest estimate as an estimation problem — requiring the engineering team to find a way to close the gap — rather than as a commitment risk that requires governance action to address the conditions. + +This behaviour feels like appropriate commercial accountability from the inside. The contract says fourteen months. The estimate says seventeen. Someone needs to find three months of efficiency. That is the engineering team's job. But the three months were not hidden in the estimate. The estimate was honest. The three months represented the risk that would materialise if the conditions attached to the estimate were not met. Pressuring the estimate to close the gap did not change the conditions. It made the conditions invisible — removing the honest signal from the governance process and proceeding on a fiction. + +**Governing through task status rather than risks and issues.** After the risk register was noted, the programme board review moved to task status — the percentage of tasks complete, the milestone progress, the RAG dashboard. The specific leadership behaviour is the choice of information source — selecting the information that provides reassurance over the information that requires decisions. + +Task status provides the appearance of control. It tells the governance layer that things are happening, that progress is being made, that the programme is moving. It does not tell the governance layer whether the programme is moving in a direction that will produce the outcome it is committed to. The risks that will determine that outcome are not visible in task status. They are visible in the risk register — which the governance layer reviewed, noted, and set aside in favour of the more comfortable information. + +**Displacing accountability when programmes fail.** When the programme missed its delivery date by eight months, the lessons-learned exercise concluded that the engineering team had not delivered what they committed to. The specific leadership behaviour is the attribution of failure to execution rather than to governance — the conclusion that the people who held accurate information and were overridden are accountable for the outcome, rather than the people who made the governance decisions that determined it. + +This behaviour is not conscious dishonesty. The attribution is genuinely believed. The programme had a committed date. The engineering team delivered eight months after it. The engineering team did not deliver what they committed to. The logic is internally consistent. What it omits is the chain of governance decisions that determined the outcome — the commitment made without engineering input, the risk register noted rather than acted on, the conditions stripped from the estimate, the honest signal suppressed in favour of the comfortable one. + +--- + +## The authority-experience gap — what it requires of leaders + +Chapter 2 described the authority-experience gap — the structural condition in which leaders with significant programme authority may not have the direct experience of delivery that would allow them to recognise the difference between an honest estimate and a conservative one, between a risk that is being managed and a risk that has been documented, between a governance process that is functioning and one that is performing. + +This is not a reason for leaders without delivery experience to defer entirely to engineers on all matters — commercial context, portfolio priorities, and strategic alignment are genuinely the governance layer's responsibility. It is a reason for leaders without delivery experience to develop specific compensating behaviours that substitute for the experiential knowledge they do not have. + +**Asking questions rather than asserting conclusions.** The leader who does not have the operational knowledge to evaluate whether an estimate is honest or conservative cannot improve their evaluation by asserting the conclusion. "This estimate is too conservative" is not a governance insight. It is a commercial preference presented as an engineering assessment. The question that produces better outcomes is: "Help me understand what makes this the most likely case rather than the best case. What conditions are you assuming about the risks attached to this estimate?" The question creates the space for the honest information. The assertion closes it. + +**Treating uncomfortable information as valuable.** The leader who responds to bad news with questions about why it was not caught earlier, or with pressure to find a way to solve the problem without the governance implication, has taught their team that bad news is dangerous. The leader who responds to bad news with "thank you for telling me early — what do you need from me to resolve this?" has taught their team that bad news is welcome. The difference in the information that subsequently flows is substantial and immediate. + +**Understanding what you are approving.** Gate reviews approved without adequate understanding of what is being approved are not governance. The leader who approves a gate because the programme management team has prepared a good presentation has performed governance theatre. The leader who asks "what would have to be wrong for this gate to not be ready?" — and who is genuinely prepared to receive the answer — is performing governance. + +**Distinguishing between process compliance and outcome achievement.** The process has been followed. The risk register has been reviewed. The gate criteria have been met. The programme is proceeding. This is not the same as the programme being in good health. The leader who conflates process compliance with outcome achievement will be surprised when a programme that followed the process produces the outcome Chapter 1 describes. The question is not "have the governance activities occurred?" It is "do the governance activities produce informed decisions that address the programme's real risks?" + +--- + +## What it costs to change + +The behaviours described above are not comfortable to change. They are embedded in governance cultures, reinforced by the incentive structures that determine what gets rewarded and what gets punished, and protected by the social norms that make certain conversations difficult and others impossible. + +Accepting a three-point estimate rather than challenging it requires the leader to have a different conversation with the client — a conversation about realistic timelines rather than the commercially convenient one. That conversation is uncomfortable. It risks the relationship. It may cost the contract. The leader who has that conversation is taking a risk that the leader who does not is avoiding. The programme in Chapter 1 represents the consequence of avoiding that risk — an eight-month overrun that cost the programme, the client relationship, and the business substantially more than the month-one conversation would have. + +Treating the risk register as a decision instrument rather than a reporting artefact requires the leader to make decisions in programme board meetings that they might prefer to defer. Decisions are uncomfortable. They can be wrong. They carry accountability. Deferral — asking for more information, requesting a mitigation plan, moving to task status — feels more prudent than a decision made on incomplete information. It is not more prudent. It is the systematic accumulation of deferred decisions that arrive as programme failures. + +Displacing accountability accurately — attributing failure to the governance decisions that produced it rather than to the execution of the people who held accurate information — requires the leader to examine their own governance behaviour as a cause of the outcome. This is uncomfortable in proportion to the degree of accountability it involves. The attribution to execution is easier. It locates the failure externally. It does not require the leader to change their own behaviour to prevent the next one. + +None of these changes are easy. They are, however, available. And they are what the difference between programmes that fail in the way Chapter 1 describes and programmes that do not actually looks like in practice. + +--- + +## The political reality + +Chapter 5 described the political obstacle most commonly encountered in organisations attempting to change delivery governance: not a named person resisting change, but a system — process as power, plan as authority — that has been designed around the assumption that change is failure and that control produces outcomes. + +The leader who wants to change their governance behaviour in an organisation whose culture has not changed will encounter this system. The decision to accept a realistic estimate in an organisation that treats estimate challenge as commercial discipline will be visible as a deviation from the norm. The decision to make decisions on escalated risks rather than request mitigation plans will be visible as a change in governance behaviour that challenges implicit cultural expectations about what programme board members do. + +This is not a reason not to change. It is a reason to change with awareness of the political context. + +The most effective path through the political obstacle is the one that produces visible results before the politics can close it. A governance behaviour that generates decisions rather than notes, and that prevents a programme failure that the previous behaviour would have produced, makes the political argument for itself. Organisations change their delivery governance norms when they see a different approach produce better outcomes — not when they are presented with a framework that argues for different outcomes in principle. + +The practical implication: the first change of governance behaviour should be applied to the programme most likely to succeed with better governance, not to the programme most resistant to it. The first genuine risk register review that generates decisions rather than notes should be on the programme whose risks are most clearly actionable, not the one whose governance culture is most entrenched. Evidence of better outcomes is the most powerful political argument available. + +--- + +## The leader who is the problem + +Part One made the structural argument that failure is produced by systems rather than by individuals. This chapter has maintained that framing. But there is a specific case that requires direct treatment: the leader whose own behaviour — not the system they are in — is the primary source of the programme's dysfunction. + +The leader who creates fear rather than psychological safety. Who punishes honest risk escalation. Who treats the risk register as evidence of engineering failure rather than as a management instrument. Who makes decisions by overriding the people with the most relevant knowledge and then holds those people accountable for the consequences. Who confuses the appearance of control with the reality of good governance. + +This is not a character assessment. These are specific, observable behaviours with specific, predictable consequences. They produce the information environment described in Chapter 18 — one in which bad news travels upward only when it can no longer be avoided, in which risk registers reflect what the governance layer wants to see rather than what the programme needs them to know, in which estimates are calibrated to what will be accepted rather than to what the work requires. + +A leader who exhibits these behaviours and reads Firmitas will recognise the failure patterns. They may not recognise themselves in them — for the reasons described at the opening of this chapter. The diagnostic question is not "do these failure patterns occur in my organisation?" It is "when I am under commercial and delivery pressure, what do I do with information that is uncomfortable? How do I respond to an engineering team that tells me the estimate is not achievable? How do I respond when a programme board member raises a risk that requires a decision I would prefer not to make?" + +The answers to those questions, observed honestly across the specific decisions of the last several programmes, are the diagnosis. The framework provides the alternative. The change is a choice that only the leader can make. + +--- + +## What changing looks like in practice + +Structural change does not require dramatic announcements or comprehensive framework rollouts. It requires specific, consistent changes in specific, observable behaviours — the governance behaviours that determine what information the programme produces, how it is used, and what outcomes it generates. + +The programme board meeting changes. It starts with escalated risks and issues. It does not move on until the decisions those items require have been made or explicitly deferred with an owner and a date. Task status is reviewed after the risk and issues discussion, not before, and not as the primary governance instrument. + +The estimate review changes. The response to a three-point estimate is not a challenge to reduce the most likely case. It is an assessment of the conditions attached to the estimate and the decisions the governance layer must make to create the conditions for the best case to be achievable. + +The gate review changes. The question is not "is the gate pack complete and well-presented?" It is "what would have to be wrong for this gate to not be ready, and is there evidence that those things are not wrong?" + +The failure attribution changes. When a programme misses its commitment, the first question is not "why did the engineering team not deliver?" It is "which governance decisions shaped the conditions under which this programme was running, and what would different governance decisions have produced?" + +None of these are process changes. They are governance behaviour changes — specific, deliberate changes in how the leader uses their authority in the governance moments that determine whether the programme's honest information reaches the people who can act on it, and whether those people act on it. + +That is what change looks like. It is not a transformation programme. It is a decision, made repeatedly in the specific moments when it is uncomfortable, to govern differently from the way the system has always governed. + +It is available to every leader who is willing to look honestly at the specific decisions they have made and ask whether different decisions would have produced different outcomes. diff --git a/content/firmitas/part-3-making-it-stick/25-assessing-readiness-and-adopting-progressively.md b/content/firmitas/part-3-making-it-stick/25-assessing-readiness-and-adopting-progressively.md new file mode 100644 index 0000000..b2988a7 --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/25-assessing-readiness-and-adopting-progressively.md @@ -0,0 +1,279 @@ +--- +title: "Chapter 25: Assessing Readiness and Adopting Progressively" +--- + +# Chapter 25: Assessing Readiness and Adopting Progressively + +Firmitas is not installed. It is grown — incrementally, within real organisations, by real leaders who have competing priorities, imperfect information, and limited political capital. The organisations that have successfully changed their delivery governance have not done so through comprehensive framework rollouts or transformation programmes. They have done so through specific, visible changes in specific, observable behaviours, starting where the conditions for success are most favourable and building from there. + +This chapter provides two instruments for that process. The Firmitas Delivery Diagnostic — twenty-six questions that assess how a real programme compares to what the framework requires — and the progressive adoption model that guides how an organisation moves from where it is to where the framework asks it to be, without collapsing under the weight of trying to change everything simultaneously. + +Both instruments start from the same premise: the honest assessment of current conditions is more valuable than the aspiration toward ideal ones. An organisation that knows where it actually is has the information required to make deliberate progress. An organisation that believes it is further along than it is will make the mistakes that Chapter 5 described — drawing the wrong lessons, adding the wrong processes, and wondering why the outcomes do not improve. + +--- + +## Assessing organisational readiness + +Before attempting adoption, an organisation must honestly assess whether the conditions for adoption exist. Not whether leadership is enthusiastic — enthusiasm is a weak predictor of sustained change. Whether the systems the framework requires can be established, maintained, and protected in the organisational context that actually exists. + +Firmitas readiness is assessed across six dimensions that correspond to the six system types described in Chapter 4. + +**Human systems readiness.** Do people feel safe to surface honest information — about risks, about estimates, about the gap between the programme's official narrative and its operational reality? Is failure treated as learning or as blame assignment? Are engineering concerns heard and acted upon, or noted and bypassed? If the human system makes honesty expensive and comfortable reporting rational, the framework cannot function — because the framework depends on accurate information flowing to the governance layer, and that flow requires a human system that does not punish the people who produce honest information. + +**Organisational systems readiness.** Does decision authority align with delivery knowledge? Are the people making programme commitments the people with enough understanding of what the programme requires to make those commitments honestly? Are incentives structured around outcomes or around outputs and appearance? Can the organisation make portfolio prioritisation decisions based on current reality rather than the inertia of a previous plan? If the organisational system rewards the appearance of delivery over its reality, the framework's governance model will be adopted superficially — the artefacts will exist, but the decisions they are supposed to generate will not be made. + +**Socio-technical systems readiness.** Is the tooling capable of supporting the governance practices the framework requires? Can the risk register be maintained as a live instrument rather than a filing obligation? Can documentation be produced and maintained alongside code rather than as a separate governance exercise? Can traceability be established and maintained without requiring dedicated documentation teams? If the tools are designed to produce governance artefacts rather than to support governance decisions, the framework's documentation and traceability disciplines will produce compliance theatre rather than the living evidence the framework requires. + +**Product and delivery systems readiness.** Does the delivery system have the feedback loops that the iterative development core requires? Can the organisation validate continuously rather than deferring all validation to a late phase? Are batch sizes manageable? Are handoffs between teams designed to preserve rather than destroy context? If the delivery system has been designed for sequential, batch-heavy delivery, the iterative core will require structural changes that cannot be made without disrupting the delivery of programmes already in flight. + +**Technical and architectural systems readiness.** Does the existing architecture support the team autonomy and the integration cadence that the delivery model requires? Is technical debt visible and managed, or invisible and accumulating? Are architectural decisions documented and accessible? If the existing architecture enforces the coordination patterns that the framework is designed to eliminate, improving governance without addressing the architecture will produce teams that are governed better but still constrained by the structural conditions that make flow expensive. + +**Economic and business systems readiness.** Are engineering programmes evaluated on outcomes or on output and activity metrics? Is the business capable of having honest conversations about what is achievable within commercial constraints, or does the commercial culture require optimistic commitments regardless of evidence? Are the funding models compatible with the delivery approach the framework requires? If the economic system rewards commercial commitments made before scope is understood and penalises the honest acknowledgement of uncertainty, the framework's commitment and estimation disciplines will be adopted in form while being violated in substance. + +--- + +## Signals of low readiness + +Some conditions indicate that adoption will fail before it has produced enough evidence to survive the first wave of organisational resistance. These are not reasons to abandon adoption — but they are reasons to address the conditions before attempting adoption, or to choose an entry point that is insulated from them. + +Fear-driven cultures. When the primary motivation for producing governance artefacts is to avoid blame rather than to inform decisions, the artefacts will be designed to provide cover rather than to surface risks. The risk register will contain the risks that the governance layer is already aware of. The status report will describe the programme as it should be rather than as it is. The gate review will be prepared to pass rather than to assess readiness. + +Output-heavy performance metrics. When the primary measure of engineering performance is delivery throughput — features shipped, tasks completed, milestones hit — the framework's quality and estimation disciplines will be traded for delivery pace at every point of pressure. The definition of done that includes quality standards will be quietly revised. The three-point estimate that shows a realistic most likely case will be challenged until it reaches the committed date. The technical debt that should be entered in the register will be absorbed silently to preserve the velocity metric. + +Tool-led transformation narratives. When the organisational response to delivery problems is to introduce new tools — a new project management platform, a new risk register template, a new governance dashboard — without addressing the governance behaviours and cultural norms that determine how those tools are used, the new tools will be used in the same way as the old ones. The risk register will be produced in the new format and filed with the same consequences. + +Zero-slack delivery expectations. When every team is at full utilisation and every hour of capacity is pre-committed to delivery activity, there is no capacity for the foundations work, the verification discipline, and the governance attention that the framework requires. Adoption cannot occur in a system that has eliminated the slack that adoption requires. Creating the conditions for adoption requires protecting capacity for it — which requires a leadership decision that something else will not be started, continued, or resourced at the level it previously was. + +--- + +## The Firmitas Delivery Diagnostic + +The diagnostic that follows is designed to be uncomfortable to answer honestly. That discomfort is the point. Easy questions produce comfortable answers that reveal nothing. The questions here require specific evidence rather than general assessment. If the evidence is not available, the answer is no. + +**How to use this diagnostic** + +Answer yes or no to each question. If you hesitate, the answer is no. If you answer yes, the evidence check applies. If you cannot produce the evidence in five minutes, the answer reverts to no. + +Score one point for each confirmed yes with evidence. The score and the clustering of no answers will indicate both the overall readiness of the programme and which dimensions require the most urgent attention. + +--- + +### Dimension 1 — Commitments and Planning + +**Q1.** Was your current delivery date set by the people who have to meet it? + +*Evidence check: Name the person who set the date and the person accountable for delivering it. If they are different people, at what point was the delivery team's honest estimate compared against the committed date — and what happened when they differed?* + +**Q2.** Does your current plan include a worst case estimate with the specific risks that would cause it? + +*Evidence check: Find the worst case number in your current plan. Find the named risks used to derive it. Find the owner and current mitigation status of each risk today. If the worst case exists without attached risks, or the risks exist without owners, the answer is no.* + +**Q3.** When the last estimate your engineering team produced differed significantly from the committed date, was the estimate preserved in programme documentation alongside the committed date? + +*Evidence check: Find that estimate. Is it still visible in programme documentation or was it replaced by the committed date? If it was replaced, the programme is running on a date, not an estimate.* + +**Q4.** Has a programme gate ever stopped or fundamentally changed a programme in your organisation? + +*Evidence check: Name the programme, the gate, and the decision. If every gate in your organisation's history has resulted in proceed, your gates are not functioning as decision points.* + +**Q5.** When commercial or schedule pressure has been applied to your programme, has the risk register been formally updated to reflect the increased risk of the accelerated plan? + +*Evidence check: Find the most recent schedule compression decision on your programme. Find the corresponding risk register update. If the schedule changed but the risks did not, the risk register is not being maintained as a live instrument.* + +--- + +### Dimension 2 — Requirements + +**Q6.** Can you trace any requirement from the customer need that generated it, through system and subsystem levels, to the test that will prove it has been met? + +*Evidence check: Pick one requirement from the middle of your requirements document. Not a simple one. Not one you wrote. Trace it upward to the customer need and downward to the test case right now. If you cannot do it in five minutes the answer is no.* + +**Q7.** Are your requirements granular enough that no single requirement could be satisfied by two meaningfully different implementations? + +*Evidence check: Take your three highest-risk requirements. For each one, try to describe two different implementations that would both technically satisfy the requirement as written. If you can do it for any of them, those requirements are not granular enough.* + +**Q8.** Are your requirements free of catch-all language that defers specification decisions to the implementer? + +*Evidence check: Search your requirements document for the words "appropriate," "adequate," "as required," "suitable," "etc," "as necessary," and "to be defined." Count the occurrences. More than five per hundred requirements and the answer is no.* + +**Q9.** Was every requirement written and signed off by both the stakeholder who owns the need and the engineer who will implement it? + +*Evidence check: Take three requirements at random. Name the person who wrote each one. Name the stakeholder who approved it. Name the engineer who reviewed it for technical feasibility before sign-off. If you cannot name both parties for any requirement, the process was not adequate.* + +**Q10.** Does every requirement in your programme have a defined, specific, testable acceptance criterion that produces an unambiguous pass or fail? + +*Evidence check: Read three requirements at random. For each one, state the specific pass/fail condition right now without referring to another document. If any produce "it depends" or require interpretation, the answer is no.* + +**Q11.** When requirements change on your programme, can you assess the full impact on design, dependent requirements, test cases, and delivery estimates within one working day? + +*Evidence check: Take the last significant requirements change on your programme. How long did impact assessment actually take? Were gaps discovered later during implementation? If yes, the traceability was insufficient.* + +--- + +### Dimension 3 — Knowledge, People, and Capability + +**Q12.** Is the reasoning behind your three most critical architectural decisions written down and findable by someone who joined the programme in the last three months? + +*Evidence check: Ask someone who joined recently to find the rationale for your most significant architectural decision without asking anyone. If it takes more than ten minutes or requires asking a person, the answer is no.* + +**Q13.** Do you have a documented assessment of the capability gaps between what your programme requires and what your current team can deliver? + +*Evidence check: Find that assessment. When was it last updated? What actions were taken as a result? If the assessment does not exist or has not changed a programme decision, capability gaps are being carried as invisible risk.* + +**Q14.** Is your technical debt documented, risk-assessed, formally accepted by the business, and allocated budget for resolution? + +*Evidence check: Find your technical debt register. For each item confirm four things — who accepted it, when, with what risk assessment, and what budget is allocated to resolve it. If you cannot answer all four for any item, it is not managed debt.* + +**Q15.** If your three most experienced engineers left tomorrow, is what they know about why the system was built the way it was written down and accessible? + +*Evidence check: Write down right now what would be lost. That list is your knowledge risk register. If writing it takes less than five minutes either your knowledge is well preserved or you do not know what you would lose — and those require different responses.* + +**Q16.** When you last needed a capability your team did not have, was the response proportional to how critical and long-lived that need was? + +*Evidence check: Name the capability gap, the response chosen, and the outcome. If the response was on-the-job upskilling under delivery pressure and the capability was critical, the gap was not adequately addressed.* + +--- + +### Dimension 4 — Delivery and Operations + +**Q17.** Was your operations team involved in design reviews and NFR definition early enough to influence what was built? + +*Evidence check: Name the first design review your operations team attended. Name one NFR that exists because ops raised it. If you cannot name either, ops involvement was too late to influence the product.* + +**Q18.** Could your operations team diagnose and resolve your three most likely production incidents today without involving the delivery team? + +*Evidence check: Ask your operations lead to walk through the diagnosis and resolution of your three most likely incidents right now. If delivery team knowledge is required for any of them, the handover is incomplete.* + +**Q19.** Do your engineers have direct, regular visibility of how end customers actually use the product they are building? + +*Evidence check: Name the last time an engineer spoke directly to or observed an end customer. Name what changed in the programme as a result. If nothing changed, the visibility is not feeding back into delivery decisions.* + +**Q20.** When your programme produces a workaround to meet a deadline, is that workaround formally recorded as technical debt with an owner, risk assessment, business acceptance, and resolution budget? + +*Evidence check: Find the last three deadline-driven workarounds on your programme in the technical debt register. If they are not there, they are invisible risks accumulating interest.* + +--- + +### Dimension 5 — Leadership and Governance + +**Q21.** Do your engineers know without asking which decisions they are empowered to make and which require escalation? + +*Evidence check: Ask three engineers independently and without warning to describe their decision authority. If their answers differ significantly or any answer is "it depends," the boundaries are not clear enough to be functional.* + +**Q22.** Does your governance process improve the quality of decisions rather than primarily create a record that decisions were made? + +*Evidence check: Name the last decision your governance process changed significantly — not approved, but changed. If you cannot name one, your governance is producing records, not improving decisions.* + +**Q23.** Does your senior programme governance focus primarily on escalated risks and issues rather than task-level status? + +*Evidence check: Find the agenda and minutes of your last senior programme review. Calculate the proportion of time spent on task status versus risks and issues requiring decisions. If task status dominated, the governance model is inverted.* + +**Q24.** When your last significant programme failure occurred, did the accountability land on the person who made the decision that caused it? + +*Evidence check: Name the failure. Name the decision that caused it. Name who made that decision. Name who was held accountable. If those are different people, accountability is inverted in your organisation and the conditions for the next failure are in place.* + +**Q25.** When your engineering team's estimate has been challenged or compressed under commercial or schedule pressure, has the final number turned out to be more accurate than the original estimate? + +*Evidence check: Name the last three instances where an estimate was challenged and a lower number was agreed under pressure. Find the actual outcome for each. If the original estimates were more accurate than the compressed ones, pressure is not extracting truth. It is suppressing it.* + +--- + +### Dimension 6 — Culture and Psychological Safety + +**Q26 — unscored.** If you answered all twenty-five of these questions honestly, wrote your answers down, and showed them to your engineering team — would they agree with what you wrote? + +*Evidence check: There is no evidence check for this one. If the answer is yes, your self-assessment is probably accurate. If the answer is "I'm not sure," your self-assessment may be optimistic. If the answer is "they would disagree significantly," the gap between leadership perception and engineering reality is the most important finding of this diagnostic — and it is the gap the entire framework exists to close.* + +--- + +## Scoring and interpretation + +**Score one point for each confirmed yes with evidence.** + +**22–25:** Your delivery system has genuine structural health. The nos are your improvement priorities in order of their position in the diagnostic — commitments and requirements failures compound everything else. + +**17–21:** Meaningful gaps in at least two dimensions. The nos are active programme risks, not future improvements. + +**12–16:** Your programme is carrying significantly more risk than your reporting shows. Start with Dimension 1 and Dimension 2 — commitment and requirements failures are the primary risk multipliers for everything downstream. + +**7–11:** Your programme is being sustained by people rather than systems. That is not a performance problem. It is a structural one, and the people sustaining it will not do so indefinitely. + +**0–6:** The conditions that produce programme failure are present across most dimensions. The failure may not have arrived yet. When it does, it will not be a surprise to the engineers. + +--- + +## The clustering insight + +Your score matters less than where your nos cluster. + +**Nos in Dimension 1 only:** Foundations are weak but execution is sound. Fix the commitment and planning discipline before the next programme starts — it cannot be fixed mid-flight. + +**Nos in Dimension 2 only:** Delivery is working but building on uncertain ground. Requirements gaps will surface as integration failures, acceptance failures, or field failures. The later they surface the more expensive they are. + +**Nos in Dimension 3 only:** Systems are reasonable but knowledge is fragile. One or two key departures will expose this. + +**Nos in Dimensions 1 and 2 together:** The most common high-risk pattern. Uncommitted dates and inadequate requirements are the two primary causes of the failure cycle described in Part One. If both are weak, the programme is already in the failure pattern whether or not it is visible yet. + +**Nos in Dimension 6:** The formal diagnostic is unreliable. Before acting on scores in other dimensions, address the culture and psychological safety gaps — otherwise decisions are being made on information the system has filtered to be acceptable rather than accurate. + +--- + +## Progressive adoption + +The diagnostic reveals where the gaps are. The maturity model guides how to close them — not all at once, but in a sequence that builds capability and trust progressively rather than attempting a transformation that the organisation's current conditions cannot support. + +### Stage 1 — Localised discipline + +Firmitas principles applied within a single team or programme. The risk register is being used as a decision instrument. Estimates are three-point with conditions attached. Gates are generating decisions rather than approvals. The team is managing its own delivery with genuine autonomy within defined boundaries. + +This stage builds credibility through results. It does not require organisational change. It requires a leader with the authority to govern one programme differently and the willingness to demonstrate what different governance produces. + +Diagnostic score range: 7–12. + +### Stage 2 — Cross-team alignment + +Shared artefacts and decision logic emerge across teams. Interfaces are explicitly defined and maintained. Dependencies are managed as programme risks with trigger dates and owners. Architecture decisions are documented and accessible. The traceability spine connects requirements to evidence across team boundaries. + +This stage requires the architectural and requirements disciplines described in Chapters 11 and 13. It also requires the team topology and communication disciplines described in Chapters 17 and 18 — because cross-team alignment requires that team boundaries are designed to support the integration cadence the delivery model requires. + +Diagnostic score range: 12–17. + +### Stage 3 — Organisational integration + +Portfolio-level thinking replaces project-level optimisation. Economic trade-offs are explicit. Quality, risk, and compliance are integrated into delivery flow. Leadership behaviour starts to align with system needs — governance reviews are driven by risks and issues rather than task status, estimates are accepted with their conditions, and accountability is attributed accurately when programmes fail. + +This stage is where the political obstacles described in Chapter 24 are most active. The changes required reach the governance layer, which means they require leaders who own the governance to change their own behaviour — not just to adopt new processes, but to govern differently in the specific moments when governance matters. + +Diagnostic score range: 17–22. + +### Stage 4 — System stewardship + +Leadership actively manages system health. Slack is deliberately protected. Learning is institutionalised. The framework's disciplines are embedded in how the organisation works rather than being maintained as an explicit framework adoption. Adaptation becomes continuous rather than episodic. + +At this stage, Firmitas is no longer "a framework being adopted." It is how the organisation delivers — with the principles internationalised, the governance disciplines embedded, and the continuous improvement that the learning system described in Chapter 5 produces when it is functioning correctly. + +Diagnostic score range: 22–25. + +--- + +## Avoiding the framework layering trap + +The most consistent adoption failure mode is layering Firmitas on top of existing frameworks without removing or adjusting the conflicting structures. Retaining approval-heavy governance while claiming to have adopted guardrails-based governance. Introducing three-point estimation while continuing to govern programmes against the committed date. Demanding outcomes while measuring outputs. + +The layering trap produces a programme with two governance systems — the formal one that was adopted and the informal one that actually governs. Teams learn quickly which one matters. They comply with the formal system when it is observed and revert to the informal system when the pressure is on. + +Firmitas requires subtraction as well as addition. The governance processes that conflict with the framework's principles must be removed, not supplemented. The commitment practices that produce reverse-engineered plans must be replaced, not accompanied by a three-point estimation process that produces the honest estimate while the programme is governed against the committed date anyway. + +The adoption sequence matters. The philosophy layer must be established before the framework layer. The framework layer must be established before the process layer. Changing the process without engaging with the philosophy produces compliance theatre. The artefacts exist. The governance behaviour does not change. The outcomes do not improve. + +--- + +## What success looks like + +Firmitas is never fully adopted. The framework evolves as understanding improves. The organisation changes as markets shift, programmes complete, and people move. The disciplines that are embedded today require active maintenance to remain embedded as the conditions that produced them change. + +The measure of adoption is not compliance with the framework's artefact requirements. It is the quality of the governance decisions the framework produces. Can the programme board look at the risk register at week three and see the decisions it requires? Can the engineering team produce an honest three-point estimate with the confidence that it will be accepted, conditions and all? Can the leader who receives uncomfortable information treat it as the valuable signal it is rather than as the threat it used to be? + +When the answer to those questions is yes — when the governance behaviour has genuinely changed, not just the governance process — the framework is working. The specific artefacts, the specific processes, the specific cadences are secondary. They are the means. The ends are decisions that are informed, accountable, and made at the right level, by the right people, with the right information. + +That is what the diagnostic is measuring. That is what the maturity model is building toward. And that is what the rest of Part Three is designed to sustain. diff --git a/content/firmitas/part-3-making-it-stick/26-common-failure-modes-and-anti-patterns.md b/content/firmitas/part-3-making-it-stick/26-common-failure-modes-and-anti-patterns.md new file mode 100644 index 0000000..788175d --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/26-common-failure-modes-and-anti-patterns.md @@ -0,0 +1,131 @@ +--- +title: "Chapter 26: Common Failure Modes and Anti-Patterns" +--- + +# Chapter 26: Common Failure Modes and Anti-Patterns + +Frameworks fail in predictable ways. The specific failure modes of Firmitas adoption are not random — they follow from the specific mechanisms that the framework is designed to address and the specific ways in which organisations resist the changes those mechanisms require. + +This chapter catalogues those failure modes. Not as warnings to be avoided through careful reading, but as diagnostic signals — specific, observable patterns that indicate the framework is being adopted in form while the underlying system remains unchanged. Each one has a symptom that is visible, a cause that is structural, and a correction that is specific. + +The chapter also addresses the hardest failure mode of all: the adoption that appears successful by every formal measure while the governance behaviour that produces the outcomes the framework is supposed to prevent continues unchanged beneath the surface. + +--- + +## The primary failure mode — philosophy discarded, process retained + +Every subsequent failure mode in this chapter is a variant of this one. + +Firmitas is a philosophy-led framework. The philosophy layer — the twelve principles that establish the non-negotiables — constrains the framework layer, which constrains the process layer. When the philosophy is engaged with and the framework is understood in its context, the process layer can be applied in ways that produce the governance outcomes the framework is designed for. + +When the philosophy is skipped and the process layer is installed directly, the framework becomes a set of artefacts and ceremonies. The risk register exists but is still used as a reporting tool. The three-point estimate is produced but is still compressed toward the committed date. The gate review is held but still results in approval regardless of the evidence state. The ADRs are written but are still produced to satisfy a process requirement rather than to preserve reasoning. The retrospective is held but still produces a list of actions nobody has time to implement. + +The observable signal of this failure mode is a specific question. Teams that have genuinely engaged with the framework ask: "are outcomes improving?" Teams that have adopted the process without the philosophy ask: "are we doing this correctly?" The shift from the first question to the second is the first sign that the philosophy has been lost and the process has become the point. + +The correction is not to add more process. It is to return to the principles — to ask, for each artefact and each ceremony, what outcome it is supposed to produce and whether it is producing it. If the risk register review is not generating decisions, the question is not whether the register is in the correct format. It is why the governance culture treats the register as a reporting artefact rather than a decision instrument, and what needs to change about the governance behaviour to change that treatment. + +--- + +## Treating the committed date as a governance anchor + +The most consequential single anti-pattern in engineering programme management — and the one that most directly produces the failure in Chapter 1 — is the treatment of the committed date as the primary governance anchor against which everything else is calibrated. + +When the committed date is the anchor, the estimate is expected to reach it. The risk register is expected to show that the date is achievable. The gate review is expected to confirm that the programme is on track. The status report is expected to show green. Everything is calibrated to confirm the date rather than to reveal the programme's real state. + +The observable signals are consistent. Estimates that are challenged until they show the committed date as most likely. Risk registers that show amber rather than red because red would threaten the date narrative. Gate reviews that approve programmes that are not genuinely ready because readiness is assessed against the narrative rather than the evidence. Status reports that are green while the corridors contain conversations about how the programme is actually doing. + +The correction is the commitment model described in Chapter 9 — separating the commercial commitment from the honest estimate, managing the gap as a programme risk, and governing the programme against the honest estimate rather than the commercial commitment. This requires the leadership behaviour change described in Chapter 24 — specifically, the willingness to accept a most likely estimate that differs from the committed date and to treat the gap as information rather than as an engineering failure. + +--- + +## Cargo cult governance + +Cargo cult governance is the adoption of governance artefacts and ceremonies without the understanding of what they are supposed to produce. + +The programme has a risk register. The risk register is reviewed at every programme board. The programme board has reviewed the risk register. The programme board has performed risk management. This is cargo cult governance — the performance of the activities associated with risk management without the substance of it, in the same way that cargo cults perform the activities associated with receiving aircraft without understanding the conditions that cause aircraft to arrive. + +The observable signals: risk registers that grow without items ever being retired through decision and action. Programme boards that consistently spend less than five minutes on the risk register before moving to task status. Gate reviews that approve programmes without the evidence required having been produced. ADRs that describe decisions that were made differently from how the ADR describes them. Retrospectives that end with action items nobody owns. + +The correction is not to make the ceremonies more elaborate. It is to reconnect the ceremonies to their function. The risk register review is not a ceremony. It is a meeting whose purpose is to make decisions on escalated items. It ends when those decisions have been made — not when the items have been reviewed. If the programme board is spending five minutes on the risk register, the risk register is not functioning as a decision instrument. The governance structure needs to change, not the risk register format. + +--- + +### Empowerment theatre + +Empowerment theatre is the declaration of team autonomy without the structural changes that autonomy requires. Teams are told they own the delivery. They are given autonomy over their ways of working. They are empowered to make decisions. + +And then the plan is changed without consultation. The architecture decision is overridden in a review the team was not invited to. The estimate is replaced by the number that makes the business case work. The risk that was escalated is noted and the conversation moves on. + +The observable signals: teams that use the language of autonomy in formal settings and behave defensively in operational ones. Decision boundaries that are unclear enough that teams spend time seeking approval for decisions they should be making independently, while governance layers make operational decisions they should be deferring to teams. Engineers who route around the governance system because they have learned that engaging with it produces worse outcomes than working around it. + +The correction is the decision authority alignment described in Chapter 8 — explicit decision boundaries that are clear enough for teams to operate within without seeking approval for routine decisions, and that are stable enough that teams can trust them to hold under pressure. Autonomy that disappears when the programme is under pressure was never autonomy. It was permission, granted conditionally and revoked when the conditions changed. + +--- + +### Output obsession in outcome language + +This failure mode occurs when an organisation adopts the language of outcomes — "we focus on value delivery, not feature factories" — while continuing to measure, incentivise, and govern based on outputs. + +Velocity targets dressed as "delivery predictability." Feature counts reported as "value creation." OKRs defined as delivery milestones. Teams optimised for throughput in an organisation that claims to care about impact. The language has changed. The measurement has not. The behaviour that measurement produces has not changed. + +The observable signals: teams that know the outcome language and use it correctly in formal settings while working in ways that optimise for the metrics they are actually measured on. Sprint reviews that report completed features without discussion of whether those features produced the customer outcomes they were supposed to produce. Portfolio reviews that assess commercial milestones without assessing customer impact. + +The correction is the alignment between stated values and actual measurement described in Chapter 22. If the organisation values outcomes, the governance metrics must measure outcomes. If the metrics measure outputs while the language claims to measure outcomes, the teams will measure outcomes with the language and deliver outputs with the work. + +--- + +### "Lightweight" becoming "no discipline" + +A specific reaction to heavy governance processes — the approval gates, the documentation requirements, the formal reviews — is to eliminate them entirely in the name of agility or pragmatism. The team is "lightweight" and "pragmatic." It does not produce unnecessary documentation. It does not hold ceremonies that do not add value. + +What it also does not do is document architectural decisions, maintain a live risk register, define done criteria that include quality standards, or produce the traceability that regulated compliance requires. The reaction to bureaucracy has overshot the target, eliminating the discipline as well as the unnecessary overhead. + +The observable signals: decisions that are undocumented and must be reconstructed from memory when they are challenged. Risk registers that contain no risks because the team has concluded that risk management is bureaucratic overhead. Technical debt that is invisible because documenting it would take time that could be spent on delivery. Integration failures that could have been anticipated if an interface definition had been maintained. + +The correction is the distinction between ceremony and discipline — between the governance activities that exist to produce records and the governance activities that exist to produce outcomes. Eliminating the ceremony while preserving the discipline requires the governance culture to be able to articulate, for each activity, what outcome it is supposed to produce and whether a lighter-weight version of the activity produces that outcome adequately. An ADR need not be long. It must record the decision and the reasoning. A risk register need not be elaborate. It must surface the decisions the programme needs to make. Lightweight governance is not the absence of discipline. It is discipline at the minimum necessary scale to produce the outcomes required. + +--- + +### Delegating system health to teams alone + +This failure mode occurs when leadership claims to have adopted the framework's philosophy — teams own their delivery, psychological safety is important, learning from failure is valued — while retaining the structural constraints that make those aspirations impossible. + +Teams blamed for outcomes they cannot influence because the incentive structures, funding models, or governance constraints that produce those outcomes are unchanged. Repeated "coaching" and "culture work" at the team level while the systemic causes of poor performance remain at the leadership level. Retrospectives that surface the same issues cycle after cycle because the authority to address the structural causes resides above the team's authority level. + +The observable signals: teams that have been through multiple cycles of "improvement initiatives" without sustained improvement. Retrospective action items that are team-level interventions for organisation-level problems. High team turnover in areas where the structural conditions are most misaligned with the framework's requirements. + +The correction is the leadership accountability described in Chapter 7 — the recognition that system health is a leadership responsibility that cannot be delegated to the teams operating within the system. Teams can identify what is wrong. Only leadership has the authority to change it. The coaching and the culture work that does not change the structural conditions is not investment in improvement. It is the appearance of investment, producing the wrong lesson in the same way that the lessons-learned exercise in Chapter 5 produces the wrong lesson from programme failure. + +--- + +### The meta-failure — not learning from failure + +The failure mode that all the others feed into is the meta-failure: the organisation that experiences programme failures, framework adoption failures, and governance failures repeatedly without updating its understanding of why they occur. + +The retrospective is held. The action items are generated. Nothing changes. The next programme fails in the same way. The same action items are generated. The cycle repeats. + +The observable signals: the same failure patterns appearing across multiple programmes without the root cause being addressed. Lessons learned libraries that contain insights from programmes going back years without evidence that those insights have been incorporated into subsequent programme governance. Leaders who can describe the failure patterns accurately and in detail without connecting them to the governance decisions that produced them. + +The correction is the learning system described in Chapter 5 — the governance capacity to identify failure causes accurately, to trace those causes to the governance decisions that produced them, and to change the governance behaviour that those decisions represent. This requires the diagnostic honesty that Chapter 24 described — the willingness to examine one's own governance decisions as causes of programme outcomes rather than as context within which execution fails. + +A framework that is being genuinely applied produces learning. Each programme reveals something about the governance system that the previous programme did not reveal, and the governance system is adjusted accordingly. A framework that is being performed produces compliance records. Each programme produces documentation that the process was followed. The next programme begins without the governance system having changed. + +The difference between these two is not the sophistication of the framework. It is the honesty of the learning. + +--- + +### The first warning signs + +The failure modes described in this chapter share a common early signal that appears before the consequences become visible: the governance layer stops asking the question that the framework is designed to answer. + +The question is: are outcomes improving? + +When a programme board stops asking whether the governance process is producing better decisions, and starts asking whether the governance process is being followed correctly — the philosophy has been lost. When a team stops asking whether the risk register is surfacing the decisions the programme needs, and starts asking whether the risk register format is correct — the ceremony has replaced the substance. When a leader stops asking whether the estimate reflects honest engineering assessment, and starts asking whether the estimate has been challenged sufficiently — the committed date has become the governance anchor. + +Each of these shifts is visible before the consequences arrive. The shift from outcome questions to process questions is the earliest reliable signal that the framework is being adopted in form while the governance behaviour remains unchanged. + +The practical response: at every governance meeting, in every framework review, the primary question is "what decisions did this produce?" Not "did this occur?" Not "was the artefact in the correct format?" Not "did the process step happen?" What decisions did this produce, and were those the right decisions? + +When the answer is consistently yes — when the governance process is consistently producing informed, accountable decisions at the right level — the framework is working. When the answer is consistently "the process occurred" rather than "decisions were produced," the framework is being performed rather than applied. + +That distinction is the only one that matters. diff --git a/content/firmitas/part-3-making-it-stick/27-measuring-what-actually-matters.md b/content/firmitas/part-3-making-it-stick/27-measuring-what-actually-matters.md new file mode 100644 index 0000000..1428567 --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/27-measuring-what-actually-matters.md @@ -0,0 +1,111 @@ +--- +title: "Chapter 27: Measuring What Actually Matters" +--- + +# Chapter 27: Measuring What Actually Matters + +Measurement is one of the most powerful forces in any engineering organisation. It shapes behaviour, priorities, and decision-making more strongly than stated values, framework principles, or leadership intent. What gets measured gets optimised. What gets optimised is what the measurement rewards, regardless of whether that is what the organisation intended to reward. + +This creates a specific and consequential risk in framework adoption: measurement systems designed for the wrong outcomes silently undermine the framework's governance disciplines while appearing to support them. The organisation measures velocity and gets velocity — at the cost of quality. It measures milestone completion and gets milestone completion — at the cost of honest assessment. It measures defect counts and gets low defect counts — because teams stop looking for defects rather than stop creating them. + +Firmitas treats measurement as a system design problem, not a reporting exercise. Poorly designed measures distort behaviour, hide risk, and erode the conditions the framework requires. Well-designed measures make reality visible, enable learning, and support the governance decisions that the framework is built around. + +This chapter addresses what to measure, what to avoid measuring, and how measurement connects to the diagnostic dimensions established in Chapter 25. + +--- + +## Measurement is never neutral + +Every measure creates an incentive. Every incentive changes behaviour. The question is not whether measurement influences outcomes — it always does. The question is whether it influences them deliberately or accidentally. + +The most common measurement failure in engineering programme management is the adoption of metrics that are easy to collect but inversely correlated with what they are supposed to measure. Defect counts that fall because teams stop reporting defects. Test coverage percentages that rise because tests are written to cover lines rather than to catch failures. Story point velocity that increases because the definition of done is relaxed. Feature counts that grow because features are subdivided into smaller units. Each of these metrics improves without the underlying quality improving. Each of them teaches teams what to optimise — and what to sacrifice to optimise it. + +The governance layer that sees improving metrics and concludes that the programme is improving has been deceived not by the teams but by the measurement system. The teams are behaving rationally — optimising the thing they are measured on. The governance layer is drawing conclusions from the optimised measurement rather than from the operational reality it was supposed to represent. The gap between the measurement and the reality it claims to represent is the governance blindspot that allows the failure patterns of Chapter 1 to accumulate invisibly. + +--- + +## Three categories of measurement + +Firmitas distinguishes between three categories of measurement that serve different governance purposes and must not substitute for each other. + +**Output measures** describe what has been produced — features delivered, tasks completed, milestones reached, documents produced, tests written. Output measures are not meaningless. They provide a view of engineering activity that can be useful for understanding capacity, identifying bottlenecks, and tracking progress against commitments. But output measures say nothing about whether the activity produced the outcomes it was supposed to produce, whether the quality of the output is adequate, or whether the delivery system is sustainable. + +Output measures become damaging when they are used as success criteria. A team that is measured on feature throughput and told that throughput is the primary indicator of success will optimise for throughput — by reducing test coverage, by deferring quality work, by narrowing the definition of done, by taking the shortcuts that allow tasks to be closed faster without the work being genuinely complete. The measurement has not made the team less capable or less well-intentioned. It has made throughput the rational objective and quality the available sacrifice. + +Firmitas allows output measures as local, transient signals — useful for identifying capacity constraints and understanding near-term delivery pace — but never as success criteria or primary governance instruments. + +**Outcome measures** describe the effect of engineering work on the world outside the engineering system — customer adoption, operational efficiency, support burden reduction, revenue enabled, regulatory compliance achieved. Outcome measures are the primary success criteria in Firmitas. They answer the question that justifies the engineering investment: did what was built produce the value it was supposed to produce? + +Outcome measures are harder to collect than output measures. They are noisier. Their attribution is less clear — many factors beyond engineering quality determine whether a product is adopted. They are often lagging — the outcome of a design decision may not be visible for months after the decision was made. These properties make them less satisfying as management instruments than output measures, which is why organisations consistently default to outputs. + +But the difficulty of outcome measurement does not make it optional. The organisation that measures only outputs and claims to be delivering outcomes is measuring the wrong thing and drawing the wrong conclusions. The programme that delivers its features on time and within budget, to a specification that the customer cannot use in the way the customer needs, has produced outputs and failed to produce outcomes. + +Firmitas accepts imperfect outcome measures over precise output measures. A rough measure of customer adoption is more useful than a precise measure of feature throughput — because the adoption measure is trying to answer the right question, even imperfectly, while the throughput measure is answering a question that is not the right one with apparent precision. + +**System health measures** describe the capability of the engineering system to continue delivering outcomes over time — whether the conditions that make good delivery possible are present and being maintained. System health measures include defect escape rates, lead time variability, incident recurrence, technical debt trends, team cognitive load indicators, and staff retention and engagement. + +System health measures are the most important class of measure for long-term programme governance and the most consistently neglected. They are neglected because they measure conditions rather than outputs — the capability to continue delivering rather than the delivery that has occurred. Their consequences appear slowly and their causes are separated from their effects by delays that make attribution difficult. + +The organisation that ignores system health measures may maintain strong output metrics right up to the point at which the health of the underlying system collapses — when the technical debt accumulated over many delivery cycles makes new development prohibitively expensive, when the team that maintained the product's institutional knowledge departs, when the architecture that was never refactored becomes too brittle to change safely. The system health measures would have signalled the approaching collapse months or years earlier. The output measures would have shown green throughout. + +--- + +## The connection to the diagnostic dimensions + +The measurement framework connects directly to the six diagnostic dimensions in Chapter 25. Each dimension has a corresponding measurement focus that reveals whether the dimension is healthy or degraded. + +**Dimension 1 — Commitments and planning** is measured by the accuracy of three-point estimates over time. Not whether individual estimates prove correct — that is not a useful measure. Whether the most likely case is consistently more accurate than the best case, and whether the conditions attached to estimates are consistently assessed for accuracy after programmes complete. An organisation that consistently achieves best-case outcomes has either very good fortune or a governance culture that manages the conditions the best case requires. An organisation that consistently achieves worst-case outcomes has a systematic risk management failure that the measurement should surface. + +**Dimension 2 — Requirements** is measured by the phase in which requirements gaps are discovered. Requirements gaps discovered during specification cost hours to resolve. The same gaps discovered during integration cost days. Discovered during acceptance testing, they cost weeks. Discovered in the field, they cost the customer relationship. An organisation that consistently discovers requirements gaps late is investing insufficiently in specification quality. The measurement of defect escape rate by originating phase — where each defect was introduced versus where it was found — is the most revealing requirements quality measure available. + +**Dimension 3 — Knowledge and capability** is measured by the rate at which the programme's performance changes when key people leave or join. A programme whose delivery performance is stable through personnel changes has institutionalised its knowledge adequately. A programme that experiences significant degradation when specific individuals leave is carrying knowledge risk that the measurement should surface as a system health signal. + +**Dimension 4 — Delivery and operations** is measured by operational incident rates, mean time to recovery, and the proportion of incidents that could have been anticipated during design or development. A system that generates frequent incidents of types that should have been designed out has operational quality gaps that the measurement should surface as delivery quality signals rather than operational performance signals. + +**Dimension 5 — Leadership and governance** is measured by decision latency — how long it takes for an escalated risk or issue to receive a governance decision — and by decision accuracy — whether the decisions made at governance reviews prove correct in retrospect. Decision latency measures whether the governance layer is accessible and responsive. Decision accuracy measures whether the governance layer has the information and understanding to make decisions that improve programme outcomes rather than simply confirming the programme narrative. + +**Dimension 6 — Culture and psychological safety** is measured by the gap between formal and informal risk assessments. When the risk register contains risks that engineers do not regard as the programme's real risks, the measure of that gap reveals the extent to which the governance culture has made honest risk disclosure unsafe. This measure is not easily collected through dashboards. It requires deliberate leadership effort — the specific practice of asking engineers what they would say about the programme's risks if they were not being formally reported. + +--- + +## What Firmitas explicitly does not measure + +The explicit rejection of certain measurement practices is as important as the endorsement of others. Measures that consistently distort behaviour and degrade the conditions the framework requires should be named and avoided. + +**Individual performance metrics for engineers.** Measuring individual engineers on defect rates, code output, feature throughput, or story points produced creates the conditions for the behaviour those metrics reward — and penalises the behaviours the framework requires. The engineer who takes time to understand a requirement properly before implementing it will appear slower than the engineer who implements immediately and fixes defects later. The engineer who surfaces a risk honestly will appear to be the source of programme problems rather than the person managing them. Individual performance metrics for engineers consistently reward the appearance of productivity over the substance of quality, and punish the honesty the framework depends on. + +**Velocity as a commitment basis.** Using sprint velocity as the basis for release date commitments transforms a tool that is supposed to help teams understand their capacity into a rod for their backs. Velocity that is used for commitments will be gamed — not through dishonesty but through the rational adaptation to an environment that requires velocity to be high to avoid the consequences of low velocity. The gaming produces inflated velocity measurements that are used to generate unrealistic commitments that fail in the ways Chapter 1 describes, at the sprint level rather than the programme level. + +**Utilisation rates as efficiency measures.** Chapter 12 addressed utilisation in detail. The organisation that measures team utilisation as a primary efficiency indicator will drive teams toward full utilisation — and toward the fragility, quality degradation, and learning failure that full utilisation produces. Utilisation is a local measure that, when optimised, consistently degrades global outcomes. It should not be used as a primary governance metric. + +**Maturity model scores as achievement indicators.** The maturity model in Chapter 25 describes stages of adoption as a developmental framework — a way of understanding where the organisation is and what it needs to develop further. When maturity model scores become performance indicators — when teams or programmes are assessed on their maturity level — the scores will improve without the underlying capability improving. Teams will learn to answer the maturity assessment questions correctly without changing the governance behaviour the questions are supposed to reveal. The diagnostic and maturity tools become compliance exercises rather than learning instruments. + +--- + +## Measurement review cadence + +Different measures reveal different aspects of programme and organisational health and are most useful at different cadences. + +**Operational reviews** are short-cadence reviews — weekly or fortnightly — focused on the leading indicators that reveal near-term risk and enable early intervention. Queue length trends. Escalated risk item counts. Blocked work items. Integration test failure rates. These are the measures that allow the programme management layer to identify emerging problems before they become governance escalations. + +**Programme reviews** are medium-cadence reviews — monthly or at programme phase transitions — focused on the system health measures that reveal whether the programme is sustainable. Defect escape rates. Rework levels. Technical debt trends. Lead time variability. Capability gap status. These are the measures that allow the governance layer to assess whether the programme is maintaining the conditions for continued delivery. + +**Portfolio reviews** are longer-cadence reviews — quarterly or at portfolio reprioritsation points — focused on outcome measures that reveal whether the portfolio's engineering investments are producing the business outcomes they were supposed to produce. Customer adoption rates. Operational performance improvements. Revenue enabled by delivered capability. These are the measures that allow the portfolio governance layer to assess whether the investment portfolio is aligned with the business strategy it is supposed to serve. + +The governance failure is in using the wrong cadence for the wrong measure — reviewing outcome measures at operational cadence (which produces noise rather than signal, because outcomes lag activity by months), or reviewing operational measures at portfolio cadence (which misses the early signals that require immediate response). + +--- + +## The governance principle for measurement + +Measurement in Firmitas serves learning, not control. + +Measures that exist to demonstrate that the programme is on track — that are designed to show green, that are reviewed by governance layers to receive reassurance that delivery is proceeding — are not learning instruments. They are comfort instruments. They produce the same governance blindspot that the programme in Chapter 1 exhibited: a governance layer reading a mostly-green status report while the programme is accumulating risk that the status report is not designed to reveal. + +Measures that exist to surface the conditions the programme depends on, to reveal where the system is under stress, and to provide the early warning that allows governance decisions to be made while options still exist — these are learning instruments. They make reality visible. They create discomfort in proportion to the health problems they reveal. They generate the governance conversations that prevent the programme failures they anticipate. + +The governance layer that evaluates its measurement system by whether the measures make it comfortable is not governing. It is being governed by its own comfort preferences. The governance layer that evaluates its measurement system by whether the measures reveal what it needs to know to make good decisions is governing — even when what it needs to know is uncomfortable. + +Measurement is in service of governance. Governance is in service of outcomes. Outcomes are what the programme exists to produce. The measurement system that reveals whether outcomes are being produced, and what system conditions are enabling or preventing them, is the measurement system the framework requires. + +Everything else is the appearance of management. diff --git a/content/firmitas/part-3-making-it-stick/28-what-firmitas-cannot-fix.md b/content/firmitas/part-3-making-it-stick/28-what-firmitas-cannot-fix.md new file mode 100644 index 0000000..1b69526 --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/28-what-firmitas-cannot-fix.md @@ -0,0 +1,123 @@ +--- +title: "Chapter 28: What Firmitas Cannot Fix" +--- + +# Chapter 28: What Firmitas Cannot Fix + +A framework that cannot tell you when it is the wrong tool is not a framework. It is a belief system — one that attributes every failure to insufficient or incorrect application rather than to the possibility that the framework itself does not fit the problem. + +This chapter is the honest limits chapter. It names the conditions under which Firmitas will not work, the organisations it is not for, and the failure modes that it cannot prevent even when applied sincerely and competently. Some of these limits are about scope — the framework addresses specific problems and not others. Some are about context — the framework's assumptions do not hold in every organisational environment. And some are about the framework's own self-referential tension — the risk that a framework arguing against methodology-first thinking will itself be adopted as a methodology. + +Acknowledging these limits is not a disclaimer. It is the most credible thing this book can do. And it provides the reader with the tools to assess whether Firmitas is the right instrument for their specific situation, rather than applying it universally and discovering the limits the hard way. + +--- + +## The organisations Firmitas is not for + +**The very small startup without regulatory obligations.** + +A team of five to fifteen people building a product quickly, without safety-critical obligations, without a regulated compliance framework, without the integration dependencies of a multi-discipline programme, and with a direct and fast feedback loop to the customers whose behaviour will determine whether the product succeeds — this team does not need Firmitas. + +The structured foundations phase, the Requirement Breakdown Structure, the Architecture Decision Records, the risk register with trigger dates and named owners, the governance model with escalation thresholds and exception management — these are instruments designed for the complexity and consequence of programmes where the cost of getting things wrong is high, where the feedback loops are long, and where the integration dependencies between disciplines make unstructured development expensive. + +For a startup at early stage, the cost of those instruments exceeds the cost of the problems they prevent. The product will iterate rapidly. The requirements will change faster than a formal specification process can accommodate. The architecture will evolve faster than ADRs can capture. The risk register will lag the reality of a team that is discovering its product-market fit in weeks rather than months. + +The philosophy of Firmitas applies — outcomes over outputs, delivery knowledge closest to the work, honest estimation, customer line of sight. But the framework overhead does not. Applying the full Firmitas governance model to a five-person startup is not discipline. It is the imposition of a structure designed for programmes of fifty that will slow the team down without protecting anything that needs protecting at this scale. + +The practical test: does the programme involve multiple disciplines with incommensurable cadences, a regulated compliance framework, safety obligations, or integration dependencies that require coordination across organisational boundaries? If not, the philosophy applies but the framework overhead probably does not. + +**Organisations where the primary obstacle is political unwillingness rather than structural ignorance.** + +Firmitas can help a willing organisation build better systems. It cannot make an unwilling one want to. + +The organisations where the primary obstacle to better delivery is not structural — not the governance model, not the estimation practices, not the risk management discipline — but political, are organisations where people with power benefit from the current dysfunction and have no incentive to change it. + +The commercial director whose authority depends on making commitments before engineering has been consulted. The programme director whose governance role exists because decisions are centralised rather than distributed to the people with delivery knowledge. The organisational layer that exists to provide oversight of engineering teams whose performance would improve if the oversight layer's intervention were reduced. + +Each of these people is behaving rationally within a system that has structured their incentives in a particular direction. Firmitas does not restructure those incentives. It assumes a leadership layer that is willing to govern differently — to accept honest estimates, to manage risks rather than note them, to attribute accountability accurately. Where that willingness does not exist at the level of authority required to make the changes the framework requires, the framework cannot achieve what it is designed for. + +The practical test: do the people with the authority to change the governance behaviour have the incentive to change it? If the answer is no — if the current governance structure serves the interests of the people with the authority to change it — the framework will be adopted in form and rejected in substance, producing the cargo cult governance described in Chapter 26. + +**The very large multinational with structurally separated decision authority.** + +In organisations of sufficient scale and complexity, decision authority may be structurally separated from delivery knowledge in ways that no single leader within the organisation can change. Procurement frameworks that mandate specific delivery methodologies contractually. Matrix management structures that diffuse ownership so completely that no one person can be held accountable for the conditions of any specific programme. Portfolio governance imposed by a parent organisation with its own standards and incentive structures that are not aligned with the Firmitas model. + +In these environments, a willing leader can apply Firmitas principles within their own sphere of authority — their team, their programme, their immediate governance perimeter. They cannot apply it systemically without organisational change that exceeds their authority. The localised application can produce localised improvement. The systemic change that would compound those improvements across the organisation may simply not be available as an option for any individual within it. + +The practical test: does the leader with the authority to adopt Firmitas have the authority to change the governance structures that the framework requires? If the answer is no — if the relevant governance structures are determined by levels of the organisation above the leader's authority — the adoption will be partial at best. + +**Compartmentalised environments where information flow is deliberately restricted.** + +In certain defence, intelligence, and security programmes, the information flows that Firmitas depends on are deliberately constrained by security and classification requirements that are non-negotiable. + +Firmitas requires that delivery knowledge reaches decision-makers. It requires that customer reality informs engineering decisions. It requires that risk information flows honestly through the governance structure. It requires that the reasoning behind architectural decisions is preserved and accessible. + +In programmes where the customer's operational context is classified, where the programme's scope cannot be fully described to the people implementing parts of it, where the rationale for certain requirements cannot be documented because the rationale is itself classified, and where the psychological safety of honest risk disclosure is complicated by need-to-know cultures that treat information about problems as sensitive — the framework's information flow assumptions do not hold. + +Some elements of Firmitas apply in compartmentalised environments. Delivery knowledge should still reside with the people closest to the work. Governance should still act on risks rather than note them. Quality is still a system property. But significant parts of the framework assume information availability that classified programmes cannot provide. The practitioner in a compartmentalised environment must be honest about which framework elements require information flows that their security context prevents — and must not substitute assumption for the customer and operational context the framework requires without acknowledging the substitution as a risk. + +--- + +### The conditions under which Firmitas fails even when applied sincerely + +**When the programme inherits conditions it cannot change.** + +Firmitas is designed for programmes that have the authority to establish their own governance model, their own delivery structure, and their own relationship between commitment and estimation. Some programmes begin with commitments already made, architectures already designed, teams already assembled, and commercial relationships already established in ways that constrain every subsequent decision. + +A programme that inherits a reverse-engineered plan, an unvalidated architecture, a contractual commitment made without honest estimation, and a client relationship that was established on expectations the programme cannot meet is starting in the failure pattern that Firmitas is designed to prevent. It cannot undo those conditions by adopting the framework. It can apply the framework's disciplines to manage the conditions it has inherited — surfacing the risks honestly, managing what can be managed, limiting the accumulation of additional debt — but it cannot retroactively correct decisions made before it existed. + +The framework is most effective when applied from the beginning of a programme, before commitments are made and before the architecture is established. Applied mid-programme to a programme that has already entered the failure pattern, it provides better governance of the remaining programme but cannot recover the damage done before it arrived. + +**When adoption is treated as transformation rather than evolution.** + +Firmitas adopted wholesale — rolled out across all programmes simultaneously, with all governance processes changed at once, with full framework compliance expected from the first day — will fail in the ways described in Chapter 25. Not because the framework is wrong but because the organisational system cannot absorb that rate of change without collapsing into compliance theatre. + +The organisations that have improved their delivery governance sustainably have done so through progressive adoption — starting where conditions are most favourable, building evidence of better outcomes, and expanding adoption as the evidence produces the trust and the political support for wider change. This is slower than wholesale adoption. It is the only adoption that produces lasting change. + +The leader who adopts Firmitas as a transformation programme — who announces framework adoption, introduces all the artefacts and governance processes simultaneously, and measures adoption by compliance with the new process requirements — is applying a process-first approach to a framework that is explicitly designed to resist process-first adoption. + +**When the framework is used as a substitute for judgement rather than a support for it.** + +Chapter 6 established that Firmitas is a philosophy that constrains a framework that constrains process. The framework provides structure for decision-making. It does not make decisions. + +The governance layer that approves a gate because the framework's gate criteria appear to have been met, without exercising the judgement to assess whether the evidence behind those criteria is credible, has used the framework as a substitute for governance. The programme board that notes an escalated risk because the risk register format requires a governance response, without the judgement to assess whether the risk is being managed or merely being documented, has used the framework as a substitute for risk management. + +The framework supports judgement by providing structure — the questions to ask, the evidence to require, the decision boundaries to respect. It does not replace judgement. An organisation that has adopted the framework's artefacts and processes while losing the leadership judgement that gives them purpose has not adopted Firmitas. It has produced an elaborate compliance structure that will fail in the ways Chapter 26 described — because the artefacts exist but the decisions they are supposed to generate are not being made. + +--- + +### The self-referential tension + +Firmitas argues against methodology-first thinking. It will be adopted as a methodology by a significant proportion of the organisations that encounter it. Not because those organisations are stupid or dishonest — but because methodology adoption is the default organisational response to delivery problems, and the book will be given to a programme manager who has been told to fix delivery and will reach for it the way they reach for every other framework. + +The self-referential tension is real and cannot be fully resolved. A framework that argues against frameworks will be used as a framework. The philosophy layer is supposed to prevent this — by establishing the non-negotiables that constrain the process layer and by making the purpose of each element explicit. But philosophy layers are frequently skipped. They require more effort to engage with than process layers. They do not produce artefacts. They produce understanding, which takes longer and is harder to demonstrate than the completion of a template. + +The defence against this misuse is named explicitly in Chapter 26 — the earliest warning signal is the shift from "are outcomes improving?" to "are we doing this correctly?" But the defence requires leaders who are watching for it. And the leaders who are most at risk of missing it are the ones who believe they are genuinely applying the framework rather than performing it. + +The practical guidance: if the framework is producing governance artefacts more reliably than it is producing governance decisions, the philosophy has been lost. If the risk register is being reviewed more consistently than it is generating decisions, the review has become the purpose rather than the instrument. If the framework is being treated as something that can be adopted completely and then maintained through compliance monitoring, it has become a methodology. + +The test is always the outcomes. Not whether the artefacts are complete, not whether the processes are being followed, not whether the framework has been adopted — but whether the governance decisions are better, the programme outcomes are more predictable, and the conditions for honest information flow are stronger than they were before. Those are the only measures of whether Firmitas is being applied or merely performed. + +--- + +### What to do when the limits apply + +The limits described in this chapter do not leave practitioners without options. They define the scope within which the options available within the framework apply, and identify when a different instrument is needed. + +For the small startup: the philosophy applies without the framework overhead. Outcomes over outputs. Delivery knowledge closest to the work. Honest estimation within whatever lightweight structure supports it. The discipline is appropriate to the context. The overhead is not. + +For the organisation where political unwillingness is the obstacle: the entry point is evidence, not argument. A single programme governed differently, producing better outcomes, is the most powerful political argument for change that exists. The framework cannot make unwilling organisations willing. Evidence can change what unwilling organisations believe is possible — and belief about what is possible is a precursor to willingness. + +For the mid-programme rescue: apply what can be applied from the current position. Surface the risks honestly. Limit the accumulation of further debt. Make the gap between the committed plan and the honest estimate visible to the governance layer and manage it explicitly rather than silently. The framework cannot undo what has already been done. It can limit the compounding of it. + +For the compartmentalised environment: apply the elements that the security context allows and be explicit about which elements cannot be applied and why. A partial application that is honest about its limits is more useful than a complete application that is not. + +For the self-referential tension: there is no permanent resolution. There is only the repeated practice of asking whether the framework is producing outcomes or artefacts, whether governance is generating decisions or compliance records, and whether the engineering teams who know the most about the programme's real state believe the governance process is serving them or constraining them. + +--- + +### The final honest acknowledgement + +Firmitas, applied sincerely and competently in a receptive organisation with a willing leadership layer, will improve delivery outcomes. Not perfectly. Not without setbacks. Not in a linear progression from current state to ideal state. But the specific, preventable failure patterns described in Part One — the committed date that nobody verified was achievable, the risk register filed rather than acted on, the honest estimate suppressed in favour of the commercial narrative — will become less common. And when they do occur, the organisation will be more likely to identify the real cause and less likely to compound it with the wrong lesson. + +That is the claim. It is a modest one compared to the claims most frameworks make. It is an honest one. And it is the only kind worth making. diff --git a/content/firmitas/part-3-making-it-stick/29-the-long-view.md b/content/firmitas/part-3-making-it-stick/29-the-long-view.md new file mode 100644 index 0000000..8c1af77 --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/29-the-long-view.md @@ -0,0 +1,133 @@ +--- +title: "Chapter 29: The Long View" +--- + +# Chapter 29: The Long View + +The programme in Chapter 1 was fourteen months long. Or twenty-two months, depending on which number you count from. The decision that determined its outcome was made in week three, when the programme board received three red risk items, noted them, and moved to task status. + +This book has been built around that moment. The commitment that preceded it. The estimate that made it visible. The governance that failed to act on it. The compounding cycle that made the next programme more likely to fail in the same way. + +But the real cost of week three's governance failure was not the eight-month overrun. The real cost was longer and larger than that. It was paid in the systems that were built under pressure and inherited by the teams that came after. In the engineers who learned that honest estimates were not welcome and adjusted their behaviour accordingly. In the organisation that drew the wrong lesson, added the wrong process, and made the next programme harder to run well. In the institutional capability that was eroded, slowly, by the accumulation of small governance failures that individually seemed manageable and collectively produced fragility. + +The long view is the view that sees those costs. It is the perspective that engineering organisations consistently lack — because the pressures that shape their behaviour are short-term, and the costs of the decisions that those pressures produce are long-term. The programme must deliver by Q4. The budget must be maintained this year. The client must be satisfied in this engagement. These are the timescales on which engineering programmes are managed. The costs they generate appear on the timescales on which engineering organisations decay. + +--- + +## What sustainability means in engineering organisations + +Sustainability in an engineering organisation is not a welfare concept. It is an operational one. A sustainable engineering organisation is one that can continue to deliver well over time — through changes in personnel, through market shifts, through the accumulation of programme experience, and through the pressures that consistently incentivise the short-term optimisations that make long-term delivery more expensive. + +Sustainability requires three properties that this book has argued for throughout. + +**The capacity to absorb variation.** Complex programmes encounter the unexpected. The integration failure that was not anticipated. The requirement clarification that changes what was assumed. The external dependency that does not resolve on the assumed timeline. A sustainable delivery system can absorb these events without catastrophic disruption — because it has the slack that provides the capacity to respond, the architectural properties that bound the impact of unexpected events, and the governance culture that surfaces problems early enough for options to exist. + +The unsustainable system has no slack. Every unexpected event is a crisis. Every integration failure is a programme-threatening recovery. Every unexpected dependency resolution is an emergency escalation. The unsustainable system appears productive right up to the moment it breaks — because it has optimised for apparent output at the cost of the resilience that sustained output requires. + +**The capacity to learn.** The engineering organisation that learns from its programmes improves over time. The one that does not repeats them. Learning requires the honest attribution of failure causes — the willingness to examine governance decisions as causes of programme outcomes rather than as context within which execution fails. It requires the slack to reflect, to investigate, to implement the changes that the reflection recommends. It requires the psychological safety that allows honest information to flow in the post-mortem as freely as it should have flowed during the programme. + +The organisation that does not learn becomes progressively more fragile. Each failure cycle slightly worsens the conditions for the next. The compounding is slow. Over years, it is decisive. The engineering organisation that managed complex programmes competently in its early years and struggles with the same class of programme a decade later has not been subjected to a change in the difficulty of engineering. It has experienced a decade of learning failures, each of which slightly degraded the conditions for the next programme. + +**The capacity to maintain the people who carry capability.** Engineering capability is embodied in people. The knowledge of how systems work, why they were designed as they were, what the constraints are and where they come from, what the previous teams tried and why they moved on from it — this knowledge lives in people. It is transferred through practice, through the collaborative work of building systems together, through the conversations that happen in the course of engineering that never make it into documentation. + +When the people who carry this knowledge leave — not just to other employers, but to the part of the organisation where their knowledge is no longer applied — the capability they embodied is lost. Not immediately. The system continues to function. The maintenance continues. The next feature is delivered. But the decisions that require the deep knowledge become progressively less well-informed. The architectural debt accumulates. The constraints that the departed engineers held in their heads are violated by teams who cannot see them. The system becomes harder to change safely, more expensive to maintain, and less capable of evolution — not because engineering has gotten harder, but because the knowledge required to do it well has dispersed. + +Sustainability is the protection of these three capacities — variation absorption, learning, and capability retention — against the short-term pressures that consistently sacrifice them for apparent delivery pace. + +--- + +## What erodes over time + +Understanding what erodes — what the long view reveals about the cost of the short-term decisions that engineering organisations consistently make — is the prerequisite for protecting against the erosion. + +**Slack erodes under efficiency pressure.** Every budget cycle creates pressure to reduce the apparent waste of unallocated capacity. Every delivery crisis creates pressure to allocate the slack to recovery. Slack that is not explicitly protected and explicitly justified will be consumed — by the "just one more request" that seems reasonable in isolation, by the cost-cutting that treats unallocated capacity as obvious inefficiency, by the crisis response that borrows against the future without acknowledging the borrowing. + +The erosion of slack is slow. Teams at 80% utilisation can absorb most variation. Teams at 90% can absorb some. Teams at 95% can absorb almost none. The transition from the first to the third happens gradually, through many small decisions, each of which seems defensible. The consequence — the inability to respond to the unexpected without crisis, the inability to learn without heroics, the inability to maintain quality under pressure — appears suddenly, when the system is stressed in a way that the remaining slack cannot absorb. + +**Governance culture erodes under delivery pressure.** The first time a programme board confirms a delivery date without addressing the risk register items, it feels like a pragmatic decision — the risks are on the register, they are being managed, the date is commercially necessary, the board needs to move on. The second time, it feels like normal governance. The fifth time, it is normal governance. The governance culture has drifted from the one in which risk escalations generate decisions to the one in which they generate notes — not through a single decision but through the accumulation of small accommodations to delivery pressure. + +The restoration of governance culture is harder than its erosion. The first time a programme board spends forty-five minutes on three risk items generating decisions and owners and trigger dates, it feels unusual and uncomfortable. The second time, slightly less so. The restoration requires sustained leadership behaviour change across enough governance meetings that the new behaviour becomes the norm rather than the exception. + +**Technical debt accumulates at an accelerating rate.** The first year of delivery shortcuts produces manageable debt. The second year produces more, and the cost of carrying the first year's debt has already started to compound. By year five, the cost of each new feature includes the cost of navigating the accumulated debt of the previous years. The architecture that was designed for the product's original scope has been stretched to accommodate capabilities it was not designed for. The test coverage that was adequate for the first release has not kept pace with the codebase it was supposed to protect. + +The accumulation is invisible on any given day. The derivative — the rate at which the cost of delivery is increasing — is visible only when you look across years rather than sprints. The programme that is significantly slower, more expensive, and more fragile than it was two years ago has not encountered a change in the difficulty of what it is building. It has accumulated the compound interest of two years of unmanaged technical debt. + +**The people who hold the knowledge leave.** Not all at once. Not dramatically. One by one, for reasons that are individually understandable and collectively consequential. The principal engineer who was the architecture's institutional memory accepts a better offer because the organisation's technical career path did not reflect the value of what they held. The senior developer who has been on the programme since the beginning leaves because the governance culture has made honest engineering contribution unrewarding. The systems integrator who understood the dependencies between every component departs when the programme ends and the organisation has no mechanism for preserving what they knew before they go. + +Each departure is a risk event. Most are not treated as one. The knowledge that leaves with each person is not entered in the risk register. The impact on the programme's institutional capability is not assessed. The next team encounters the constraints the departed person held as invisible walls — discoverable only through the failures that violate them. + +--- + +## The compounding nature of doing it right + +The long view cuts both ways. The costs of the short-term optimisations described above compound negatively over time. The benefits of the disciplines described in this book compound positively. + +A programme that invests in honest estimation builds a history of estimation accuracy. That history changes what the governance layer believes about engineering estimates — not through argument but through evidence. The second programme benefits from a governance layer that has learned, from the first, that the honest estimate was more accurate than the challenged one. The third programme benefits from a governance layer that treats the most likely case as the planning basis without requiring the challenge that the first programme required. + +A programme that manages its risk register as a decision instrument produces governance decisions that keep risks from materialising at their worst-case cost. Those decisions produce better outcomes. The better outcomes build the governance layer's confidence in the risk management process. The confidence reduces the political resistance to acting on subsequent risk escalations. The programme that governs through its risk register more effectively in year three than it did in year one has not improved because it adopted a better risk register template. It has improved because the governance layer has experienced the benefit of treating the register as a decision instrument rather than a reporting artefact. + +A programme that maintains its technical debt register honestly and allocates budget for resolution delivers the next feature faster than the programme that does not — because it is not navigating the accumulated debt of previous decisions. The faster delivery is not attributable to better engineers or better tools. It is attributable to the governance decision, made in an earlier delivery cycle, to manage the debt rather than accumulate it. + +These are the compounding returns on the governance disciplines the framework describes. They are invisible in any single programme. Across five years and multiple programmes, they are the difference between an engineering organisation that is getting better and one that is getting worse — not because of the quality of its people but because of the quality of its systems. + +--- + +## Leadership continuity and institutional memory + +The long view requires a specific leadership challenge that most organisations handle badly: the transition of institutional knowledge when leaders change. + +Engineering programme governance is partly the governance of technical systems — the architecture, the delivery model, the risk register. But it is also the governance of understanding — the shared context that allows a governance layer to ask the right questions, interpret the answers honestly, and make decisions that reflect what the programme actually requires rather than what the narrative says it requires. + +When the leader who has built that understanding moves on, the understanding does not transfer automatically with the role. The new leader inherits the artefacts — the governance processes, the reporting structure, the programme plan — but not the tacit knowledge of what the artefacts represent, why specific risks are on the register rather than others, what the history of the programme's estimation accuracy tells them about the current estimate's reliability. + +The result is a governance layer that performs governance correctly without the understanding that makes governance effective. The risk review occurs at the right cadence. The questions asked are the right questions. But the answers are interpreted by someone who cannot yet distinguish between managed risk and documented risk, between a programme that is in better shape than it looks and one that is in worse shape. + +This is an argument for governance continuity where it is possible, for deliberate knowledge transfer where continuity is not possible, and for the governance documentation practices described in Chapter 18 — not as bureaucratic requirements but as the mechanism by which institutional knowledge is made accessible to the governance layer that comes after the one that built it. + +The ADR that captures not just the architectural decision but the context in which it was made. The risk register entry that includes not just the risk but the history of how it was managed. The programme retrospective that records not just what happened but what was learned and what was changed as a result. These artefacts are not documentation for documentation's sake. They are the transfer mechanism for the institutional understanding that the long view requires. + +--- + +## The ethical dimension + +There is an ethical dimension to the long view that engineering leaders rarely acknowledge explicitly and that this book has circled without naming directly. + +The technical debt that accumulates in a system will be maintained by engineers who were not there when it was created. The architectural decisions made under pressure will constrain teams who had no part in making them. The knowledge that is lost when key engineers leave without adequate transfer will be painfully reconstructed by teams who must do the archaeology to understand what they have inherited. + +Engineering stewardship extends beyond the programme that created these conditions to the teams that will inherit them. The decisions made in the programme are not just programme decisions. They are decisions about the environment that future engineers will work in — the constraints they will navigate, the debt they will manage, the knowledge gaps they will fill, the architectural problems they will solve that could have been prevented by different decisions earlier. + +This is not a call for perfect engineering or for the elimination of all compromise. Compromises are inevitable. Technical debt is sometimes the right choice. Architectural decisions made under conditions of genuine uncertainty will sometimes prove to be wrong. The ethical obligation is not to avoid all imperfection. It is to make the imperfections visible, to manage the compromises deliberately, and to leave the systems that future teams inherit in better condition than they found them — or at least in no worse condition, with the limitations they carry honestly acknowledged rather than hidden. + +Engineering as stewardship, not exploitation. The long view is what makes that distinction visible. + +--- + +## What the programme in Chapter 1 would look like today + +Return to the programme one final time. Not the programme that failed. The programme that did not have to fail. + +The commitment was made after the engineering assessment. The team produced a three-point estimate — best case twelve months, most likely seventeen, worst case twenty-two — and the governance layer accepted the most likely case as the planning basis. The three red items in the risk register generated decisions at the first programme board. The hardware-software interface ambiguity was resolved with the client by week five. The third-party component was validated by the end of month two. The test environment was procured in week four and available in month three. + +The programme delivered in eighteen months. One month beyond the most likely estimate — the firmware team encountered an edge case in the interface specification that required a two-week investigation. The risk register had priced this possibility. The programme board adjusted the delivery date by two weeks when the issue surfaced, rather than discovering an eight-month overrun at month six. + +The client was not surprised. The programme had told them in month one that the most likely delivery was seventeen months, that delivery in fourteen months would require specific conditions to hold, and that they would be told immediately if any of those conditions changed. The two-week extension was communicated at the point of discovery, with a clear explanation of the cause and the resolution. The client had planned their dependent programme against the most likely estimate, not the best case, and the two-week extension was manageable. + +The organisation that ran this programme learned something different from the programme in Chapter 1. It learned that honest estimates are more accurate than challenged ones. It learned that risk registers generate decisions. It learned that the gap between the committed date and the honest estimate is information to be managed rather than suppressed. It learned these things not from a framework document but from a programme that demonstrated them in practice. + +The next programme began with those lessons available. The governance layer that had learned from the first programme approached the second one differently. The estimation was accepted without compression. The risk register was managed rather than filed. The delivery was not perfect — complex programmes are never perfect — but it was honest, and the outcomes it produced were attributable to the reality of what the programme was building rather than to the fiction of what the programme had claimed to be building. + +That is the compounding return on the long view. Not perfection. Not the elimination of all programme failure. The progressive improvement of the conditions under which engineering work happens — so that the engineering teams that carry the knowledge and capability to build complex systems can do so in conditions that respect what they know, use what they can do, and produce outcomes that reflect what engineering, at its best, is capable of. + +That is what this book has been about. + +--- + +## A final note + +The people who will most benefit from this book are the ones who have been in the room when the risk register was filed rather than acted on. Who produced the honest estimate that was compressed. Who raised the concern at month two and were told to find a way. Who watched the programme fail in exactly the way they predicted, and were held accountable for it. + +This book does not give those people authority they do not have. It gives them a vocabulary, a structural argument, and a diagnostic framework for advocating for the conditions that would change the outcome. And it gives the people around them — the leaders who made the governance decisions — a framework for understanding what those decisions cost, and what different decisions would have produced. + +The programmes that fail predictably are produced by systems. The systems can be changed. The change requires leadership that is willing to govern differently in the specific moments when governance matters — not in principle, but in practice, in the programme board meetings where risk items either generate decisions or get noted, in the estimation reviews where honest numbers are either accepted or challenged, in the failure retrospectives where accountability is either accurately attributed or conveniently displaced. + +That willingness is rare. It is also, as this book has argued, the only thing that has ever actually worked. diff --git a/content/firmitas/part-3-making-it-stick/index.md b/content/firmitas/part-3-making-it-stick/index.md new file mode 100644 index 0000000..df95aa2 --- /dev/null +++ b/content/firmitas/part-3-making-it-stick/index.md @@ -0,0 +1,29 @@ +--- +title: "Part Three: Making It Stick" +--- + +# Part Three: Making It Stick + +A framework that works on paper but fails in practice is not a framework. It is a proposal. + +Part Two described the structural elements of Firmitas - the governance model, the estimation approach, the risk and issues treatment, the delivery model, the quality and compliance practices, the team design principles, and the rest. Those elements are coherent. They are grounded in the failure diagnosis of Part One. They are designed to address the specific conditions that produce the specific failures this book is built around. + +But coherence on paper and function in a real organisation are different things. Real organisations have history, politics, inertia, and incentive structures that were not designed with this framework in mind. Leaders who will need to change their behaviour to make the framework work may not yet know they need to change. Adoption attempts will go wrong in predictable ways. And the framework itself has limits - contexts in which it will not work, conditions under which it will be misused, and honest acknowledgements of what it cannot fix. + +Part Three addresses all of this directly. + +Chapter 24 is the hardest chapter in the book. It is addressed to the leader who recognises their own behaviour in what Part One described - the one who set the committed date, who noted the risk register, who applied the pressure that converted an honest estimate into a convenient one. It does not accuse. It diagnoses. And it describes, specifically, what changing that behaviour requires. + +Chapter 25 provides the Firmitas Delivery Diagnostic - twenty-six questions that assess how a real programme compares to what the framework requires. The diagnostic is designed to be uncomfortable to answer honestly. That discomfort is useful. It points directly at where to start. + +Chapter 26 catalogues the failure modes that occur when Firmitas is adopted without the philosophy behind it - when the process layer is installed while the system that produces the failure remains unchanged. These are the patterns to watch for in the first six months of any adoption attempt. + +Chapter 27 addresses measurement - what to track, what to avoid tracking, and how to measure system health without creating the perverse incentives that most measurement systems generate. + +Chapter 28 is the honest limits chapter. What Firmitas cannot fix. Who it is not for. The conditions under which it will fail even when applied sincerely and competently. A framework that cannot name its own failure modes is not credible. + +Chapter 29 closes the book with the long view - what sustainable delivery looks like over years rather than programmes, and what it means to build an engineering organisation that can deliver well not just once but repeatedly, through personnel changes, market shifts, and the accumulated pressure of an industry that will always find new ways to ask for the impossible by Friday. + +Part Three does not soften what Part One and Part Two have argued. It extends the argument into the territory that most frameworks avoid - adoption failure, honest limits, and the specific human and organisational reality of trying to change a system from inside it. + +Read it with the same honesty that Part One asked for. diff --git a/content/glossary/a-c.md b/content/glossary/a-c.md deleted file mode 100644 index 58c3b13..0000000 --- a/content/glossary/a-c.md +++ /dev/null @@ -1,917 +0,0 @@ ---- -title: Glossary, A to C ---- - -# A to C - -## A - -### A-law -A companding algorithm that squeezes the dynamic range of speech before it is coded as 8-bit PCM. It is one of the two variants of ITU-T G.711, used in Europe and most of the world outside North America and Japan. - -See also: [Mu-law](l-n.md#mu-law), [G.711](g-k.md#g711), [PCM](o-r.md#pcm). - -### AA -**Authoritative Answer.** A flag in a DNS response that says the server answering owns the zone for the name asked about, rather than passing on a cached answer. - -See also: [DNS](d-f.md#dns), [Authoritative Answer](#authoritative-answer). - -### AAC -**Advanced Audio Coding.** A lossy audio coding format, standardised in MPEG-2 and MPEG-4 as the successor to MP3. At the same bit rate it usually sounds better than MP3. HE-AAC (AAC+) is a variant tuned for low bit rates, used in digital radio such as DAB+. - -See also: [MP3](l-n.md#mp3), [MPEG](l-n.md#mpeg), [Audio Codecs](#audio-codecs). - -### AAL1 -**ATM Adaptation Layer 1.** The ATM adaptation layer for constant bit rate, connection-oriented traffic with tight timing, such as uncompressed voice and circuit emulation. It does not retransmit lost cells; it reports them and leaves recovery to the application. - -See also: [ATM](#atm), [AAL2](#aal2), [AAL5](#aal5). - -### AAL2 -**ATM Adaptation Layer 2.** The ATM adaptation layer for low-delay, variable bit rate traffic, mainly compressed voice. It packs short packets from several users into one ATM connection, and was used on the Iu interfaces of UMTS. It is defined in ITU-T I.363.2. - -See also: [ATM](#atm), [ALCAP](#alcap), [UMTS](s-z.md#umts). - -### AAL5 -**ATM Adaptation Layer 5.** The ATM adaptation layer for variable-length data packets of up to 65,535 octets, and the one most used for carrying IP over ATM. It puts its control information in an 8-octet trailer, including a 32-bit CRC, rather than a header. - -See also: [ATM](#atm), [CRC](#crc). - -### Acceptance Test -A formal test, agreed with the customer, whose result decides whether a release is accepted. - -See also: [Acceptance Testing](#acceptance-testing), [ATDD](#atdd). - -### Acceptance Testing -Testing that checks a system against the behaviour its customer or users expect, usually written as concrete examples or scenarios. The aim is often to automate those examples so they run with every build. - -See also: [Acceptance Test](#acceptance-test), [ATDD](#atdd), [BDD](#bdd). - -### Activity -A piece of work in a project that uses time and resources. Activities and tasks are near synonyms. PMBOK treats tasks as what an activity breaks down into. - -See also: [Task](s-z.md#task), [Work Breakdown Structure](s-z.md#work-breakdown-structure). - -### Activity Model -A model, such as a UML activity diagram, that shows the steps of a process or workflow and the flow of control between them. - -See also: [UML](s-z.md#uml), [Model](l-n.md#model). - -### AD -**Active Directory.** Microsoft's directory service for Windows domain networks. Domain controllers running Active Directory Domain Services authenticate users and computers, and apply security policy across the domain. - -See also: [LDAP](l-n.md#ldap), [Kerberos](g-k.md#kerberos). - -### Ad Hoc Testing -See [Informal Testing](g-k.md#informal-testing). - -### Ada -A strongly and statically typed programming language, descended from Pascal, with built-in support for concurrency, contracts and protected objects. It is used where safety matters, such as avionics and rail. The language is standardised as ISO/IEC 8652. - -See also: [Pascal](o-r.md#pascal). - -### ADR -**Architecture Decision Record.** A short document that records one significant architecture decision: its context, the options considered, the choice made and its consequences. ADRs are kept with the code and are not edited once accepted; a later ADR supersedes an earlier one. - -### ADSL -**Asymmetric Digital Subscriber Line.** A form of DSL that carries broadband over an ordinary copper telephone line, with more bandwidth downstream to the customer than upstream. - -See also: [DSL](d-f.md#dsl), [SDSL](s-z.md#sdsl), [VDSL](s-z.md#vdsl), [DSLAM](d-f.md#dslam). - -### AES -**Advanced Encryption Standard.** A symmetric block cipher, chosen by NIST in 2001 from the Rijndael family. It has a 128-bit block and keys of 128, 192 or 256 bits, and replaced DES as the default cipher for bulk encryption. - -See also: [DES](d-f.md#des), [Triple DES](s-z.md#triple-des), [NIST](l-n.md#nist). - -### Agent -In a project charter, another name for the [Project Sponsor](o-r.md#project-sponsor). - -### Agile -An approach to projects that delivers requirements in small increments, with frequent feedback, rather than in one pass at the end. - -See also: [Agile development](#agile-development), [Agile Manifesto](#agile-manifesto), [Scrum](s-z.md#scrum). - -### Agile development -An umbrella term for iterative, incremental ways of developing software, such as Scrum, XP, Lean and DSDM. - -See also: [Agile](#agile), [Scrum](s-z.md#scrum), [XP](s-z.md#xp), [DSDM](d-f.md#dsdm). - -### Agile Manifesto -The 2001 statement of the values behind agile development. It prefers individuals and interactions to processes and tools, and working software to comprehensive documentation. It also prefers customer collaboration to contract negotiation, and responding to change to following a plan. - -See also: [Agile](#agile). - -### AIS -**Alarm Indication Signal.** A signal, all ones, that a transmission element sends downstream in place of traffic when the link feeding it has failed. It keeps the framing intact and tells equipment further along that the fault is upstream. - -See also: [SDH](s-z.md#sdh), [PDH](o-r.md#pdh). - -### ALCAP -**Access Link Control Application Part.** The UMTS control protocol that sets up and releases AAL2 connections in the radio access network, multiplexing many users onto one AAL2 path. It is the 3GPP version of ITU-T Q.2630.2. - -See also: [AAL2](#aal2), [UTRAN](s-z.md#utran). - -### Amazon EC2 -**Amazon Elastic Compute Cloud.** The AWS service that rents virtual machines, called instances, started from machine images and paid for by the second while they run. - -See also: [AWS](#aws), [IaaS](g-k.md#iaas). - -### AMP -**Asymmetric Multiprocessing.** A multiprocessor design in which the processors have different roles, for example one running the operating system and the others running dedicated tasks. It is common in embedded systems that pair a general-purpose core with a real-time one. - -See also: [SMP](s-z.md#smp), [RTOS](o-r.md#rtos). - -### AMR-NB -**Adaptive Multi-Rate Narrowband.** The 3GPP speech codec for narrowband (300 to 3,400 Hz) speech, at eight bit rates from 4.75 to 12.2 kbit/s. The network picks the rate to suit the radio conditions. It is the standard codec for GSM and UMTS voice. - -See also: [AMR-WB](#amr-wb), [GSM AMR](g-k.md#gsm-amr), [Audio Codecs](#audio-codecs). - -### AMR-WB -**Adaptive Multi-Rate Wideband.** The wideband (50 to 7,000 Hz) version of AMR, also standardised as ITU-T G.722.2. It gives the "HD voice" quality used in VoLTE. - -See also: [AMR-NB](#amr-nb), [VoLTE](s-z.md#volte), [G.722](g-k.md#g722). - -### Analogy Estimation -Estimating new work by comparing it with similar work done before, and adjusting for the differences. - -See also: [Estimation](d-f.md#estimation), [Expert Judgement](d-f.md#expert-judgement). - -### Android -Google's mobile operating system, built on the Linux kernel and used on most smartphones and tablets, as well as TVs, cars and watches. - -See also: [Linux](l-n.md#linux). - -### ANDSF -**Access Network Discovery and Selection Function.** A 3GPP function that tells a mobile device which non-3GPP access networks, such as Wi-Fi, are nearby. It also gives the device policies for choosing between them and the mobile network. - -See also: [E-UTRAN](d-f.md#e-utran), [Wi-Fi](s-z.md#wi-fi). - -### ANSI -**American National Standards Institute.** The private, non-profit body that coordinates voluntary standards in the United States and represents the country at ISO. - -See also: [ISO](g-k.md#iso). - -### Antipattern -A common response to a recurring problem that looks reasonable but usually makes things worse. - -See also: [Pattern](o-r.md#pattern), [Design Pattern](d-f.md#design-pattern). - -### Anycast -An addressing and routing method in which one address is announced from several places, and the network delivers each packet to the nearest one by its routing metrics. DNS root servers and content delivery networks use it. - -See also: [Unicast](s-z.md#unicast), [Multicast](l-n.md#multicast), [Broadcast](#broadcast), [Geocast](g-k.md#geocast). - -### AOP -**Aspect-Oriented Programming.** A programming paradigm that separates cross-cutting concerns, such as logging or security checks, from the main logic. Code called advice is added at points chosen by a pattern called a pointcut, without editing the code it applies to. - -See also: [Programming paradigm](o-r.md#programming-paradigm), [OO](o-r.md#oo). - -### API -**Application Programming Interface.** A defined set of calls, messages or endpoints through which one piece of software uses another. - -See also: [REST](o-r.md#rest), [SDK](s-z.md#sdk). - -### APN -**Access Point Name.** The name a mobile device gives when it opens a data connection. It tells the operator which packet data network to connect it to, such as the public internet or a private company network, and so which addresses and policies apply. - -See also: [GGSN](g-k.md#ggsn), [PGW](o-r.md#pgw), [GPRS](g-k.md#gprs). - -### Architecture -The top-level structure of a system: its main parts, how they relate, and the principles behind its design. - -See also: [ADR](#adr), [High Level Design](g-k.md#high-level-design), [TOGAF](s-z.md#togaf). - -### ARP spoofing -An attack on a local network in which the attacker sends forged ARP replies. Other hosts then link the attacker's MAC address to someone else's IP address, often the default gateway's. Traffic then flows through the attacker, who can read, change or drop it. It is also called ARP cache poisoning. - -See also: [RARP](o-r.md#rarp), [MAC Spoofing](l-n.md#mac-spoofing), [DNS Spoofing](d-f.md#dns-spoofing). - -### Artefact -A tangible product of work, such as a document, a model, source code or a built system. - -See also: [Deliverable](d-f.md#deliverable), [Work Product](s-z.md#work-product). - -### ASCII -**American Standard Code for Information Interchange.** A 7-bit character encoding for English text, control codes and common symbols. UTF-8 and most other modern encodings keep ASCII as their first 128 characters. - -See also: [Unicode](s-z.md#unicode), [UTF-8](s-z.md#utf-8). - -### ASN.1 {#asn1} -**Abstract Syntax Notation One.** A language for describing data structures independently of how they are encoded, together with standard encoding rules such as BER, DER and PER. It is widely used in telecoms protocols and in cryptography, for example in X.509 certificates. - -See also: [BER](#ber), [DER](d-f.md#der), [PER](o-r.md#per), [CER](#cer), [X.509](s-z.md#x509). - -### Assessment -A review of a project's or an organisation's state or practices, often by someone independent. - -See also: [Audit](#audit), [Review](o-r.md#review). - -### ATDD -**Acceptance Test-Driven Development.** A practice in which customer, developer and tester agree acceptance tests before the feature is built, and the tests then drive the work. - -See also: [TDD](s-z.md#tdd), [BDD](#bdd), [Three Amigos](s-z.md#three-amigos). - -### ATM -**Asynchronous Transfer Mode.** A connection-oriented switching technology that carries voice, video and data in fixed 53-byte cells. It was the basis of broadband ISDN and was widely used in carrier backbones and DSL access networks before IP and Ethernet replaced it. - -See also: [AAL1](#aal1), [AAL2](#aal2), [AAL5](#aal5), [VPI](s-z.md#vpi), [VCI](s-z.md#vci). - -### Audio Codecs -Software or hardware that encodes and decodes digital audio in a given coding format, usually compressing it. Speech codecs such as G.711, AMR and Opus are tuned for voice calls; music codecs such as MP3, AAC and Vorbis for general audio. - -See also: [Codecs](#codecs), [G.711](g-k.md#g711), [OPUS](o-r.md#opus), [AAC](#aac). - -### Audit -A formal, independent assessment, usually against a standard or a contract. - -See also: [Assessment](#assessment). - -### Author -In a review, the person who wrote the artefact under review and who represents its viewpoint. - -See also: [Review](o-r.md#review), [Inspection](g-k.md#inspection). - -### Authoritative Answer -A DNS response from a server that is an authority for the zone holding the name asked about. The AA flag in the response header marks it. - -See also: [AA](#aa), [DNS](d-f.md#dns). - -### Authority -In a project charter, the person who funds and champions the project: the [Project Sponsor](o-r.md#project-sponsor). - -### Automated Build -A build that runs from start to finish without anyone stepping in, using only what is in version control, so that anyone can repeat it at any time. - -See also: [Build](#build), [Continuous Integration](#continuous-integration). - -### Automated Testing -Running tests and checking their results with tools rather than by hand. - -See also: [Manual Testing](l-n.md#manual-testing), [Regression testing](o-r.md#regression-testing). - -### Awk -A small language for processing text line by line, found on every Unix-like system. An Awk script is a list of patterns and actions, and it is often used for one-line reports and data extraction in pipelines. - -See also: [Sed](s-z.md#sed), [Shell Scripting](s-z.md#shell-scripting). - -### AWS -**Amazon Web Services.** Amazon's cloud computing platform: virtual machines, storage, databases, networking and many managed services, rented on demand. - -See also: [Amazon EC2](#amazon-ec2), [Azure](#azure), [Google Cloud Platform](g-k.md#google-cloud-platform). - -### Azure -Microsoft's cloud computing platform, offering infrastructure, platform and software as a service. - -See also: [AWS](#aws), [Google Cloud Platform](g-k.md#google-cloud-platform), [IaaS](g-k.md#iaas), [PaaS](o-r.md#paas). - -## B - -### B2B -**Business-to-business.** Trade between businesses, such as a manufacturer buying parts or a retailer buying stock, as opposed to selling to consumers. - -See also: [B2C](#b2c). - -### B2C -**Business-to-consumer.** Selling goods or services directly to the people who use them. - -See also: [B2B](#b2b). - -### Backhaul -The links that connect the edge of a network, such as mobile cell sites or exchange equipment, back to the core. Commercially, it also means wholesale capacity bought from another operator for that purpose. - -See also: [RAN](o-r.md#ran), [Leased Line](l-n.md#leased-line). - -### Backlog -An ordered list of everything that might be needed to deliver an outcome, such as a product. The items at the top are the most valuable and the best understood. - -See also: [Backlog Grooming](#backlog-grooming), [Product Owner](o-r.md#product-owner), [User Stories](s-z.md#user-stories). - -### Backlog Grooming -Regular work by the product owner and the team to keep the backlog in order. They add and remove items, prioritise them, and break down the items at the top so they are ready to start. It is also called backlog refinement. - -See also: [Backlog](#backlog), [Definition of Ready](d-f.md#definition-of-ready). - -### Bandwidth -In networking, the maximum rate at which data can cross a path, in bits per second. In signal processing and radio, it means the range of frequencies a signal occupies, in hertz. - -See also: [Latency](l-n.md#latency), [Jitter](g-k.md#jitter). - -### Baseline -An agreed version of an artefact that later work builds on. Changes to a baselined artefact go through change control. - -See also: [Change Control](#change-control), [Configuration Management](#configuration-management). - -### Bash -The GNU Project's Unix shell, a free replacement for the Bourne shell first released in 1989. It is the default shell on most Linux distributions. - -See also: [Sh](s-z.md#sh), [Ksh](g-k.md#ksh), [Csh](#csh), [Shell Scripting](s-z.md#shell-scripting). - -### BASIC -**Beginner's All-purpose Symbolic Instruction Code.** A family of high-level languages designed in 1964 at Dartmouth College to be easy to learn. It was the language most home computers of the 1980s started with. - -See also: [VB](s-z.md#vb). - -### BAU -**Business as usual.** The normal day-to-day running of an organisation, as opposed to projects that bring change. - -### BDD -**Behaviour-Driven Development.** A practice in which the team describes how a system should behave through concrete examples, often in the Given, When, Then form, and uses them as automated tests. - -See also: [Gherkin](g-k.md#gherkin), [Given When Then](g-k.md#given-when-then), [TDD](s-z.md#tdd), [ATDD](#atdd). - -### Bench Test -A test run in the development environment on the part of the system being worked on. - -See also: [Unit Testing](s-z.md#unit-testing), [Developer Integration Test](d-f.md#developer-integration-test). - -### BER -**Basic Encoding Rules.** The original encoding rules for ASN.1, in which each value is written as a type, a length and the value itself. BER allows several encodings of the same value. - -See also: [ASN.1](#asn1), [CER](#cer), [DER](d-f.md#der), [TLV](s-z.md#tlv). - -### Best Practice -A practice or technique that experience shows to work well for a given goal or risk. - -### BFF -**Backend for Frontend.** An architecture pattern in which each kind of client, such as a mobile app or a web app, has its own server-side gateway. The gateway shapes the data and calls that client needs from the services behind it. - -See also: [Microservice](l-n.md#microservice), [API](#api). - -### BGCF -**Breakout Gateway Control Function.** In IMS, the SIP server that decides where a call leaves the IMS network for the circuit-switched telephone network, based on the dialled number. - -See also: [IMS](g-k.md#ims), [MGCF](l-n.md#mgcf), [CSCF](#cscf). - -### BGP -**Border Gateway Protocol.** The routing protocol that exchanges reachability information between autonomous systems on the internet. It is a path vector protocol, and routes are chosen by policy as well as by path. - -See also: [IGP](g-k.md#igp), [OSPF](o-r.md#ospf), [Router](o-r.md#router). - -### Bjarne Stroustrup -The Danish computer scientist who created C++. - -See also: [C++](#c_1). - -### Black-Box Testing -Testing a system through its interfaces, without looking at how it is built. Also called [Functional Testing](d-f.md#functional-testing). - -### Bluetooth -A short-range radio standard in the 2.4 GHz band for connecting devices such as headsets, keyboards and phones into personal area networks. The Bluetooth Special Interest Group manages it. - -See also: [ZigBee](s-z.md#zigbee), [Wi-Fi](s-z.md#wi-fi). - -### BNF -**Backus-Naur Form.** A notation for writing the grammar of a language as a set of production rules. Extended BNF (EBNF) and augmented BNF (ABNF), the variant used in RFCs, add shorthand for repetition and options. - -### BNG -**Broadband Network Gateway.** The router at the edge of a broadband provider's network that terminates subscriber sessions, applies policy and passes their traffic on to the provider's IP network. It is the successor to the BRAS. - -See also: [BRAS](#bras), [PPPoE](o-r.md#pppoe), [DSLAM](d-f.md#dslam). - -### BOOTP -**Bootstrap Protocol.** An early protocol, defined in RFC 951, with which a device gets an IP address and boot information from a server when it starts. DHCP grew out of it and uses the same UDP ports, 67 and 68. - -See also: [DHCP](d-f.md#dhcp), [RARP](o-r.md#rarp). - -### Boston Matrix -A two-by-two chart that sorts a company's products by market share and market growth into stars, cash cows, question marks and dogs. It helps decide where to invest. It is also called the growth-share matrix. - -### Bottom-Up Estimation -Estimating a system by breaking it into small pieces, estimating each one, and adding up the results. - -See also: [Top-Down Estimation](s-z.md#top-down-estimation), [Decomposition Estimation](d-f.md#decomposition-estimation), [Work Breakdown Structure](s-z.md#work-breakdown-structure). - -### BPEL -**Business Process Execution Language.** An OASIS XML language for describing business processes as orchestrations of web service calls. It is also written WS-BPEL. - -See also: [BPMN](#bpmn), [SOA](s-z.md#soa), [WS-BPEL](s-z.md#ws-bpel). - -### BPMN -**Business Process Model and Notation.** A graphical notation, maintained by the Object Management Group, for drawing business processes. Version 2.0 also defines how the diagrams execute. - -See also: [BPEL](#bpel), [UML](s-z.md#uml). - -### BRAS -**Broadband Remote Access Server.** The router that aggregates broadband sessions from DSLAMs and routes them on to the internet service provider's network. Newer designs call it a BNG. - -See also: [BNG](#bng), [DSLAM](d-f.md#dslam), [PPPoE](o-r.md#pppoe). - -### BRI -**Basic Rate Interface.** The ISDN access for homes and small businesses. It has two 64 kbit/s bearer (B) channels for voice or data and one 16 kbit/s D channel for signalling, often written 2B+D. In the UK, BT sold it as ISDN2. - -See also: [ISDN](g-k.md#isdn), [PRI](o-r.md#pri). - -### Broadcast -Sending a message to every receiver on a network segment at once, such as an Ethernet broadcast to all hosts on a LAN. - -See also: [Unicast](s-z.md#unicast), [Multicast](l-n.md#multicast), [Anycast](#anycast). - -### BSC -**Base Station Controller.** In GSM, the node that controls a group of base transceiver stations: it allocates radio channels, manages handovers between them, and concentrates their traffic towards the MSC. - -See also: [BTS](#bts), [MSC](l-n.md#msc), [GSM](g-k.md#gsm), [RNC](o-r.md#rnc). - -### BSS -**Business Support Systems.** The systems a telecoms operator uses to run its business with customers, such as ordering, billing and customer management. They sit beside the operations support systems (OSS). In mobile networks, BSS also stands for base station subsystem: the BTSs and BSCs. - -See also: [OSS](o-r.md#oss). - -### BT 21CN -**BT 21st Century Network.** BT's programme, from the mid-2000s, to replace its separate legacy networks with one IP and MPLS network, using multi-service access nodes (MSANs) in exchanges. The plan to move voice onto it was shelved at the time; the later switch-off of the PSTN finishes that job. - -See also: [MSAN](l-n.md#msan), [NGN](l-n.md#ngn), [PSTN](o-r.md#pstn). - -### BTNUP -**BT National User Part.** BT's variant of ISDN user part signalling. See [IUP](g-k.md#iup). - -### BTS -**Base Transceiver Station.** The radio equipment at a GSM cell site that talks to mobile phones. In UMTS the equivalent is the Node B, and in LTE the eNodeB. - -See also: [BSC](#bsc), [Node B](l-n.md#node-b), [E NodeB](d-f.md#e-nodeb). - -### Bug -A fault in software that makes it behave other than intended. - -See also: [Defect](d-f.md#defect). - -### Build -The process that turns source code and other assets into a runnable system. The word also means the result: a version of the system ready to test. - -See also: [Automated Build](#automated-build), [Build Environment](#build-environment). - -### Build Environment -A separate environment kept for making builds, usually on one or more dedicated build machines. - -See also: [Build Machine](#build-machine), [Development Environment](d-f.md#development-environment). - -### Build Machine -A computer used only to make software builds. - -See also: [Build Environment](#build-environment). - -### Build tool -A tool that automates building software, such as Make, Maven or Gradle. - -See also: [Build](#build). - -### Burndown Chart -A chart that shows the work remaining in an iteration or project against time. A burnup chart shows the work completed instead. - -See also: [Velocity](s-z.md#velocity), [Sprints](s-z.md#sprints). - -### Burst -A short transmission at a high rate, or a group of errors that arrive close together. - -### Business Agility Ownership -Business agility is an organisation's ability to notice change, inside or outside it, and respond quickly enough to keep delivering value to its customers. - -See also: [Agile](#agile). - -### Business Requirement -A high-level goal of the organisation or customer asking for a system: why it is wanted. Also called a [Why Requirement](s-z.md#why-requirement). - -See also: [Requirement](o-r.md#requirement), [Functional Requirement](d-f.md#functional-requirement). - -### Business Schedule -A high-level schedule of a project's main milestones and the business goals tied to them. It sets the top-down limits that the detailed schedule must fit. - -See also: [Detailed Schedule](d-f.md#detailed-schedule), [Milestone](l-n.md#milestone). - -### BYOD -**Bring Your Own Device.** A policy that lets staff use their own laptops, tablets and phones for work, including access to company systems and data. It brings security and support questions that the policy needs to answer. - -## C - -### C Language -A general-purpose, imperative programming language created by Dennis Ritchie at Bell Labs in the early 1970s to write Unix. Its constructs map closely to machine instructions, so it is still the main language for operating systems and embedded software. It is standardised by ISO as ISO/IEC 9899. - -See also: [C++](#c_1), [K & R C](g-k.md#k-r-c), [Embedded systems programming](d-f.md#embedded-systems-programming). - -### C++ {#c_1} -A general-purpose language, created by Bjarne Stroustrup as an extension of C, that adds classes, templates and other abstractions while keeping low-level control of memory. It is standardised by ISO, with new versions every three years. - -See also: [C Language](#c-language), [Bjarne Stroustrup](#bjarne-stroustrup). - -### C-Tag -**Customer VLAN tag.** In IEEE 802.1ad double tagging (Q-in-Q), the inner VLAN tag, set by the customer. The service provider adds an outer S-Tag. - -See also: [S-Tag](s-z.md#s-tag), [VLAN](s-z.md#vlan), [Dot1Q](d-f.md#dot1q). - -### C# {#c_2} -A general-purpose, object-oriented language from Microsoft, created around 2000 for the .NET platform. It also supports functional and generic programming, and is standardised as ECMA-334 and ISO/IEC 23270. - -See also: [.NET](s-z.md#net), [Java](g-k.md#java). - -### CA -**Certificate Authority.** A trusted organisation that issues digital certificates, signing each to bind a public key to the named subject. Browsers trust HTTPS sites because a CA they trust signed the site's certificate. - -See also: [X.509](s-z.md#x509), [TLS](s-z.md#tls), [HTTPS](g-k.md#https). - -### Cacti -An open-source web tool that polls devices, usually over SNMP, and graphs the results with RRDtool, typically to show network traffic and CPU load over time. - -See also: [SNMP](s-z.md#snmp). - -### Cadence -The regular rhythm of a team's work, such as the length of its sprints or its release cycle. - -See also: [Sprints](s-z.md#sprints), [Iteration](g-k.md#iteration), [Timebox](s-z.md#timebox). - -### Calendar Days -The elapsed time an activity takes on the calendar. How staff days turn into calendar days depends on how many people work on it, and when. - -See also: [Staff Days](s-z.md#staff-days). - -### CAMEL -**Customised Applications for Mobile networks Enhanced Logic.** A set of 3GPP standards, based on Intelligent Network ideas, that lets operators add services such as prepaid calling to GSM and UMTS, including while the subscriber is roaming. It uses the CAP protocol. - -See also: [CAP](#cap), [IN](g-k.md#in), [INAP](g-k.md#inap). - -### Camel Case -Writing compound words with no spaces and a capital letter at the start of each word after the first, as in `camelCase`. Pascal case (`PascalCase`) capitalises the first word too. Many coding standards use one or both for identifiers. - -See also: [Snake Case](s-z.md#snake-case), [Hungarian notation](g-k.md#hungarian-notation). - -### CAP -**CAMEL Application Part.** The signalling protocol that CAMEL uses between the switch and the service control point. It runs over TCAP in SS7. - -See also: [CAMEL](#camel), [TCAP](s-z.md#tcap), [SS7](s-z.md#ss7). - -### Capability Maturity Model -**CMM.** A model of software process maturity, developed by the Software Engineering Institute for the US military. It defines five levels: initial, repeatable, defined, managed and optimising. Each level names the key process areas an organisation must have in place. Capability Maturity Model Integration (CMMI) replaced it. - -### Cat 5 -**Category 5 cable.** Twisted-pair cable rated to 100 MHz, used for 100 Mbit/s Ethernet and telephony. Category 5e superseded it in 2001. - -See also: [Cat 5e](#cat-5e), [Cat 6](#cat-6), [Twisted pair](s-z.md#twisted-pair). - -### Cat 5e -**Category 5e cable.** An enhanced Category 5 cable with tighter limits on crosstalk, rated to 100 MHz and good for Gigabit Ethernet up to 100 m. - -See also: [Cat 5](#cat-5), [Cat 6](#cat-6). - -### Cat 6 -**Category 6 cable.** Twisted-pair cable rated to 250 MHz, with stricter crosstalk limits than Cat 5e. It carries 10 Gigabit Ethernet up to about 55 m; Category 6A, rated to 500 MHz, does so up to 100 m. - -See also: [Cat 5](#cat-5), [Cat 5e](#cat-5e), [Ethernet over twisted pair](d-f.md#ethernet-over-twisted-pair). - -### CBOR -**Concise Binary Object Representation.** A binary data format, defined in RFC 8949, based on the JSON data model but designed for small messages and small decoders. It is widely used in constrained IoT devices, with CoAP, and in security tokens. - -See also: [JSON](g-k.md#json), [LwM2M](l-n.md#lwm2m). - -### CCITT -**Comité Consultatif International Téléphonique et Télégraphique.** The former name, until 1993, of the [ITU-T](g-k.md#itu-t). - -### CCS -**Common Channel Signalling.** Telephone signalling carried on a separate channel shared by many voice circuits, rather than in or alongside each circuit. SS7 is the main example. The alternative is channel-associated signalling (CAS). - -See also: [SS7](s-z.md#ss7), [ISUP](g-k.md#isup). - -### CCTA -**Central Computer and Telecommunications Agency.** A former UK government agency that supported departments' computing and telecoms. It created ITIL and PRINCE. - -See also: [ITIL](g-k.md#itil), [PRINCE2](o-r.md#prince2). - -### CDMA -**Code-Division Multiple Access.** A channel access method in which many transmitters share the same frequency band at the same time, each spread with its own code. It is the basis of cdmaOne, CDMA2000 and, as wideband CDMA, UMTS. - -See also: [CDMA2000](#cdma2000), [W-CDMA](s-z.md#w-cdma), [OFDMA](o-r.md#ofdma). - -### CDMA2000 -The 3G family of mobile standards developed by 3GPP2 as the successor to cdmaOne (IS-95), used mainly in North America and South Korea. It competed with UMTS. - -See also: [CDMA](#cdma), [UMTS](s-z.md#umts). - -### CDR -**Call Detail Record.** A record made by a switch or other network element for each call or session, with details such as the numbers, start time, duration and outcome. CDRs feed billing, fraud detection and reporting. - -See also: [Mediation](l-n.md#mediation). - -### CE -**Customer Edge.** The customer's router that connects to the service provider's provider edge (PE) router in an MPLS VPN. It exchanges routes with the PE, statically or with a routing protocol. - -See also: [VRF](s-z.md#vrf), [L3VPN](l-n.md#l3vpn), [NPE](l-n.md#npe). - -### CER -**Canonical Encoding Rules.** A restricted form of BER for ASN.1 that allows only one encoding for each value, so that encoded data can be compared or signed. - -See also: [BER](#ber), [DER](d-f.md#der), [ASN.1](#asn1). - -### Ceremonies -The regular meetings of an agile process, such as sprint planning, the daily stand-up, the review and the retrospective. - -See also: [Daily Scrum](d-f.md#daily-scrum), [Sprint Planning](s-z.md#sprint-planning), [Sprint retrospective](s-z.md#sprint-retrospective). - -### CESG -**Communications-Electronics Security Group.** The part of GCHQ that was the UK government's technical authority for information assurance. Its role passed to the National Cyber Security Centre in 2016. - -See also: [GCHQ](g-k.md#gchq), [NCSC](l-n.md#ncsc), [IA](g-k.md#ia). - -### CFi -**Canonical Format Indicator.** A one-bit field in the IEEE 802.1Q VLAN tag that showed whether MAC addresses were in canonical form. It is always 0 on Ethernet, and later revisions renamed it the drop eligible indicator. - -See also: [Dot1Q](d-f.md#dot1q), [VLAN](s-z.md#vlan). - -### CGI -**Common Gateway Interface.** A standard way for a web server to run an external command to answer a request, passing it the request details and returning its output as the page. - -See also: [HTTP](g-k.md#http). - -### Change Control -The part of change management that decides which artefacts are controlled and makes sure changes to them are proposed, assessed and approved before they are made. The term is also used for change management as a whole. - -See also: [Change Control Board](#change-control-board), [Change Request](#change-request), [Explicit Change Control](d-f.md#explicit-change-control), [Implicit Change Control](g-k.md#implicit-change-control). - -### Change Control Board -**CCB.** The group that decides on change requests to artefacts under change control. - -See also: [Change Control](#change-control), [Change Request](#change-request). - -### Change Control Plan -A plan that sets out what change control applies to which project artefacts. - -See also: [Change Control](#change-control). - -### Change Management -Managing requested changes to a project's scope, features or artefacts in a controlled way. - -See also: [Change Control](#change-control), [Configuration Management](#configuration-management). - -### Change Request -**CR.** A request to change something under change control, usually to add, change or remove a requirement, or to change the project plan. - -See also: [Change Control Board](#change-control-board), [Software Change Request](s-z.md#software-change-request). - -### CHAP -**Challenge-Handshake Authentication Protocol.** A PPP authentication protocol, defined in RFC 1994. The server sends a challenge and the client answers with a hash of the challenge and a shared secret, so the secret never crosses the link. It is safer than PAP. - -See also: [PAP](o-r.md#pap), [PPP](o-r.md#ppp), [EAP](d-f.md#eap). - -### Chrome OS -Google's operating system, based on the Linux kernel, that uses the Chrome browser as its main interface and runs mostly web applications. - -See also: [Linux](l-n.md#linux), [Android](#android). - -### Circuit Switched -A network in which a dedicated path is set up between the two ends before they communicate, and held for the whole call, as in the traditional telephone network. - -See also: [Packet Switched](o-r.md#packet-switched), [PSTN](o-r.md#pstn), [TDM](s-z.md#tdm). - -### Class 4 Softswitch -A softswitch that routes large volumes of voice calls between carriers, the VoIP equivalent of a class 4 (trunk) exchange. What matters most is protocol support and conversion, transcoding and call capacity. - -See also: [Softswitch](s-z.md#softswitch), [Class 5 Softswitch](#class-5-softswitch). - -### Class 5 Softswitch -A softswitch that serves end users, the VoIP equivalent of a class 5 (local) exchange. It provides subscriber features such as IP PBX functions, call centres and voicemail. - -See also: [Softswitch](s-z.md#softswitch), [Class 4 Softswitch](#class-4-softswitch). - -### Class Model -A model of the static structure of an object-oriented system: its classes and how they relate. - -See also: [UML](s-z.md#uml), [OO](o-r.md#oo). - -### CLF -**Connectivity Session Location and Repository Function.** In the ETSI TISPAN next-generation network, the function that records each subscriber's access session, linking their IP address to their line and location. It is not to be confused with the IMS subscriber location function (SLF), which finds the right HSS for a user when there are several. - -See also: [NGN](l-n.md#ngn), [HSS](g-k.md#hss), [IMS](g-k.md#ims). - -### CMDB -**Configuration Management Database.** A store of the configuration items in an IT estate and the relationships between them, used in ITIL to understand the effect of changes and incidents. - -See also: [Configuration Item](#configuration-item), [ITIL](g-k.md#itil). - -### CNG -**Comfort Noise Generation.** Adding artificial background noise at the receiving end of a call during silences, when voice activity detection has stopped the sender transmitting. Without it, the line goes dead quiet and callers think the call has dropped. - -See also: [VAD](s-z.md#vad), [VoIP](s-z.md#voip). - -### COBOL -**Common Business-Oriented Language.** An English-like, compiled language designed in 1959 for business data processing. A great deal of banking and government software on mainframes is still written in it. - -See also: [Fortran](d-f.md#fortran). - -### Code and Fix Lifecycle -Developing a system from a rough idea through repeated rounds of coding and fixing, with little planning or design, until it seems ready to release. - -See also: [Lifecycle](l-n.md#lifecycle), [Waterfall](s-z.md#waterfall). - -### Codecs -A codec (coder-decoder) is a device or piece of software that encodes a signal or data stream for transmission or storage, usually compressing it. It also decodes it again for use. - -See also: [Audio Codecs](#audio-codecs), [MPEG](l-n.md#mpeg). - -### Coding -Writing the source code that defines how software behaves; the core activity of construction. - -See also: [Construction](#construction), [Source Code](s-z.md#source-code). - -### Coding Standard -See [Construction Standard](#construction-standard). - -### CoE -**Centre of Excellence.** A team or group that provides leadership, good practice, support and training in one area, such as a technology or a skill, across an organisation. - -### COFDM -**Coded Orthogonal Frequency-Division Multiplexing.** OFDM with forward error correction and interleaving added, so the signal survives multipath fading. Digital terrestrial TV (DVB-T) and DAB use it. - -See also: [OFDMA](o-r.md#ofdma), [DTT](d-f.md#dtt). - -### Collaboration Model -A model, such as a UML communication diagram, that shows a set of objects or roles and the messages they exchange. - -See also: [Sequence Model](s-z.md#sequence-model), [UML](s-z.md#uml). - -### Collaborative Construction -A way of building software in which a small group of developers work closely together on the same functionality. They talk often and informally, and share ownership of the code and its tests. - -See also: [Pair Programming](o-r.md#pair-programming), [Mob Programming](l-n.md#mob-programming), [Collective code ownership](#collective-code-ownership). - -### Collective code ownership -The convention that anyone on the team may change any part of the code, to finish a task, fix a defect or improve its structure. - -See also: [XP](s-z.md#xp), [Pair Programming](o-r.md#pair-programming). - -### COLO -**Colocation.** Housing equipment in another party's building, typically placing a provider's equipment in a carrier's exchange or a data centre. - -### Compatibility Test -See [Configuration Test](#configuration-test). - -### Compiled Language -A programming language whose usual implementations translate source code ahead of time into machine code, rather than interpret it. The line is blurred: many languages compile to bytecode that a virtual machine then interprets or compiles just in time. - -See also: [Compiler](#compiler), [Interpreted Language](g-k.md#interpreted-language). - -### Compiler -Software that translates code from one language, usually a high-level language, into another, usually machine code or bytecode. A typical compiler has a front end, a middle end and a back end. A cross-compiler produces code for a different machine from the one it runs on. - -See also: [Compiler Front end](#compiler-front-end), [Compiler Middle end](#compiler-middle-end), [Compiler Back end](#compiler-back-end), [Compiler preprocessor](#compiler-preprocessor), [Machine code](l-n.md#machine-code). - -### Compiler Back end -The stage of a compiler that turns the optimised intermediate representation into code for one target processor: choosing instructions, allocating registers and scheduling. - -See also: [Compiler](#compiler), [Compiler Middle end](#compiler-middle-end). - -### Compiler Front end -The stage of a compiler that reads the source language: it checks syntax and semantics, reports errors, and produces an intermediate representation for the later stages. - -See also: [Compiler](#compiler), [Compiler Middle end](#compiler-middle-end). - -### Compiler Middle end -The stage of a compiler that optimises the intermediate representation in ways that do not depend on the target processor, such as removing dead code and propagating constants. - -See also: [Compiler](#compiler), [Compiler Front end](#compiler-front-end), [Compiler Back end](#compiler-back-end). - -### Compiler preprocessor -A tool that transforms source code before compilation, for example the C preprocessor, which expands macros, includes files and handles conditional compilation. - -See also: [Compiler](#compiler), [C Language](#c-language). - -### Component -A part of a software system with a defined interface and responsibility. - -See also: [Component Test](#component-test). - -### Component Test -A test of one software component on its own, apart from the rest of the system. - -See also: [Unit Testing](s-z.md#unit-testing), [Integration Test](g-k.md#integration-test). - -### Cone of Uncertainty -The observation that estimates made early in a project can be wrong by a large factor either way. The range narrows as the project goes on and decisions are made. - -See also: [Estimation](d-f.md#estimation), [Progressive Elaboration](o-r.md#progressive-elaboration). - -### Configuration Item -**CI.** An artefact, or a group of them, that is placed under configuration management and tracked as a unit. - -See also: [Configuration Management](#configuration-management), [CMDB](#cmdb). - -### Configuration Management -**CM.** Identifying, recording, releasing and controlling the versions of a system's artefacts so that its state is known at every point. - -See also: [Configuration Item](#configuration-item), [Version Control](s-z.md#version-control), [Software Configuration Management](s-z.md#software-configuration-management). - -### Configuration Test -A test of how a system behaves on different platforms, settings and environments. - -### Construction -Turning a design into working software: detailed design, coding, unit testing, integration and debugging. - -See also: [Coding](#coding), [Construction Standard](#construction-standard). - -### Construction Environment -See [Development Environment](d-f.md#development-environment). - -### Construction Lead -The person responsible for construction on a project: integration, builds, the development environment, technology problems and deployment. - -### Construction Standard -The conventions and style developers follow when writing a system's source code. Also called a coding standard. - -See also: [Coding Standard](#coding-standard). - -### Construction Test Environment -See [Local Test Environment](l-n.md#local-test-environment). - -### Construction Testing -The tests a developer runs while building a component, to check changes both in the component and in the system around it. - -See also: [Unit Testing](s-z.md#unit-testing), [Developer Integration Test](d-f.md#developer-integration-test). - -### Continuous Deployment -Releasing every change that passes the automated pipeline straight to production, so that code reaches users soon after it is written. - -See also: [Continuous Integration](#continuous-integration), [DevOps](d-f.md#devops). - -### Continuous Integration -**CI.** Merging every developer's changes into a shared mainline several times a day, with an automated build and tests on each merge. The product is then always in a releasable state. - -See also: [Automated Build](#automated-build), [Continuous Deployment](#continuous-deployment). - -### Control Plane -The part of a network device or network that decides where traffic goes: running routing protocols and building the tables that the data plane uses to forward packets. - -See also: [Data Plane](d-f.md#data-plane), [Management Plane](l-n.md#management-plane), [User Plane](s-z.md#user-plane), [SDN](s-z.md#sdn). - -### Copyleft -Licensing that lets anyone copy, change and share a work on condition that copies and derived works are shared under the same terms. The GPL is the best-known copyleft licence. - -See also: [GPL](g-k.md#gpl), [GNU](g-k.md#gnu). - -### CoS -**Class of Service.** A marking in a frame or packet that puts it into a traffic class, so the network can give some classes priority over others. - -See also: [QoS](o-r.md#qos), [PCP](o-r.md#pcp), [DSCP](d-f.md#dscp). - -### COTS -**Commercial off-the-shelf.** Products bought ready-made, and configured or adapted, rather than built to order. - -See also: [Mil-COTS](l-n.md#mil-cots). - -### CPE -**Customer Premises Equipment.** Equipment at the customer's site that connects to the operator's network at the demarcation point, such as a router, modem or phone. - -See also: [NTU](l-n.md#ntu). - -### CPM -**Critical Path Method.** A scheduling technique that finds the longest chain of dependent activities in a project. That chain sets the shortest possible duration, and any delay on it delays the end date. - -See also: [PERT chart](o-r.md#pert-chart), [Gantt chart](g-k.md#gantt-chart). - -### CPS -**Carrier Pre-Selection.** A UK telephone service that sent a customer's calls through another carrier's network automatically, without dialling a prefix, while the line stayed with the incumbent. - -See also: [WLR](s-z.md#wlr), [LCR](l-n.md#lcr). - -### CRC -**Cyclic Redundancy Check.** An error-detecting code calculated as the remainder of a polynomial division of the data. The sender appends it; the receiver recalculates it and treats any mismatch as corruption. Ethernet's frame check sequence is a CRC-32. - -See also: [FCS](d-f.md#fcs), [Hash Function](g-k.md#hash-function). - -### CRC Cards -**Class-Responsibility-Collaborator cards.** Index cards used in object-oriented design sessions, one for each class, listing what the class is responsible for and which classes it works with. - -See also: [OO](o-r.md#oo), [Class Model](#class-model). - -### CSCF -**Call Session Control Function.** The SIP servers at the heart of IMS. The proxy CSCF (P-CSCF) is the device's first contact with the network. The interrogating CSCF (I-CSCF) finds the right serving CSCF for a user. The serving CSCF (S-CSCF) registers users and controls their sessions. - -See also: [IMS](g-k.md#ims), [SIP](s-z.md#sip), [HSS](g-k.md#hss). - -### Csh -The C shell, a Unix shell written by Bill Joy at Berkeley in the late 1970s, with a syntax modelled on C. Tcsh is its improved successor. - -See also: [Tcsh](s-z.md#tcsh), [Bash](#bash), [Sh](s-z.md#sh). - -### CSI -**Customer Site Interconnect.** A physical link between an operator's network and a customer's site or another operator's equipment. - -See also: [Leased Line](l-n.md#leased-line). - -### CSMA -**Carrier-Sense Multiple Access.** A media access method in which a device listens for traffic on a shared medium and transmits only when it is idle. Classic Ethernet added collision detection (CSMA/CD); Wi-Fi uses collision avoidance (CSMA/CA). - -See also: [MAC](l-n.md#mac), [Wi-Fi](s-z.md#wi-fi). - -### CSS -**Cascading Style Sheets.** The language that describes how HTML documents are presented, such as layout, colours and fonts, keeping presentation apart from content. - -See also: [XML](s-z.md#xml), [W3C](s-z.md#w3c). - -### CUE -**Configure Unify Execute.** An open-source language for validating, templating and generating configuration and data. It treats types and values as one thing, so a schema, a constraint and concrete data can be combined and checked in the same way. - -See also: [JSON](g-k.md#json), [YAML](s-z.md#yaml). - -### Customer Development -A four-step framework from Steve Blank (discovery, validation, creation and building the company) for testing assumptions about a product and its market by experiment. - -See also: [MVP](l-n.md#mvp). - -### CVSD -**Continuously Variable Slope Delta modulation.** A speech coding method that sends one bit per sample, saying whether the signal went up or down, with a step size that adapts to the signal. It is robust against bit errors and is used in military radio and Bluetooth voice. - -See also: [PCM](o-r.md#pcm), [Audio Codecs](#audio-codecs). diff --git a/content/glossary/a.md b/content/glossary/a.md new file mode 100644 index 0000000..ec72eed --- /dev/null +++ b/content/glossary/a.md @@ -0,0 +1,294 @@ +--- +title: A +--- + +# A + +## A-law +A companding algorithm that squeezes the dynamic range of speech before it is coded as 8-bit PCM. It is one of the two variants of ITU-T G.711, used in Europe and most of the world outside North America and Japan. + +See also: [Mu-law](m.md#mu-law), [G.711](g.md#g711), [PCM](p.md#pcm). + +## AA +**Authoritative Answer.** A flag in a DNS response that says the server answering owns the zone for the name asked about, rather than passing on a cached answer. + +See also: [DNS](d.md#dns), [Authoritative Answer](#authoritative-answer). + +## AAC +**Advanced Audio Coding.** A lossy audio coding format, standardised in MPEG-2 and MPEG-4 as the successor to MP3. At the same bit rate it usually sounds better than MP3. HE-AAC (AAC+) is a variant tuned for low bit rates, used in digital radio such as DAB+. + +See also: [MP3](m.md#mp3), [MPEG](m.md#mpeg), [Audio Codecs](#audio-codecs). + +## AAL1 +**ATM Adaptation Layer 1.** The ATM adaptation layer for constant bit rate, connection-oriented traffic with tight timing, such as uncompressed voice and circuit emulation. It does not retransmit lost cells; it reports them and leaves recovery to the application. + +See also: [ATM](#atm), [AAL2](#aal2), [AAL5](#aal5). + +## AAL2 +**ATM Adaptation Layer 2.** The ATM adaptation layer for low-delay, variable bit rate traffic, mainly compressed voice. It packs short packets from several users into one ATM connection, and was used on the Iu interfaces of UMTS. It is defined in ITU-T I.363.2. + +See also: [ATM](#atm), [ALCAP](#alcap), [UMTS](u.md#umts). + +## AAL5 +**ATM Adaptation Layer 5.** The ATM adaptation layer for variable-length data packets of up to 65,535 octets, and the one most used for carrying IP over ATM. It puts its control information in an 8-octet trailer, including a 32-bit CRC, rather than a header. + +See also: [ATM](#atm), [CRC](c.md#crc). + +## Acceptance Test +A formal test, agreed with the customer, whose result decides whether a release is accepted. + +See also: [Acceptance Testing](#acceptance-testing), [ATDD](#atdd). + +## Acceptance Testing +Testing that checks a system against the behaviour its customer or users expect, usually written as concrete examples or scenarios. The aim is often to automate those examples so they run with every build. + +See also: [Acceptance Test](#acceptance-test), [ATDD](#atdd), [BDD](b.md#bdd). + +## Activity +A piece of work in a project that uses time and resources. Activities and tasks are near synonyms. PMBOK treats tasks as what an activity breaks down into. + +See also: [Task](t.md#task), [Work Breakdown Structure](w.md#work-breakdown-structure). + +## Activity Model +A model, such as a UML activity diagram, that shows the steps of a process or workflow and the flow of control between them. + +See also: [UML](u.md#uml), [Model](m.md#model). + +## AD +**Active Directory.** Microsoft's directory service for Windows domain networks. Domain controllers running Active Directory Domain Services authenticate users and computers, and apply security policy across the domain. + +See also: [LDAP](l.md#ldap), [Kerberos](k.md#kerberos). + +## Ad Hoc Testing +See [Informal Testing](i.md#informal-testing). + +## Ada +A strongly and statically typed programming language, descended from Pascal, with built-in support for concurrency, contracts and protected objects. It is used where safety matters, such as avionics and rail. The language is standardised as ISO/IEC 8652. + +See also: [Pascal](p.md#pascal). + +## ADR +**Architecture Decision Record.** A short document that records one significant architecture decision: its context, the options considered, the choice made and its consequences. ADRs are kept with the code and are not edited once accepted; a later ADR supersedes an earlier one. + +## ADSL +**Asymmetric Digital Subscriber Line.** A form of DSL that carries broadband over an ordinary copper telephone line, with more bandwidth downstream to the customer than upstream. + +![ADSL](images/15007760.png) + +[[CAPTION:Figure]] ADSL. + +See also: [DSL](d.md#dsl), [SDSL](s.md#sdsl), [VDSL](v.md#vdsl), [DSLAM](d.md#dslam). + +## AES +**Advanced Encryption Standard.** A symmetric block cipher, chosen by NIST in 2001 from the Rijndael family. It has a 128-bit block and keys of 128, 192 or 256 bits, and replaced DES as the default cipher for bulk encryption. + +![AES](images/15007768.png) + +[[CAPTION:Figure]] AES. + +See also: [DES](d.md#des), [Triple DES](t.md#triple-des), [NIST](n.md#nist). + +## Agent +In a project charter, another name for the [Project Sponsor](p.md#project-sponsor). + +## Agile +An approach to projects that delivers requirements in small increments, with frequent feedback, rather than in one pass at the end. + +See also: [Agile development](#agile-development), [Agile Manifesto](#agile-manifesto), [Scrum](s.md#scrum). + +## Agile development +An umbrella term for iterative, incremental ways of developing software, such as Scrum, XP, Lean and DSDM. + +See also: [Agile](#agile), [Scrum](s.md#scrum), [XP](x.md#xp), [DSDM](d.md#dsdm). + +## Agile Manifesto +The 2001 statement of the values behind agile development. It prefers individuals and interactions to processes and tools, and working software to comprehensive documentation. It also prefers customer collaboration to contract negotiation, and responding to change to following a plan. + +See also: [Agile](#agile). + +## AIS +**Alarm Indication Signal.** A signal, all ones, that a transmission element sends downstream in place of traffic when the link feeding it has failed. It keeps the framing intact and tells equipment further along that the fault is upstream. + +See also: [SDH](s.md#sdh), [PDH](p.md#pdh). + +## ALCAP +**Access Link Control Application Part.** The UMTS control protocol that sets up and releases AAL2 connections in the radio access network, multiplexing many users onto one AAL2 path. It is the 3GPP version of ITU-T Q.2630.2. + +See also: [AAL2](#aal2), [UTRAN](u.md#utran). + +## Amazon EC2 +**Amazon Elastic Compute Cloud.** The AWS service that rents virtual machines, called instances, started from machine images and paid for by the second while they run. + +See also: [AWS](#aws), [IaaS](i.md#iaas). + +## AMP +**Asymmetric Multiprocessing.** A multiprocessor design in which the processors have different roles, for example one running the operating system and the others running dedicated tasks. It is common in embedded systems that pair a general-purpose core with a real-time one. + +![AMP](images/15007774.png) + +[[CAPTION:Figure]] AMP. + +See also: [SMP](s.md#smp), [RTOS](r.md#rtos). + +## AMR-NB +**Adaptive Multi-Rate Narrowband.** The 3GPP speech codec for narrowband (300 to 3,400 Hz) speech, at eight bit rates from 4.75 to 12.2 kbit/s. The network picks the rate to suit the radio conditions. It is the standard codec for GSM and UMTS voice. + +See also: [AMR-WB](#amr-wb), [GSM AMR](g.md#gsm-amr), [Audio Codecs](#audio-codecs). + +## AMR-WB +**Adaptive Multi-Rate Wideband.** The wideband (50 to 7,000 Hz) version of AMR, also standardised as ITU-T G.722.2. It gives the "HD voice" quality used in VoLTE. + +See also: [AMR-NB](#amr-nb), [VoLTE](v.md#volte), [G.722](g.md#g722). + +## Analogy Estimation +Estimating new work by comparing it with similar work done before, and adjusting for the differences. + +See also: [Estimation](e.md#estimation), [Expert Judgement](e.md#expert-judgement). + +## Android +Google's mobile operating system, built on the Linux kernel and used on most smartphones and tablets, as well as TVs, cars and watches. + +See also: [Linux](l.md#linux). + +## ANDSF +**Access Network Discovery and Selection Function.** A 3GPP function that tells a mobile device which non-3GPP access networks, such as Wi-Fi, are nearby. It also gives the device policies for choosing between them and the mobile network. + +![ANDSF](images/15007780.png) + +[[CAPTION:Figure]] ANDSF. + +See also: [E-UTRAN](e.md#e-utran), [Wi-Fi](w.md#wi-fi). + +## ANSI +**American National Standards Institute.** The private, non-profit body that coordinates voluntary standards in the United States and represents the country at ISO. + +See also: [ISO](i.md#iso). + +## Antipattern +A common response to a recurring problem that looks reasonable but usually makes things worse. + +See also: [Pattern](p.md#pattern), [Design Pattern](d.md#design-pattern). + +## Anycast +An addressing and routing method in which one address is announced from several places, and the network delivers each packet to the nearest one by its routing metrics. DNS root servers and content delivery networks use it. + +![Anycast](images/15007786.png) + +[[CAPTION:Figure]] Anycast. + +See also: [Unicast](u.md#unicast), [Multicast](m.md#multicast), [Broadcast](b.md#broadcast), [Geocast](g.md#geocast). + +## AOP +**Aspect-Oriented Programming.** A programming paradigm that separates cross-cutting concerns, such as logging or security checks, from the main logic. Code called advice is added at points chosen by a pattern called a pointcut, without editing the code it applies to. + +See also: [Programming paradigm](p.md#programming-paradigm), [OO](o.md#oo). + +## API +**Application Programming Interface.** A defined set of calls, messages or endpoints through which one piece of software uses another. + +See also: [REST](r.md#rest), [SDK](s.md#sdk). + +## APN +**Access Point Name.** The name a mobile device gives when it opens a data connection. It tells the operator which packet data network to connect it to, such as the public internet or a private company network, and so which addresses and policies apply. + +See also: [GGSN](g.md#ggsn), [PGW](p.md#pgw), [GPRS](g.md#gprs). + +## Architecture +The top-level structure of a system: its main parts, how they relate, and the principles behind its design. + +See also: [ADR](#adr), [High Level Design](h.md#high-level-design), [TOGAF](t.md#togaf). + +## ARP spoofing +An attack on a local network in which the attacker sends forged ARP replies. Other hosts then link the attacker's MAC address to someone else's IP address, often the default gateway's. Traffic then flows through the attacker, who can read, change or drop it. It is also called ARP cache poisoning. + +![ARP spoofing](images/15007792.png) + +[[CAPTION:Figure]] ARP spoofing. + +See also: [RARP](r.md#rarp), [MAC Spoofing](m.md#mac-spoofing), [DNS Spoofing](d.md#dns-spoofing). + +## Artefact +A tangible product of work, such as a document, a model, source code or a built system. + +See also: [Deliverable](d.md#deliverable), [Work Product](w.md#work-product). + +## ASCII +**American Standard Code for Information Interchange.** A 7-bit character encoding for English text, control codes and common symbols. UTF-8 and most other modern encodings keep ASCII as their first 128 characters. + +See also: [Unicode](u.md#unicode), [UTF-8](u.md#utf-8). + +## ASN.1 {#asn1} +**Abstract Syntax Notation One.** A language for describing data structures independently of how they are encoded, together with standard encoding rules such as BER, DER and PER. It is widely used in telecoms protocols and in cryptography, for example in X.509 certificates. + +See also: [BER](b.md#ber), [DER](d.md#der), [PER](p.md#per), [CER](c.md#cer), [X.509](x.md#x509). + +## Assessment +A review of a project's or an organisation's state or practices, often by someone independent. + +See also: [Audit](#audit), [Review](r.md#review). + +## ATDD +**Acceptance Test-Driven Development.** A practice in which customer, developer and tester agree acceptance tests before the feature is built, and the tests then drive the work. + +See also: [TDD](t.md#tdd), [BDD](b.md#bdd), [Three Amigos](t.md#three-amigos). + +## ATM +**Asynchronous Transfer Mode.** A connection-oriented switching technology that carries voice, video and data in fixed 53-byte cells. It was the basis of broadband ISDN and was widely used in carrier backbones and DSL access networks before IP and Ethernet replaced it. + +![ATM](images/15007798.png) + +[[CAPTION:Figure]] ATM. + +See also: [AAL1](#aal1), [AAL2](#aal2), [AAL5](#aal5), [VPI](v.md#vpi), [VCI](v.md#vci). + +## Audio Codecs +Software or hardware that encodes and decodes digital audio in a given coding format, usually compressing it. Speech codecs such as G.711, AMR and Opus are tuned for voice calls; music codecs such as MP3, AAC and Vorbis for general audio. + +![Audio Codecs](images/15007804.png) + +[[CAPTION:Figure]] Audio Codecs. + +See also: [Codecs](c.md#codecs), [G.711](g.md#g711), [OPUS](o.md#opus), [AAC](#aac). + +## Audit +A formal, independent assessment, usually against a standard or a contract. + +See also: [Assessment](#assessment). + +## Author +In a review, the person who wrote the artefact under review and who represents its viewpoint. + +See also: [Review](r.md#review), [Inspection](i.md#inspection). + +## Authoritative Answer +A DNS response from a server that is an authority for the zone holding the name asked about. The AA flag in the response header marks it. + +See also: [AA](#aa), [DNS](d.md#dns). + +## Authority +In a project charter, the person who funds and champions the project: the [Project Sponsor](p.md#project-sponsor). + +## Automated Build +A build that runs from start to finish without anyone stepping in, using only what is in version control, so that anyone can repeat it at any time. + +See also: [Build](b.md#build), [Continuous Integration](c.md#continuous-integration). + +## Automated Testing +Running tests and checking their results with tools rather than by hand. + +See also: [Manual Testing](m.md#manual-testing), [Regression testing](r.md#regression-testing). + +## Awk +A small language for processing text line by line, found on every Unix-like system. An Awk script is a list of patterns and actions, and it is often used for one-line reports and data extraction in pipelines. + +See also: [Sed](s.md#sed), [Shell Scripting](s.md#shell-scripting). + +## AWS +**Amazon Web Services.** Amazon's cloud computing platform: virtual machines, storage, databases, networking and many managed services, rented on demand. + +See also: [Amazon EC2](#amazon-ec2), [Azure](#azure), [Google Cloud Platform](g.md#google-cloud-platform). + +## Azure +Microsoft's cloud computing platform, offering infrastructure, platform and software as a service. + +See also: [AWS](#aws), [Google Cloud Platform](g.md#google-cloud-platform), [IaaS](i.md#iaas), [PaaS](p.md#paas). diff --git a/content/glossary/b.md b/content/glossary/b.md new file mode 100644 index 0000000..06abc5e --- /dev/null +++ b/content/glossary/b.md @@ -0,0 +1,259 @@ +--- +title: B +--- + +# B + +## B2B +**Business-to-business.** Trade between businesses, such as a manufacturer buying parts or a retailer buying stock, as opposed to selling to consumers. + +See also: [B2C](#b2c). + +## B2C +**Business-to-consumer.** Selling goods or services directly to the people who use them. + +See also: [B2B](#b2b). + +## Backhaul +The links that connect the edge of a network, such as mobile cell sites or exchange equipment, back to the core. Commercially, it also means wholesale capacity bought from another operator for that purpose. + +See also: [RAN](r.md#ran), [Leased Line](l.md#leased-line). + +## Backlog +An ordered list of everything that might be needed to deliver an outcome, such as a product. The items at the top are the most valuable and the best understood. + +See also: [Backlog Grooming](#backlog-grooming), [Product Owner](p.md#product-owner), [User Stories](u.md#user-stories). + +## Backlog Grooming +Regular work by the product owner and the team to keep the backlog in order. They add and remove items, prioritise them, and break down the items at the top so they are ready to start. It is also called backlog refinement. + +See also: [Backlog](#backlog), [Definition of Ready](d.md#definition-of-ready). + +## Bandwidth +In networking, the maximum rate at which data can cross a path, in bits per second. In signal processing and radio, it means the range of frequencies a signal occupies, in hertz. + +See also: [Latency](l.md#latency), [Jitter](j.md#jitter). + +## Baseline +An agreed version of an artefact that later work builds on. Changes to a baselined artefact go through change control. + +See also: [Change Control](c.md#change-control), [Configuration Management](c.md#configuration-management). + +## Bash +The GNU Project's Unix shell, a free replacement for the Bourne shell first released in 1989. It is the default shell on most Linux distributions. + +See also: [Sh](s.md#sh), [Ksh](k.md#ksh), [Csh](c.md#csh), [Shell Scripting](s.md#shell-scripting). + +## BASIC +**Beginner's All-purpose Symbolic Instruction Code.** A family of high-level languages designed in 1964 at Dartmouth College to be easy to learn. It was the language most home computers of the 1980s started with. + +See also: [VB](v.md#vb). + +## BAU +**Business as usual.** The normal day-to-day running of an organisation, as opposed to projects that bring change. + +## BDD +**Behaviour-Driven Development.** A practice in which the team describes how a system should behave through concrete examples, often in the Given, When, Then form, and uses them as automated tests. + +See also: [Gherkin](g.md#gherkin), [Given When Then](g.md#given-when-then), [TDD](t.md#tdd), [ATDD](a.md#atdd). + +## Bench Test +A test run in the development environment on the part of the system being worked on. + +See also: [Unit Testing](u.md#unit-testing), [Developer Integration Test](d.md#developer-integration-test). + +## BER +**Basic Encoding Rules.** The original encoding rules for ASN.1, in which each value is written as a type, a length and the value itself. BER allows several encodings of the same value. + +See also: [ASN.1](a.md#asn1), [CER](c.md#cer), [DER](d.md#der), [TLV](t.md#tlv). + +## Best Practice +A practice or technique that experience shows to work well for a given goal or risk. + +## BFF +**Backend for Frontend.** An architecture pattern in which each kind of client, such as a mobile app or a web app, has its own server-side gateway. The gateway shapes the data and calls that client needs from the services behind it. + +See also: [Microservice](m.md#microservice), [API](a.md#api). + +## BGCF +**Breakout Gateway Control Function.** In IMS, the SIP server that decides where a call leaves the IMS network for the circuit-switched telephone network, based on the dialled number. + +![BGCF](images/15007813.png) + +[[CAPTION:Figure]] BGCF. + +See also: [IMS](i.md#ims), [MGCF](m.md#mgcf), [CSCF](c.md#cscf). + +## BGP +**Border Gateway Protocol.** The routing protocol that exchanges reachability information between autonomous systems on the internet. It is a path vector protocol, and routes are chosen by policy as well as by path. + +See also: [IGP](i.md#igp), [OSPF](o.md#ospf), [Router](r.md#router). + +## Bjarne Stroustrup +The Danish computer scientist who created C++. + +See also: [C++](c.md#c_1). + +## Black-Box Testing +Testing a system through its interfaces, without looking at how it is built. Also called [Functional Testing](f.md#functional-testing). + +## Bluetooth +A short-range radio standard in the 2.4 GHz band for connecting devices such as headsets, keyboards and phones into personal area networks. The Bluetooth Special Interest Group manages it. + +See also: [ZigBee](z.md#zigbee), [Wi-Fi](w.md#wi-fi). + +## BNF +**Backus-Naur Form.** A notation for writing the grammar of a language as a set of production rules. Extended BNF (EBNF) and augmented BNF (ABNF), the variant used in RFCs, add shorthand for repetition and options. + +## BNG +**Broadband Network Gateway.** The router at the edge of a broadband provider's network that terminates subscriber sessions, applies policy and passes their traffic on to the provider's IP network. It is the successor to the BRAS. + +See also: [BRAS](#bras), [PPPoE](p.md#pppoe), [DSLAM](d.md#dslam). + +## BOOTP +**Bootstrap Protocol.** An early protocol, defined in RFC 951, with which a device gets an IP address and boot information from a server when it starts. DHCP grew out of it and uses the same UDP ports, 67 and 68. + +See also: [DHCP](d.md#dhcp), [RARP](r.md#rarp). + +## Boston Matrix +A two-by-two chart that sorts a company's products by market share and market growth into stars, cash cows, question marks and dogs. It helps decide where to invest. It is also called the growth-share matrix. + +## Bottom-Up Estimation +Estimating a system by breaking it into small pieces, estimating each one, and adding up the results. + +See also: [Top-Down Estimation](t.md#top-down-estimation), [Decomposition Estimation](d.md#decomposition-estimation), [Work Breakdown Structure](w.md#work-breakdown-structure). + +## BPEL +**Business Process Execution Language.** An OASIS XML language for describing business processes as orchestrations of web service calls. It is also written WS-BPEL. + +See also: [BPMN](#bpmn), [SOA](s.md#soa), [WS-BPEL](w.md#ws-bpel). + +## BPMN +**Business Process Model and Notation.** A graphical notation, maintained by the Object Management Group, for drawing business processes. Version 2.0 also defines how the diagrams execute. + +![BPMN](images/15007821.png) + +[[CAPTION:Figure]] BPMN. + +See also: [BPEL](#bpel), [UML](u.md#uml). + +## BRAS +**Broadband Remote Access Server.** The router that aggregates broadband sessions from DSLAMs and routes them on to the internet service provider's network. Newer designs call it a BNG. + +![BRAS](images/15007827.png) + +[[CAPTION:Figure]] BRAS. + +![BRAS](images/15007833.png) + +[[CAPTION:Figure]] BRAS. + +See also: [BNG](#bng), [DSLAM](d.md#dslam), [PPPoE](p.md#pppoe). + +## BRI +**Basic Rate Interface.** The ISDN access for homes and small businesses. It has two 64 kbit/s bearer (B) channels for voice or data and one 16 kbit/s D channel for signalling, often written 2B+D. In the UK, BT sold it as ISDN2. + +See also: [ISDN](i.md#isdn), [PRI](p.md#pri). + +## Broadcast +Sending a message to every receiver on a network segment at once, such as an Ethernet broadcast to all hosts on a LAN. + +![Broadcast](images/15007839.png) + +[[CAPTION:Figure]] Broadcast. + +See also: [Unicast](u.md#unicast), [Multicast](m.md#multicast), [Anycast](a.md#anycast). + +## BSC +**Base Station Controller.** In GSM, the node that controls a group of base transceiver stations: it allocates radio channels, manages handovers between them, and concentrates their traffic towards the MSC. + +![BSC](images/15007845.png) + +[[CAPTION:Figure]] BSC. + +See also: [BTS](#bts), [MSC](m.md#msc), [GSM](g.md#gsm), [RNC](r.md#rnc). + +## BSS +**Business Support Systems.** The systems a telecoms operator uses to run its business with customers, such as ordering, billing and customer management. They sit beside the operations support systems (OSS). In mobile networks, BSS also stands for base station subsystem: the BTSs and BSCs. + +![BSS](images/15007851.png) + +[[CAPTION:Figure]] BSS. + +See also: [OSS](o.md#oss). + +## BT 21CN +**BT 21st Century Network.** BT's programme, from the mid-2000s, to replace its separate legacy networks with one IP and MPLS network, using multi-service access nodes (MSANs) in exchanges. The plan to move voice onto it was shelved at the time; the later switch-off of the PSTN finishes that job. + +![BT 21CN](images/15007857.png) + +[[CAPTION:Figure]] BT 21CN. + +![BT 21CN](images/15007863.png) + +[[CAPTION:Figure]] BT 21CN. + +See also: [MSAN](m.md#msan), [NGN](n.md#ngn), [PSTN](p.md#pstn). + +## BTNUP +**BT National User Part.** BT's variant of ISDN user part signalling. See [IUP](i.md#iup). + +## BTS +**Base Transceiver Station.** The radio equipment at a GSM cell site that talks to mobile phones. In UMTS the equivalent is the Node B, and in LTE the eNodeB. + +See also: [BSC](#bsc), [Node B](n.md#node-b), [E NodeB](e.md#e-nodeb). + +## Bug +A fault in software that makes it behave other than intended. + +See also: [Defect](d.md#defect). + +## Build +The process that turns source code and other assets into a runnable system. The word also means the result: a version of the system ready to test. + +See also: [Automated Build](a.md#automated-build), [Build Environment](#build-environment). + +## Build Environment +A separate environment kept for making builds, usually on one or more dedicated build machines. + +See also: [Build Machine](#build-machine), [Development Environment](d.md#development-environment). + +## Build Machine +A computer used only to make software builds. + +See also: [Build Environment](#build-environment). + +## Build tool +A tool that automates building software, such as Make, Maven or Gradle. + +See also: [Build](#build). + +## Burndown Chart +A chart that shows the work remaining in an iteration or project against time. A burnup chart shows the work completed instead. + +![Burndown Chart](images/15007869.png) + +[[CAPTION:Figure]] Burndown Chart. + +See also: [Velocity](v.md#velocity), [Sprints](s.md#sprints). + +## Burst +A short transmission at a high rate, or a group of errors that arrive close together. + +## Business Agility Ownership +Business agility is an organisation's ability to notice change, inside or outside it, and respond quickly enough to keep delivering value to its customers. + +See also: [Agile](a.md#agile). + +## Business Requirement +A high-level goal of the organisation or customer asking for a system: why it is wanted. Also called a [Why Requirement](w.md#why-requirement). + +See also: [Requirement](r.md#requirement), [Functional Requirement](f.md#functional-requirement). + +## Business Schedule +A high-level schedule of a project's main milestones and the business goals tied to them. It sets the top-down limits that the detailed schedule must fit. + +See also: [Detailed Schedule](d.md#detailed-schedule), [Milestone](m.md#milestone). + +## BYOD +**Bring Your Own Device.** A policy that lets staff use their own laptops, tablets and phones for work, including access to company systems and data. It brings security and support questions that the policy needs to answer. diff --git a/content/glossary/c.md b/content/glossary/c.md new file mode 100644 index 0000000..b3a9659 --- /dev/null +++ b/content/glossary/c.md @@ -0,0 +1,488 @@ +--- +title: C +--- + +# C + +## C Language +A general-purpose, imperative programming language created by Dennis Ritchie at Bell Labs in the early 1970s to write Unix. Its constructs map closely to machine instructions, so it is still the main language for operating systems and embedded software. It is standardised by ISO as ISO/IEC 9899. + +See also: [C++](#c_1), [K & R C](k.md#k-r-c), [Embedded systems programming](e.md#embedded-systems-programming). + +## C++ {#c_1} +A general-purpose language, created by Bjarne Stroustrup as an extension of C, that adds classes, templates and other abstractions while keeping low-level control of memory. It is standardised by ISO, with new versions every three years. + +See also: [C Language](#c-language), [Bjarne Stroustrup](b.md#bjarne-stroustrup). + +## C-Tag +**Customer VLAN tag.** In IEEE 802.1ad double tagging (Q-in-Q), the inner VLAN tag, set by the customer. The service provider adds an outer S-Tag. + +![C-Tag](images/15007942.png) + +[[CAPTION:Figure]] C-Tag. + +![C-Tag](images/15007948.png) + +[[CAPTION:Figure]] C-Tag. + +See also: [S-Tag](s.md#s-tag), [VLAN](v.md#vlan), [Dot1Q](d.md#dot1q). + +## C# {#c_2} +A general-purpose, object-oriented language from Microsoft, created around 2000 for the .NET platform. It also supports functional and generic programming, and is standardised as ECMA-334 and ISO/IEC 23270. + +See also: [.NET](symbols.md#net), [Java](j.md#java). + +## CA +**Certificate Authority.** A trusted organisation that issues digital certificates, signing each to bind a public key to the named subject. Browsers trust HTTPS sites because a CA they trust signed the site's certificate. + +![CA](images/15007880.png) + +[[CAPTION:Figure]] CA. + +See also: [X.509](x.md#x509), [TLS](t.md#tls), [HTTPS](h.md#https). + +## Cacti +An open-source web tool that polls devices, usually over SNMP, and graphs the results with RRDtool, typically to show network traffic and CPU load over time. + +See also: [SNMP](s.md#snmp). + +## Cadence +The regular rhythm of a team's work, such as the length of its sprints or its release cycle. + +See also: [Sprints](s.md#sprints), [Iteration](i.md#iteration), [Timebox](t.md#timebox). + +## Calendar Days +The elapsed time an activity takes on the calendar. How staff days turn into calendar days depends on how many people work on it, and when. + +See also: [Staff Days](s.md#staff-days). + +## CAMEL +**Customised Applications for Mobile networks Enhanced Logic.** A set of 3GPP standards, based on Intelligent Network ideas, that lets operators add services such as prepaid calling to GSM and UMTS, including while the subscriber is roaming. It uses the CAP protocol. + +See also: [CAP](#cap), [IN](i.md#in), [INAP](i.md#inap). + +## Camel Case +Writing compound words with no spaces and a capital letter at the start of each word after the first, as in `camelCase`. Pascal case (`PascalCase`) capitalises the first word too. Many coding standards use one or both for identifiers. + +See also: [Snake Case](s.md#snake-case), [Hungarian notation](h.md#hungarian-notation). + +## CAP +**CAMEL Application Part.** The signalling protocol that CAMEL uses between the switch and the service control point. It runs over TCAP in SS7. + +See also: [CAMEL](#camel), [TCAP](t.md#tcap), [SS7](s.md#ss7). + +## Capability Maturity Model +**CMM.** A model of software process maturity, developed by the Software Engineering Institute for the US military. It defines five levels: initial, repeatable, defined, managed and optimising. Each level names the key process areas an organisation must have in place. Capability Maturity Model Integration (CMMI) replaced it. + +## Cat 5 +**Category 5 cable.** Twisted-pair cable rated to 100 MHz, used for 100 Mbit/s Ethernet and telephony. Category 5e superseded it in 2001. + +See also: [Cat 5e](#cat-5e), [Cat 6](#cat-6), [Twisted pair](t.md#twisted-pair). + +## Cat 5e +**Category 5e cable.** An enhanced Category 5 cable with tighter limits on crosstalk, rated to 100 MHz and good for Gigabit Ethernet up to 100 m. + +See also: [Cat 5](#cat-5), [Cat 6](#cat-6). + +## Cat 6 +**Category 6 cable.** Twisted-pair cable rated to 250 MHz, with stricter crosstalk limits than Cat 5e. It carries 10 Gigabit Ethernet up to about 55 m; Category 6A, rated to 500 MHz, does so up to 100 m. + +See also: [Cat 5](#cat-5), [Cat 5e](#cat-5e), [Ethernet over twisted pair](e.md#ethernet-over-twisted-pair). + +## CBOR +**Concise Binary Object Representation.** A binary data format, defined in RFC 8949, based on the JSON data model but designed for small messages and small decoders. It is widely used in constrained IoT devices, with CoAP, and in security tokens. + +See also: [JSON](j.md#json), [LwM2M](l.md#lwm2m). + +## CCITT +**Comité Consultatif International Téléphonique et Télégraphique.** The former name, until 1993, of the [ITU-T](i.md#itu-t). + +## CCS +**Common Channel Signalling.** Telephone signalling carried on a separate channel shared by many voice circuits, rather than in or alongside each circuit. SS7 is the main example. The alternative is channel-associated signalling (CAS). + +See also: [SS7](s.md#ss7), [ISUP](i.md#isup). + +## CCTA +**Central Computer and Telecommunications Agency.** A former UK government agency that supported departments' computing and telecoms. It created ITIL and PRINCE. + +See also: [ITIL](i.md#itil), [PRINCE2](p.md#prince2). + +## CDMA +**Code-Division Multiple Access.** A channel access method in which many transmitters share the same frequency band at the same time, each spread with its own code. It is the basis of cdmaOne, CDMA2000 and, as wideband CDMA, UMTS. + +See also: [CDMA2000](#cdma2000), [W-CDMA](w.md#w-cdma), [OFDMA](o.md#ofdma). + +## CDMA2000 +The 3G family of mobile standards developed by 3GPP2 as the successor to cdmaOne (IS-95), used mainly in North America and South Korea. It competed with UMTS. + +See also: [CDMA](#cdma), [UMTS](u.md#umts). + +## CDR +**Call Detail Record.** A record made by a switch or other network element for each call or session, with details such as the numbers, start time, duration and outcome. CDRs feed billing, fraud detection and reporting. + +See also: [Mediation](m.md#mediation). + +## CE +**Customer Edge.** The customer's router that connects to the service provider's provider edge (PE) router in an MPLS VPN. It exchanges routes with the PE, statically or with a routing protocol. + +![CE](images/15007888.png) + +[[CAPTION:Figure]] CE. + +See also: [VRF](v.md#vrf), [L3VPN](l.md#l3vpn), [NPE](n.md#npe). + +## CER +**Canonical Encoding Rules.** A restricted form of BER for ASN.1 that allows only one encoding for each value, so that encoded data can be compared or signed. + +See also: [BER](b.md#ber), [DER](d.md#der), [ASN.1](a.md#asn1). + +## Ceremonies +The regular meetings of an agile process, such as sprint planning, the daily stand-up, the review and the retrospective. + +See also: [Daily Scrum](d.md#daily-scrum), [Sprint Planning](s.md#sprint-planning), [Sprint retrospective](s.md#sprint-retrospective). + +## CESG +**Communications-Electronics Security Group.** The part of GCHQ that was the UK government's technical authority for information assurance. Its role passed to the National Cyber Security Centre in 2016. + +See also: [GCHQ](g.md#gchq), [NCSC](n.md#ncsc), [IA](i.md#ia). + +## CFi +**Canonical Format Indicator.** A one-bit field in the IEEE 802.1Q VLAN tag that showed whether MAC addresses were in canonical form. It is always 0 on Ethernet, and later revisions renamed it the drop eligible indicator. + +See also: [Dot1Q](d.md#dot1q), [VLAN](v.md#vlan). + +## CGI +**Common Gateway Interface.** A standard way for a web server to run an external command to answer a request, passing it the request details and returning its output as the page. + +See also: [HTTP](h.md#http). + +## Change Control +The part of change management that decides which artefacts are controlled and makes sure changes to them are proposed, assessed and approved before they are made. The term is also used for change management as a whole. + +See also: [Change Control Board](#change-control-board), [Change Request](#change-request), [Explicit Change Control](e.md#explicit-change-control), [Implicit Change Control](i.md#implicit-change-control). + +## Change Control Board +**CCB.** The group that decides on change requests to artefacts under change control. + +See also: [Change Control](#change-control), [Change Request](#change-request). + +## Change Control Plan +A plan that sets out what change control applies to which project artefacts. + +See also: [Change Control](#change-control). + +## Change Management +Managing requested changes to a project's scope, features or artefacts in a controlled way. + +See also: [Change Control](#change-control), [Configuration Management](#configuration-management). + +## Change Request +**CR.** A request to change something under change control, usually to add, change or remove a requirement, or to change the project plan. + +See also: [Change Control Board](#change-control-board), [Software Change Request](s.md#software-change-request). + +## CHAP +**Challenge-Handshake Authentication Protocol.** A PPP authentication protocol, defined in RFC 1994. The server sends a challenge and the client answers with a hash of the challenge and a shared secret, so the secret never crosses the link. It is safer than PAP. + +See also: [PAP](p.md#pap), [PPP](p.md#ppp), [EAP](e.md#eap). + +## Chrome OS +Google's operating system, based on the Linux kernel, that uses the Chrome browser as its main interface and runs mostly web applications. + +See also: [Linux](l.md#linux), [Android](a.md#android). + +## Circuit Switched +A network in which a dedicated path is set up between the two ends before they communicate, and held for the whole call, as in the traditional telephone network. + +![Circuit Switched](images/15007894.png) + +[[CAPTION:Figure]] Circuit Switched. + +See also: [Packet Switched](p.md#packet-switched), [PSTN](p.md#pstn), [TDM](t.md#tdm). + +## Class 4 Softswitch +A softswitch that routes large volumes of voice calls between carriers, the VoIP equivalent of a class 4 (trunk) exchange. What matters most is protocol support and conversion, transcoding and call capacity. + +See also: [Softswitch](s.md#softswitch), [Class 5 Softswitch](#class-5-softswitch). + +## Class 5 Softswitch +A softswitch that serves end users, the VoIP equivalent of a class 5 (local) exchange. It provides subscriber features such as IP PBX functions, call centres and voicemail. + +See also: [Softswitch](s.md#softswitch), [Class 4 Softswitch](#class-4-softswitch). + +## Class Model +A model of the static structure of an object-oriented system: its classes and how they relate. + +See also: [UML](u.md#uml), [OO](o.md#oo). + +## CLF +**Connectivity Session Location and Repository Function.** In the ETSI TISPAN next-generation network, the function that records each subscriber's access session, linking their IP address to their line and location. It is not to be confused with the IMS subscriber location function (SLF), which finds the right HSS for a user when there are several. + +See also: [NGN](n.md#ngn), [HSS](h.md#hss), [IMS](i.md#ims). + +## CMDB +**Configuration Management Database.** A store of the configuration items in an IT estate and the relationships between them, used in ITIL to understand the effect of changes and incidents. + +See also: [Configuration Item](#configuration-item), [ITIL](i.md#itil). + +## CNG +**Comfort Noise Generation.** Adding artificial background noise at the receiving end of a call during silences, when voice activity detection has stopped the sender transmitting. Without it, the line goes dead quiet and callers think the call has dropped. + +See also: [VAD](v.md#vad), [VoIP](v.md#voip). + +## COBOL +**Common Business-Oriented Language.** An English-like, compiled language designed in 1959 for business data processing. A great deal of banking and government software on mainframes is still written in it. + +See also: [Fortran](f.md#fortran). + +## Code and Fix Lifecycle +Developing a system from a rough idea through repeated rounds of coding and fixing, with little planning or design, until it seems ready to release. + +See also: [Lifecycle](l.md#lifecycle), [Waterfall](w.md#waterfall). + +## Codecs +A codec (coder-decoder) is a device or piece of software that encodes a signal or data stream for transmission or storage, usually compressing it. It also decodes it again for use. + +![Codecs](images/15007900.png) + +[[CAPTION:Figure]] Codecs. + +See also: [Audio Codecs](a.md#audio-codecs), [MPEG](m.md#mpeg). + +## Coding +Writing the source code that defines how software behaves; the core activity of construction. + +See also: [Construction](#construction), [Source Code](s.md#source-code). + +## Coding Standard +See [Construction Standard](#construction-standard). + +## CoE +**Centre of Excellence.** A team or group that provides leadership, good practice, support and training in one area, such as a technology or a skill, across an organisation. + +## COFDM +**Coded Orthogonal Frequency-Division Multiplexing.** OFDM with forward error correction and interleaving added, so the signal survives multipath fading. Digital terrestrial TV (DVB-T) and DAB use it. + +See also: [OFDMA](o.md#ofdma), [DTT](d.md#dtt). + +## Collaboration Model +A model, such as a UML communication diagram, that shows a set of objects or roles and the messages they exchange. + +See also: [Sequence Model](s.md#sequence-model), [UML](u.md#uml). + +## Collaborative Construction +A way of building software in which a small group of developers work closely together on the same functionality. They talk often and informally, and share ownership of the code and its tests. + +See also: [Pair Programming](p.md#pair-programming), [Mob Programming](m.md#mob-programming), [Collective code ownership](#collective-code-ownership). + +## Collective code ownership +The convention that anyone on the team may change any part of the code, to finish a task, fix a defect or improve its structure. + +See also: [XP](x.md#xp), [Pair Programming](p.md#pair-programming). + +## COLO +**Colocation.** Housing equipment in another party's building, typically placing a provider's equipment in a carrier's exchange or a data centre. + +## Compatibility Test +See [Configuration Test](#configuration-test). + +## Compiled Language +A programming language whose usual implementations translate source code ahead of time into machine code, rather than interpret it. The line is blurred: many languages compile to bytecode that a virtual machine then interprets or compiles just in time. + +See also: [Compiler](#compiler), [Interpreted Language](i.md#interpreted-language). + +## Compiler +Software that translates code from one language, usually a high-level language, into another, usually machine code or bytecode. A typical compiler has a front end, a middle end and a back end. A cross-compiler produces code for a different machine from the one it runs on. + +![Compiler](images/15007906.png) + +[[CAPTION:Figure]] Compiler. + +See also: [Compiler Front end](#compiler-front-end), [Compiler Middle end](#compiler-middle-end), [Compiler Back end](#compiler-back-end), [Compiler preprocessor](#compiler-preprocessor), [Machine code](m.md#machine-code). + +## Compiler Back end +The stage of a compiler that turns the optimised intermediate representation into code for one target processor: choosing instructions, allocating registers and scheduling. + +See also: [Compiler](#compiler), [Compiler Middle end](#compiler-middle-end). + +## Compiler Front end +The stage of a compiler that reads the source language: it checks syntax and semantics, reports errors, and produces an intermediate representation for the later stages. + +See also: [Compiler](#compiler), [Compiler Middle end](#compiler-middle-end). + +## Compiler Middle end +The stage of a compiler that optimises the intermediate representation in ways that do not depend on the target processor, such as removing dead code and propagating constants. + +See also: [Compiler](#compiler), [Compiler Front end](#compiler-front-end), [Compiler Back end](#compiler-back-end). + +## Compiler preprocessor +A tool that transforms source code before compilation, for example the C preprocessor, which expands macros, includes files and handles conditional compilation. + +See also: [Compiler](#compiler), [C Language](#c-language). + +## Component +A part of a software system with a defined interface and responsibility. + +See also: [Component Test](#component-test). + +## Component Test +A test of one software component on its own, apart from the rest of the system. + +See also: [Unit Testing](u.md#unit-testing), [Integration Test](i.md#integration-test). + +## Cone of Uncertainty +The observation that estimates made early in a project can be wrong by a large factor either way. The range narrows as the project goes on and decisions are made. + +![Cone of Uncertainty](images/15007912.png) + +[[CAPTION:Figure]] Cone of Uncertainty. + +See also: [Estimation](e.md#estimation), [Progressive Elaboration](p.md#progressive-elaboration). + +## Configuration Item +**CI.** An artefact, or a group of them, that is placed under configuration management and tracked as a unit. + +See also: [Configuration Management](#configuration-management), [CMDB](#cmdb). + +## Configuration Management +**CM.** Identifying, recording, releasing and controlling the versions of a system's artefacts so that its state is known at every point. + +See also: [Configuration Item](#configuration-item), [Version Control](v.md#version-control), [Software Configuration Management](s.md#software-configuration-management). + +## Configuration Test +A test of how a system behaves on different platforms, settings and environments. + +## Construction +Turning a design into working software: detailed design, coding, unit testing, integration and debugging. + +See also: [Coding](#coding), [Construction Standard](#construction-standard). + +## Construction Environment +See [Development Environment](d.md#development-environment). + +## Construction Lead +The person responsible for construction on a project: integration, builds, the development environment, technology problems and deployment. + +## Construction Standard +The conventions and style developers follow when writing a system's source code. Also called a coding standard. + +See also: [Coding Standard](#coding-standard). + +## Construction Test Environment +See [Local Test Environment](l.md#local-test-environment). + +## Construction Testing +The tests a developer runs while building a component, to check changes both in the component and in the system around it. + +See also: [Unit Testing](u.md#unit-testing), [Developer Integration Test](d.md#developer-integration-test). + +## Continuous Deployment +Releasing every change that passes the automated pipeline straight to production, so that code reaches users soon after it is written. + +See also: [Continuous Integration](#continuous-integration), [DevOps](d.md#devops). + +## Continuous Integration +**CI.** Merging every developer's changes into a shared mainline several times a day, with an automated build and tests on each merge. The product is then always in a releasable state. + +See also: [Automated Build](a.md#automated-build), [Continuous Deployment](#continuous-deployment). + +## Control Plane +The part of a network device or network that decides where traffic goes: running routing protocols and building the tables that the data plane uses to forward packets. + +![Control Plane](images/15007918.png) + +[[CAPTION:Figure]] Control Plane. + +See also: [Data Plane](d.md#data-plane), [Management Plane](m.md#management-plane), [User Plane](u.md#user-plane), [SDN](s.md#sdn). + +## Copyleft +Licensing that lets anyone copy, change and share a work on condition that copies and derived works are shared under the same terms. The GPL is the best-known copyleft licence. + +![Copyleft](images/15007924.png) + +[[CAPTION:Figure]] Copyleft. + +See also: [GPL](g.md#gpl), [GNU](g.md#gnu). + +## CoS +**Class of Service.** A marking in a frame or packet that puts it into a traffic class, so the network can give some classes priority over others. + +See also: [QoS](q.md#qos), [PCP](p.md#pcp), [DSCP](d.md#dscp). + +## COTS +**Commercial off-the-shelf.** Products bought ready-made, and configured or adapted, rather than built to order. + +See also: [Mil-COTS](m.md#mil-cots). + +## CPE +**Customer Premises Equipment.** Equipment at the customer's site that connects to the operator's network at the demarcation point, such as a router, modem or phone. + +![CPE](images/15007930.png) + +[[CAPTION:Figure]] CPE. + +See also: [NTU](n.md#ntu). + +## CPM +**Critical Path Method.** A scheduling technique that finds the longest chain of dependent activities in a project. That chain sets the shortest possible duration, and any delay on it delays the end date. + +See also: [PERT chart](p.md#pert-chart), [Gantt chart](g.md#gantt-chart). + +## CPS +**Carrier Pre-Selection.** A UK telephone service that sent a customer's calls through another carrier's network automatically, without dialling a prefix, while the line stayed with the incumbent. + +See also: [WLR](w.md#wlr), [LCR](l.md#lcr). + +## CRC +**Cyclic Redundancy Check.** An error-detecting code calculated as the remainder of a polynomial division of the data. The sender appends it; the receiver recalculates it and treats any mismatch as corruption. Ethernet's frame check sequence is a CRC-32. + +See also: [FCS](f.md#fcs), [Hash Function](h.md#hash-function). + +## CRC Cards +**Class-Responsibility-Collaborator cards.** Index cards used in object-oriented design sessions, one for each class, listing what the class is responsible for and which classes it works with. + +See also: [OO](o.md#oo), [Class Model](#class-model). + +## CSCF +**Call Session Control Function.** The SIP servers at the heart of IMS. The proxy CSCF (P-CSCF) is the device's first contact with the network. The interrogating CSCF (I-CSCF) finds the right serving CSCF for a user. The serving CSCF (S-CSCF) registers users and controls their sessions. + +See also: [IMS](i.md#ims), [SIP](s.md#sip), [HSS](h.md#hss). + +## Csh +The C shell, a Unix shell written by Bill Joy at Berkeley in the late 1970s, with a syntax modelled on C. Tcsh is its improved successor. + +See also: [Tcsh](t.md#tcsh), [Bash](b.md#bash), [Sh](s.md#sh). + +## CSI +**Customer Site Interconnect.** A physical link between an operator's network and a customer's site or another operator's equipment. + +![CSI](images/15007936.png) + +[[CAPTION:Figure]] CSI. + +See also: [Leased Line](l.md#leased-line). + +## CSMA +**Carrier-Sense Multiple Access.** A media access method in which a device listens for traffic on a shared medium and transmits only when it is idle. Classic Ethernet added collision detection (CSMA/CD); Wi-Fi uses collision avoidance (CSMA/CA). + +See also: [MAC](m.md#mac), [Wi-Fi](w.md#wi-fi). + +## CSS +**Cascading Style Sheets.** The language that describes how HTML documents are presented, such as layout, colours and fonts, keeping presentation apart from content. + +See also: [XML](x.md#xml), [W3C](w.md#w3c). + +## CUE +**Configure Unify Execute.** An open-source language for validating, templating and generating configuration and data. It treats types and values as one thing, so a schema, a constraint and concrete data can be combined and checked in the same way. + +See also: [JSON](j.md#json), [YAML](y.md#yaml). + +## Customer Development +A four-step framework from Steve Blank (discovery, validation, creation and building the company) for testing assumptions about a product and its market by experiment. + +See also: [MVP](m.md#mvp). + +## CVSD +**Continuously Variable Slope Delta modulation.** A speech coding method that sends one bit per sample, saying whether the signal went up or down, with a step size that adapts to the signal. It is robust against bit errors and is used in military radio and Bluetooth voice. + +See also: [PCM](p.md#pcm), [Audio Codecs](a.md#audio-codecs). diff --git a/content/glossary/d-f.md b/content/glossary/d-f.md deleted file mode 100644 index 2418d7f..0000000 --- a/content/glossary/d-f.md +++ /dev/null @@ -1,757 +0,0 @@ ---- -title: Glossary, D to F ---- - -# D to F - -## D - -### DAD -**Disciplined Agile Delivery.** A process decision framework by Scott Ambler and Mark Lines. It combines practices from Scrum, XP, Lean and agile modelling, and helps teams choose how to work across the whole delivery lifecycle, including at scale. - -See also: [Agile](a-c.md#agile), [SAFe](s-z.md#safe), [LeSS](l-n.md#less). - -### Daily Meeting -A short meeting held at the same time each day, at which the team coordinates its work. Also called the daily stand-up. - -See also: [Daily Scrum](#daily-scrum), [Three Questions](s-z.md#three-questions). - -### Daily Scrum -Scrum's daily stand-up: a meeting of 15 minutes or less in which the team reviews progress towards the sprint goal and plans the next day. - -See also: [Daily Meeting](#daily-meeting), [Scrum](s-z.md#scrum). - -### Dark Fibre -Optical fibre that is installed but not lit by its owner's equipment. Customers lease it and attach their own equipment, so they control the capacity and technology rather than buying a bandwidth service. - -See also: [Leased Line](l-n.md#leased-line). - -### DASS1 -**Digital Access Signalling System 1.** An early BT protocol for ISDN access in the UK, replaced by DASS2 and long obsolete. - -See also: [DASS2](#dass2), [ISDN](g-k.md#isdn). - -### DASS2 -**Digital Access Signalling System 2.** BT's signalling protocol for ISDN primary rate lines (2 Mbit/s, 30 channels), designed before Q.931 was finished. Standard EuroISDN (Q.931) access has replaced it. - -See also: [DASS1](#dass1), [Q.931](o-r.md#q931), [PRI](o-r.md#pri), [DPNSS](#dpnss). - -### Data modelling -Analysing the data a business uses: identifying the entities, their attributes and the relationships between them. - -See also: [RDBMS](o-r.md#rdbms), [Model](l-n.md#model). - -### Data Plane -The part of a network device that forwards packets, using tables built by the control plane. Also called the forwarding plane or [User Plane](s-z.md#user-plane). - -See also: [Control Plane](a-c.md#control-plane), [Management Plane](l-n.md#management-plane). - -### Database Lead -On projects with large or critical databases, the person responsible for the design and construction of the database parts of the system. - -### Data Warehouse -A central store of current and historical data, integrated from several operational systems and organised for reporting and analysis rather than for transactions. - -See also: [ETL](#etl), [RDBMS](o-r.md#rdbms). - -### DCE -**Data Circuit-terminating Equipment.** The device that connects a terminal or computer (the DTE) to a transmission line, such as a modem. - -See also: [DTE](#dte). - -### DDK -**Driver Development Kit.** The tools, libraries, headers and documentation needed to write device drivers for an operating system. - -See also: [Driver](#driver), [SDK](s-z.md#sdk). - -### DDOS -**Distributed Denial of Service.** An attack that floods a target with traffic from many sources at once, often a botnet, to make it unavailable to legitimate users. Because the traffic comes from everywhere, blocking one source does not stop it. - -See also: [Firewall](#firewall), [IPS](g-k.md#ips). - -### Decomposition Estimation -Estimating each part of a system after breaking it down. See [Bottom-Up Estimation](a-c.md#bottom-up-estimation). - -### Defect -A flaw in a system, process or artefact that makes it fail to meet its specification, a standard or a reasonable expectation. - -See also: [Bug](a-c.md#bug), [Defect Report](#defect-report), [Defect Management](#defect-management). - -### Defect Management -Recording, prioritising, tracking and closing defects so that none are lost. - -See also: [Defect](#defect), [Issue tracker](g-k.md#issue-tracker). - -### Defect Report -A record asking for an artefact to be changed because it does not conform to a requirement or standard, or because it fails. - -See also: [Defect](#defect). - -### Definition of Done -**DoD.** A list, agreed by the team, of what must be true before a piece of work, such as a user story, counts as finished. It usually includes tests passing, review and documentation. - -See also: [Definition of Ready](#definition-of-ready), [User Stories](s-z.md#user-stories). - -### Definition of Ready -**DoR.** Criteria a backlog item must meet before the team takes it into an iteration, such as clear acceptance criteria and no blocking dependencies. Many teams base it on INVEST. - -See also: [Definition of Done](#definition-of-done), [INVEST](g-k.md#invest), [Backlog Grooming](a-c.md#backlog-grooming). - -### Deliverable -An artefact, or set of artefacts, that a project or piece of work hands over as its output. - -See also: [Artefact](a-c.md#artefact), [Work Product](s-z.md#work-product). - -### Delphi -Embarcadero's (originally Borland's) development environment for Object Pascal, an object-oriented descendant of Pascal. The language itself is often called Delphi too. - -See also: [Pascal](o-r.md#pascal). - -### Deployment -Installing a system and making it ready for use, including any integration and configuration for the particular site. - -See also: [Deployment Plan](#deployment-plan), [Release](o-r.md#release). - -### Deployment Lead -On projects with complex deployment needs, the person who plans and oversees deployment, and often operations and maintenance. - -### Deployment Plan -A plan for releasing a system, covering the target platforms, the configuration to release and the steps to follow. - -See also: [Deployment](#deployment), [Release Management](o-r.md#release-management). - -### DER -**Distinguished Encoding Rules.** A restricted form of BER that allows exactly one encoding for each ASN.1 value. It is used where encodings must be unique, above all for data that is signed, such as X.509 certificates. - -See also: [BER](a-c.md#ber), [CER](a-c.md#cer), [ASN.1](a-c.md#asn1), [X.509](s-z.md#x509). - -### DES -**Data Encryption Standard.** A symmetric block cipher from IBM, published as a US federal standard in 1977, with a 64-bit block and a 56-bit key. The short key made it breakable by brute force by the late 1990s, and AES replaced it. - -See also: [Triple DES](s-z.md#triple-des), [AES](a-c.md#aes). - -### Design -Software design: working out the structure of a system and the models and plans for implementing its requirements. - -See also: [High Level Design](g-k.md#high-level-design), [Low Level Design](l-n.md#low-level-design), [Architecture](a-c.md#architecture). - -### Design Lead -The person responsible for a system's architecture and for overseeing design work. - -### Design Pattern -A general, reusable solution to a problem that keeps coming up in software design. A pattern is a description to adapt, not finished code. The Gang of Four book catalogued the best-known object-oriented patterns. - -See also: [GoF](g-k.md#gof), [Pattern](o-r.md#pattern), [Antipattern](a-c.md#antipattern). - -### Design to Tools -Deciding what goes into a system by what existing tools support directly, rather than by what users need. - -### Desk Check -An informal review in which the author asks one or more colleagues to read an artefact and look for defects. - -See also: [Review](o-r.md#review), [Walkthrough](s-z.md#walkthrough). - -### Detailed Design -Design work below the architecture, covering parts of both high-level and low-level design. - -See also: [High Level Design](g-k.md#high-level-design), [Low Level Design](l-n.md#low-level-design). - -### Detailed Schedule -A fine-grained schedule of tasks, effort, people and dependencies, usually built bottom-up from a work plan. It often covers only the next few weeks and rolls forward. - -See also: [Business Schedule](a-c.md#business-schedule), [Rolling Wave Planning](o-r.md#rolling-wave-planning), [Project Headlights](o-r.md#project-headlights). - -### Developer -A software engineer, especially when doing design and construction. - -### Developer Integration Test -**DIT.** An integration test that a developer runs in the system test environment after a project build, before releasing new functionality. - -See also: [Integration Test](g-k.md#integration-test). - -### Development Environment -The hardware, software and tools in which developers build and test software. - -See also: [Local Development Environment](l-n.md#local-development-environment), [IDE](g-k.md#ide). - -### Development process -The set of activities a team follows to develop software. - -See also: [Lifecycle](l-n.md#lifecycle), [SDLC](s-z.md#sdlc). - -### DevOps -A way of working that brings development and operations together, sharing responsibility for delivering and running software. It relies on automation of builds, tests, deployment and monitoring. - -See also: [Continuous Integration](a-c.md#continuous-integration), [Continuous Deployment](a-c.md#continuous-deployment). - -### DHCP -**Dynamic Host Configuration Protocol.** The protocol a device uses to get an IP address and other network settings, such as the gateway and DNS servers, from a server when it joins a network. - -See also: [BOOTP](a-c.md#bootp), [DNS](#dns). - -### Diagram -A graphical representation of a system, process or other information. - -### DLE -**Digital Local Exchange.** A digital telephone exchange that connects subscribers' lines, either directly or through remote concentrators. In the UK these were System X and AXE 10 exchanges. - -See also: [SystemX](s-z.md#systemx), [MDF](l-n.md#mdf), [Local Loop](l-n.md#local-loop). - -### DLP -**Data Link Protocol.** Any protocol at the data link layer (layer 2) of the OSI model, such as Ethernet, Token Ring, FDDI or PPP. DLP also commonly stands for data loss prevention. - -See also: [L2](l-n.md#l2). - -### DMZ -**Demilitarised Zone.** A network segment that sits between the internet and an organisation's internal network, holding the services that outsiders need to reach, such as web and mail servers. Firewalls on both sides limit what each zone can reach. - -See also: [Firewall](#firewall). - -### DN -**Directory Number.** A telephone number: the digits that identify a subscriber's line or device and route calls to it. - -See also: [E.164](#e164). - -### DNB -A Dun & Bradstreet business file: a company's credit and identity record from Dun & Bradstreet. - -### DNS -**Domain Name System.** The internet's distributed, hierarchical naming system. Its main job is to translate names such as `example.com` into IP addresses, but it holds many other kinds of record. - -See also: [Domain Name](#domain-name), [FQDN](#fqdn), [DNS - A](#dns-a), [DNS Spoofing](#dns-spoofing). - -### DNS - A {#dns-a} -**Address record.** A DNS record that maps a name to an IPv4 address. - -See also: [DNS - AAAA](#dns-aaaa), [DNS](#dns). - -### DNS - AAAA {#dns-aaaa} -A DNS record that maps a name to an IPv6 address. - -See also: [DNS - A](#dns-a), [IPv6](g-k.md#ipv6). - -### DNS - CNAME {#dns-cname} -**Canonical name record.** A DNS record that makes one name an alias for another; the lookup carries on with the target name. - -See also: [DNS - DNAME](#dns-dname). - -### DNS - DNAME {#dns-dname} -A DNS record that makes a name, and every name under it, an alias for another domain. A CNAME aliases only the exact name. - -See also: [DNS - CNAME](#dns-cname). - -### DNS - SRV {#dns-srv} -**Service record.** A DNS record that gives the host and port for a named service, such as SIP or XMPP, together with priority and weight. - -See also: [SIP](s-z.md#sip), [XMPP](s-z.md#xmpp). - -### DNS - TXT {#dns-txt} -A DNS record that holds arbitrary text. It now mostly carries machine-readable data, such as SPF, DKIM and DMARC policies for email and domain ownership proofs. - -### DNS Spoofing -An attack that puts false records into a DNS resolver's cache, so that it sends users to the wrong address, usually one the attacker controls. Also called DNS cache poisoning. - -See also: [DNS](#dns), [ARP spoofing](a-c.md#arp-spoofing). - -### DoD -See [Definition of Done](#definition-of-done). - -### DoR -See [Definition of Ready](#definition-of-ready). - -### Docker -A platform for building and running software in containers: packages that hold an application and its dependencies and run in isolation on a shared operating system kernel. - -See also: [VM](s-z.md#vm). - -### Document Collaboration -Tools that let several people work on the same document at once, from different places, to reach one final version. Wikis, Confluence and SharePoint are examples. - -See also: [Wiki](s-z.md#wiki). - -### Domain Name -A name registered in the DNS, such as `example.co.uk`, that identifies an area of administrative control on the internet. Names form a hierarchy under the unnamed root: top-level domains, then second-level domains, then subdomains. - -See also: [DNS](#dns), [FQDN](#fqdn). - -### Dot1Q -**IEEE 802.1Q.** The standard for VLANs on Ethernet. It adds a 4-byte tag to each frame with a 12-bit VLAN identifier and a 3-bit priority (802.1p). - -See also: [VLAN](s-z.md#vlan), [Dot1Q trunking](#dot1q-trunking), [PCP](o-r.md#pcp). - -### Dot1Q trunking -Carrying several VLANs over one link between switches. Each frame is tagged with its VLAN ID as it enters the trunk. The switch at the other end uses the tag to deliver it to ports in the same VLAN. - -```mermaid -flowchart LR - A10[Port, VLAN 10] --> S1[Switch 1] - A20[Port, VLAN 20] --> S1 - S1 == "trunk: frames tagged 10 or 20" ==> S2[Switch 2] - S2 --> B10[Port, VLAN 10] - S2 --> B20[Port, VLAN 20] -``` - -[[CAPTION:Figure]] Two VLANs sharing an 802.1Q trunk. - -See also: [Dot1Q](#dot1q), [VLAN](s-z.md#vlan). - -### Downstream -In a project, the later activities and artefacts, such as coding, testing and deployment. In networking, the direction towards the customer. - -See also: [Upstream](s-z.md#upstream). - -### DP -**Distribution Point.** In a UK access network, the point near the customer, often on a telegraph pole, where the drop wire to the premises joins the cable from the exchange. In fibre networks, the point where a fibre splits towards several premises. - -See also: [Local Loop](l-n.md#local-loop), [PCP](o-r.md#pcp). - -### DPI -**Deep Packet Inspection.** Examining the contents of packets, beyond the IP and transport headers, to classify, block, shape or log traffic. Firewalls, intrusion prevention and traffic management use it. - -See also: [Firewall](#firewall), [IPS](g-k.md#ips), [Packet sniffing](o-r.md#packet-sniffing). - -### DPNSS -**Digital Private Network Signalling System.** A BT-defined protocol for linking PBXs over digital private circuits, with a common set of features across vendors. QSIG is the international equivalent. - -See also: [QSIG](o-r.md#qsig), [DASS2](#dass2). - -### DQ -**Data Quality.** How accurate, complete, consistent and timely data is for its purpose. - -### Draft -An artefact under version control but not yet under change control. - -See also: [Baseline](a-c.md#baseline). - -### Driver -Software that lets an operating system and applications use a particular device, hiding the details of the hardware behind a standard interface. - -See also: [DDK](#ddk), [Kernel](g-k.md#kernel). - -### DSCP -**Differentiated Services Code Point.** A 6-bit field in the IP header that marks a packet's traffic class, so routers can give it the right treatment under DiffServ quality of service. - -See also: [QoS](o-r.md#qos), [CoS](a-c.md#cos). - -### DSDM -**Dynamic Systems Development Method.** An agile framework from the 1990s, built on rapid application development, that fixes time and cost and varies scope. It is now called the DSDM agile project framework. - -See also: [Agile development](a-c.md#agile-development), [RAD](o-r.md#rad). - -### DSL -**Digital Subscriber Line.** A family of technologies that carry digital data over copper telephone lines, using frequencies above voice. ADSL and VDSL are the most common. - -See also: [ADSL](a-c.md#adsl), [SDSL](s-z.md#sdsl), [VDSL](s-z.md#vdsl), [DSLAM](#dslam). - -### DSLAM -**Digital Subscriber Line Access Multiplexer.** The equipment in an exchange or street cabinet that terminates many customers' DSL lines and combines their traffic onto one high-speed link. - -See also: [DSL](#dsl), [MSAN](l-n.md#msan), [BRAS](a-c.md#bras). - -### DTE -**Data Terminal Equipment.** The device at the end of a data link that is the source or destination of data, such as a computer or terminal. It connects to the line through a DCE. - -```mermaid -flowchart LR - A[DTE: computer] --- B[DCE: modem] --- C((Line)) --- D[DCE: modem] --- E[DTE: computer] -``` - -[[CAPTION:Figure]] DTEs connected through DCEs. - -See also: [DCE](#dce). - -### DTLS -**Datagram Transport Layer Security.** A version of TLS for datagram transports such as UDP, giving the same confidentiality, integrity and authentication. It is used by CoAP and LwM2M in IoT and by WebRTC. The current version is DTLS 1.3, in RFC 9147. - -See also: [TLS](s-z.md#tls), [UDP](s-z.md#udp), [LwM2M](l-n.md#lwm2m). - -### DTMF -**Dual-Tone Multi-Frequency.** Telephone signalling in which each key sends two tones at once, one from a low group and one from a high group. It is standardised as ITU-T Q.23, and BT calls it MF4. - -See also: [IVR](g-k.md#ivr), [PSTN](o-r.md#pstn). - -### DTT -**Digital Terrestrial Television.** Television broadcast digitally from ground-based transmitters, such as Freeview in the UK. It fits several channels into the spectrum one analogue channel used. - -See also: [DTV](#dtv), [COFDM](a-c.md#cofdm). - -### DTV -**Digital Television.** Television sent as digital signals rather than analogue, by terrestrial, satellite, cable or IP networks. - -See also: [DTT](#dtt), [IPTV](g-k.md#iptv). - -### Duplex -A link that can carry communication in both directions. In full duplex both sides can send at once; in half duplex they take turns. A simplex link carries one direction only. - -```mermaid -flowchart LR - subgraph Simplex - S1[A] --> S2[B] - end - subgraph "Half duplex: one way at a time" - H1[A] <-.-> H2[B] - end - subgraph "Full duplex: both ways at once" - F1[A] --> F2[B] - F2 --> F1 - end -``` - -[[CAPTION:Figure]] Simplex, half duplex and full duplex. - -See also: [Full duplex](#full-duplex), [Half Duplex](g-k.md#half-duplex), [Simplex](s-z.md#simplex). - -### DVSR -**Dynamically Verified Static Routing.** A Redback (later Ericsson) routing feature: a static route that the router adds to its table only while regular pings show the next hop is reachable. - -See also: [Router](o-r.md#router). - -## E - -### E NodeB -**Evolved Node B (eNodeB, eNB).** The LTE base station. Unlike a UMTS Node B, it has no separate controller: it handles radio resource management and handover itself, and connects directly to the core. - -See also: [E-UTRAN](#e-utran), [Node B](l-n.md#node-b), [MME](l-n.md#mme), [LTE (4G)](l-n.md#lte-4g). - -### E-UTRAN -**Evolved UMTS Terrestrial Radio Access Network.** The LTE radio access network: eNodeBs and the E-UTRA air interface, which uses OFDMA downstream and SC-FDMA upstream. - -See also: [E NodeB](#e-nodeb), [UTRAN](s-z.md#utran), [LTE (4G)](l-n.md#lte-4g). - -### E.164 {#e164} -The ITU-T recommendation for the international public telephone numbering plan. An E.164 number has at most 15 digits, starting with the country code, and is written with a leading `+`. - -See also: [DN](#dn). - -### EAD -**Ethernet Access Direct.** Openreach's point-to-point fibre Ethernet circuit, used to connect business sites and for backhaul, at speeds up to 10 Gbit/s. - -See also: [EoFTTC](#eofttc), [Leased Line](l-n.md#leased-line), [Backhaul](a-c.md#backhaul). - -### EAP -**Extensible Authentication Protocol.** An authentication framework, defined in RFC 3748, that carries many authentication methods, such as EAP-TLS and PEAP. IEEE 802.1X uses it for network access control on Wi-Fi and wired networks. - -See also: [PEAP](o-r.md#peap), [WPA2](s-z.md#wpa2), [Radius](o-r.md#radius). - -### Earned Value -**EV.** A measure of progress that compares the value of the work completed with the work planned and its actual cost, to show schedule and cost variance. - -See also: [Earned Value Management](#earned-value-management). - -### Earned Value Management -**EVM.** Managing a project using earned value measures. - -See also: [Earned Value](#earned-value). - -### Echo -A delayed reflection of a signal. On telephone calls it comes from hybrids that convert between 2-wire and 4-wire circuits, or from a loudspeaker feeding back into a microphone. Delay makes it more noticeable. - -See also: [Echo cancellation](#echo-cancellation), [Sidetone](s-z.md#sidetone). - -### Echo cancellation -Removing echo from a call by modelling the echo path and subtracting an estimate of the echo from the incoming signal. It replaced echo suppression, which muted the return path while the far end was talking. Line echo cancellers handle network echo; acoustic echo cancellers handle loudspeaker-to-microphone echo. - -See also: [Echo](#echo), [Echo Cancellation - G.168](#echo-cancellation-g168). - -### Echo Cancellation - G.168 {#echo-cancellation-g168} -ITU-T Recommendation G.168, which specifies digital network echo cancellers. It covers their performance, how tones disable them for fax and data, and the laboratory tests to check them. - -See also: [Echo cancellation](#echo-cancellation). - -### EDB -**Emergency Database.** In UK telephony, the database that tells emergency services the address behind the number calling 999. - -### EDGE -**Enhanced Data rates for GSM Evolution.** An upgrade to GSM data, also called EGPRS, that uses better modulation and coding to roughly triple GPRS speeds on the same radio channels. - -See also: [GPRS](g-k.md#gprs), [GSM](g-k.md#gsm), [GERAN](g-k.md#geran). - -### Editor -In an inspection, the person who makes the fixes to the artefact afterwards, usually its author. - -See also: [Inspection](g-k.md#inspection). - -### EFM -**Ethernet in the First Mile.** IEEE 802.3ah, which defines Ethernet over the access link between provider and customer, on copper or fibre, with its own management functions. - -See also: [Ethernet over twisted pair](#ethernet-over-twisted-pair), [EoFTTC](#eofttc). - -### Egress -Traffic leaving a network or a device, as opposed to ingress traffic coming in. - -See also: [Ingress](g-k.md#ingress). - -### EIA -**Electronic Industries Alliance.** A former US standards and trade body for electronics manufacturers, known for standards such as RS-232. It closed in 2011 and its work passed to other bodies, including the TIA. - -See also: [TIA](s-z.md#tia). - -### EIGRP -**Enhanced Interior Gateway Routing Protocol.** Cisco's advanced distance-vector routing protocol, which sends only incremental updates. It replaced IGRP in 1993 and was published as RFC 7868 in 2016. - -See also: [IGRP](g-k.md#igrp), [OSPF](o-r.md#ospf), [IGP](g-k.md#igp). - -### Embedded systems programming -Writing software for an embedded system, a computer built into a device, using the interfaces and limited resources the device provides. - -See also: [RTOS](o-r.md#rtos), [MCU](l-n.md#mcu), [C Language](a-c.md#c-language). - -### Engineering Discussion -A meeting to frame a problem and look for solutions. If materials go out beforehand for people to read, it is closer to a preview. - -See also: [Preview](o-r.md#preview). - -### Engineering Management -**EM.** Planning, staffing, tracking and controlling software projects, together with managing the teams and the organisation. - -### Engineering Process -Software engineering process: defining how software engineering work is done, and improving it. - -See also: [Process](o-r.md#process). - -### EoFTTC -**Ethernet over FTTC.** A business Ethernet service over fibre to the cabinet: copper runs from the premises to the street cabinet, and fibre from there to the exchange. It gives symmetrical speeds of tens of megabits per second at lower cost than a fibre leased line. - -See also: [EAD](#ead), [VDSL](s-z.md#vdsl), [EFM](#efm). - -### EoS -**Ethernet over SDH or SONET.** Protocols that carry Ethernet over SDH or SONET networks. They usually use the generic framing procedure (GFP) and virtual concatenation to fit the Ethernet rate into SDH paths. - -See also: [SDH](s-z.md#sdh), [SONET](s-z.md#sonet). - -### ePDG -**Evolved Packet Data Gateway.** The LTE core node that lets devices on untrusted non-3GPP access, such as public Wi-Fi, reach the mobile core through an IPsec tunnel. Wi-Fi calling relies on it. - -See also: [PGW](o-r.md#pgw), [IPSec](g-k.md#ipsec), [ANDSF](a-c.md#andsf). - -### Epic -A large user story, too big for one iteration, that is split into smaller stories. - -See also: [User Stories](s-z.md#user-stories), [Story Splitting](s-z.md#story-splitting). - -### EPWS -**Ethernet Private Wire Service.** A point-to-point Ethernet service between two customer sites, carried over the provider's network. - -See also: [EVPL](#evpl), [VPWS](s-z.md#vpws). - -### ESB -**Enterprise Service Bus.** Middleware in a service-oriented architecture that connects applications through a shared bus. It routes, transforms and mediates messages between them, so that each application integrates with the bus rather than with every other application. - -See also: [SOA](s-z.md#soa), [Microservice](l-n.md#microservice). - -### Estimate -The result of estimating: the predicted size, effort, cost or duration, with the inputs, assumptions and method behind it. It can be a full project estimate or a one-line task estimate. - -See also: [Estimation](#estimation), [Project Estimate](o-r.md#project-estimate), [Task Estimate](s-z.md#task-estimate). - -### Estimation -Working out the likely size, effort, cost, schedule or quality of a project. Several techniques used together give a better estimate than one, and estimates should be redone as the project goes on. - -See also: [Estimate](#estimate), [Cone of Uncertainty](a-c.md#cone-of-uncertainty), [Planning Poker](o-r.md#planning-poker). - -### Ethernet over twisted pair -Ethernet physical layers that use twisted-pair copper cable. 10BASE-T, 100BASE-TX, 1000BASE-T and 10GBASE-T are the main ones. Single-pair variants, such as 10BASE-T1L, serve industrial and automotive uses. - -See also: [Cat 5](a-c.md#cat-5), [Cat 6](a-c.md#cat-6), [Twisted pair](s-z.md#twisted-pair). - -### Ethical Hacking -Testing an organisation's security with its permission, using attackers' methods to find weaknesses before they do. Ethical hackers are also called white hats. - -See also: [Pen testing](o-r.md#pen-testing). - -### ETL -**Extract, Transform, Load.** The three steps for moving data into a data warehouse: read data from source systems, convert and clean it into the target form, and write it into the target. - -See also: [Data Warehouse](#data-warehouse). - -### ETSI -**European Telecommunications Standards Institute.** The not-for-profit body, based in Sophia Antipolis in France, that sets European telecoms standards. Its work includes GSM, DECT and TETRA, and it is a founding partner of 3GPP. - -See also: [ITU](g-k.md#itu), [GSM](g-k.md#gsm). - -### EVC -**Ethernet Virtual Connection.** In Metro Ethernet Forum (MEF) terms, the association between two or more customer interfaces (UNIs) across a provider's network. It is a virtual pipe that defines which sites can exchange frames. - -See also: [EVPL](#evpl), [UNI](s-z.md#uni). - -### Evolutionary Delivery Lifecycle -A lifecycle that combines evolutionary prototyping with staged delivery. - -See also: [Evolutionary Prototyping Lifecycle](#evolutionary-prototyping-lifecycle), [Staged Delivery Lifecycle](s-z.md#staged-delivery-lifecycle). - -### Evolutionary Prototyping Lifecycle -A lifecycle in which a system grows from an initial concept through repeated prototypes until it is ready to deliver. - -See also: [Lifecycle](l-n.md#lifecycle), [Iterative Development](g-k.md#iterative-development). - -### EVPL -**Ethernet Virtual Private Line.** An MEF E-Line service that carries several point-to-point virtual connections over one customer interface, separated by VLAN. Its sibling, Ethernet Private Line (EPL), gives one transparent connection per port. - -See also: [EVC](#evc), [EPWS](#epws). - -### Expert Judgement -Estimating from the experience of people who know the work. It works best when combined with other techniques, such as analogy and decomposition. - -See also: [Estimation](#estimation), [Analogy Estimation](a-c.md#analogy-estimation). - -### Explicit Change Control -Change control of artefacts, such as requirements and plans, that a change control board manages directly. - -See also: [Implicit Change Control](g-k.md#implicit-change-control), [Change Control Board](a-c.md#change-control-board). - -### Explicit Risk Management -See [Extrinsic Risk Management](#extrinsic-risk-management). - -### Exploratory Testing -Testing in which the tester learns about the software, designs tests and runs them all at once, rather than following a script. - -See also: [Informal Testing](g-k.md#informal-testing). - -### Extrinsic Risk Management -Formal risk management added to a project, such as keeping a top-ten risk list and reviewing it regularly. - -See also: [Intrinsic Risk Management](g-k.md#intrinsic-risk-management), [Risk Management](o-r.md#risk-management). - -## F - -### Facilitation -Running a meeting or workshop so the group reaches its goal, without taking part in the content. The person who does it is the facilitator. - -### FCS -**Frame Check Sequence.** An error-detecting code, usually a CRC, added to the end of a frame. A receiver that finds a mismatch discards the frame; protocols higher up, such as TCP, recover the loss. - -See also: [CRC](a-c.md#crc). - -### FDDI -**Fibre Distributed Data Interface.** A 100 Mbit/s token-passing LAN standard over a dual fibre ring. Fast Ethernet made it obsolete. - -See also: [Token Ring](s-z.md#token-ring), [100 Mbps (Fast Ethernet)](s-z.md#100-mbps-fast-ethernet). - -### Feature -A piece of a system's functionality, seen from the user's side. The term also refers to a project's scope, measured in features. - -See also: [Feature Test](#feature-test), [Feature creep](#feature-creep). - -### Feature creep -The tendency for requirements to keep growing during a project, adding features that were never planned, at a cost to schedule and quality. - -See also: [Change Control](a-c.md#change-control). - -### Feature Test -A test of one area of system functionality once its construction is finished. - -### FemtoCell -A small, low-power mobile base station for a home or small office, connected to the operator over the customer's broadband. It improves indoor coverage for a handful of phones. - -See also: [Mobile Cell](l-n.md#mobile-cell), [E NodeB](#e-nodeb). - -### FIPS -**Federal Information Processing Standards.** US federal government standards for computer systems, issued by NIST, covering areas such as cryptography where no suitable industry standard exists. - -See also: [FIPS 140-2](#fips-140-2), [NIST](l-n.md#nist). - -### FIPS 140-2 -The US standard, from 2001, of security requirements for cryptographic modules. It defines four security levels and covers module design, ports, roles, physical security and key management. FIPS 140-3 replaced it in 2019, and 140-2 certificates are being retired. - -See also: [FIPS](#fips), [NIST](l-n.md#nist). - -### Firewall -A network security device or software that allows or blocks traffic by rules, usually between a trusted internal network and an untrusted one. Network firewalls protect whole networks; host-based firewalls protect one machine. - -See also: [DMZ](#dmz), [IPS](g-k.md#ips), [DPI](#dpi). - -### FNA -**Full Network Authentication.** A broadband login method in which the network authenticates the line itself, by its network identifier, through RADIUS, rather than a username and password typed by the customer. - -See also: [Radius](o-r.md#radius), [PPPoE](o-r.md#pppoe). - -### Formal Testing -Testing that follows documented test cases. - -See also: [Informal Testing](g-k.md#informal-testing), [Test Case](s-z.md#test-case). - -### Fortran -The first widely used high-level language, developed at IBM in the 1950s for scientific computing. Modern Fortran is still used for numerical work and supercomputing. - -See also: [COBOL](a-c.md#cobol). - -### Forwarding plane -See [Data Plane](#data-plane). - -### FOTA -**Firmware Over-The-Air.** Updating device firmware remotely over a wireless link. A safe design verifies a signature on each image and keeps two partitions, so a failed update rolls back. It also rolls updates out to the fleet in stages. - -See also: [Root of Trust](o-r.md#root-of-trust), [LwM2M](l-n.md#lwm2m). - -### FQDN -**Fully Qualified Domain Name.** A domain name that names every level of the DNS hierarchy down from the root, such as `www.example.com.`, so it can mean only one thing. - -See also: [Domain Name](#domain-name), [DNS](#dns). - -### Frame Relay -A packet-switched WAN technology, widely sold by carriers in the 1990s, that connects sites through virtual circuits over a shared network. It was cheaper than leased lines; MPLS VPNs have since replaced it. - -See also: [X.25](s-z.md#x25), [PVC](o-r.md#pvc), [L3VPN](l-n.md#l3vpn). - -### Frequent Releases -Putting the product into users' hands often, so the team gets feedback early and acts on it. - -See also: [Continuous Deployment](a-c.md#continuous-deployment), [Release](o-r.md#release). - -### Frequently Asked Question -**FAQ.** A list of common questions and their answers. - -### FSK -**Frequency-Shift Keying.** Modulation that sends digital data by switching the carrier between frequencies; in binary FSK, one frequency for 1 and another for 0. Caller ID and early modems use it. - -See also: [QAM](o-r.md#qam). - -### FTP -**File Transfer Protocol.** A protocol for transferring files between a client and a server, using separate control and data connections. It sends credentials in clear text, so FTPS or SFTP should be used instead. - -See also: [FTPS](#ftps), [SFTP](s-z.md#sftp). - -### FTPS -**FTP Secure.** FTP with TLS encryption added. It is a different protocol from SFTP, which runs over SSH. - -See also: [FTP](#ftp), [SFTP](s-z.md#sftp), [TLS](s-z.md#tls). - -### Full duplex -Communication in which both ends can send and receive at the same time, like a telephone call. - -See also: [Duplex](#duplex), [Half Duplex](g-k.md#half-duplex). - -### Function Point -A measure of software size based on the functionality it gives users, counted from its inputs, outputs, enquiries, files and interfaces. - -See also: [Lines of Code](l-n.md#lines-of-code), [Estimation](#estimation). - -### Functional Requirement -Something a system must do to meet the business requirements. Also called a [What Requirement](s-z.md#what-requirement). - -See also: [Non-functional Requirement](l-n.md#non-functional-requirement), [Business Requirement](a-c.md#business-requirement). - -### Functional Testing -Testing a system's behaviour against what is expected, without regard to how it works inside. Also called black-box testing. - -See also: [Black-Box Testing](a-c.md#black-box-testing). - -### Fuzzy Front End -The early, ill-defined stage of a project, from the first idea to the approval of the project charter. - -See also: [Project Charter](o-r.md#project-charter). - -### Fuzzy Logic -In estimation, sorting items into rough size bands, such as small, medium and large, each standing for an overlapping range of values. In computing generally, fuzzy logic means reasoning with degrees of truth between 0 and 1. - -See also: [T-Shirt Sizing](s-z.md#t-shirt-sizing), [Analogy Estimation](a-c.md#analogy-estimation). - -### FXO -**Foreign Exchange Office.** The port on a device, such as a phone, fax or VoIP gateway, that plugs into an analogue telephone line and receives dial tone and ringing from it. - -See also: [FXS](#fxs). - -### FXS -**Foreign Exchange Station.** The port that provides an analogue telephone line: it supplies power, dial tone and ringing to the phone connected to it. An exchange line or an analogue adapter has FXS ports. - -See also: [FXO](#fxo). diff --git a/content/glossary/d.md b/content/glossary/d.md new file mode 100644 index 0000000..16012c8 --- /dev/null +++ b/content/glossary/d.md @@ -0,0 +1,493 @@ +--- +title: D +--- + +# D + +## DAD +**Disciplined Agile Delivery.** A process decision framework by Scott Ambler and Mark Lines. It combines practices from Scrum, XP, Lean and agile modelling, and helps teams choose how to work across the whole delivery lifecycle, including at scale. + +See also: [Agile](a.md#agile), [SAFe](s.md#safe), [LeSS](l.md#less). + +## Daily Meeting +A short meeting held at the same time each day, at which the team coordinates its work. Also called the daily stand-up. + +See also: [Daily Scrum](#daily-scrum), [Three Questions](t.md#three-questions). + +## Daily Scrum +Scrum's daily stand-up: a meeting of 15 minutes or less in which the team reviews progress towards the sprint goal and plans the next day. + +See also: [Daily Meeting](#daily-meeting), [Scrum](s.md#scrum). + +## Dark Fibre +Optical fibre that is installed but not lit by its owner's equipment. Customers lease it and attach their own equipment, so they control the capacity and technology rather than buying a bandwidth service. + +See also: [Leased Line](l.md#leased-line). + +## DASS1 +**Digital Access Signalling System 1.** An early BT protocol for ISDN access in the UK, replaced by DASS2 and long obsolete. + +See also: [DASS2](#dass2), [ISDN](i.md#isdn). + +## DASS2 +**Digital Access Signalling System 2.** BT's signalling protocol for ISDN primary rate lines (2 Mbit/s, 30 channels), designed before Q.931 was finished. Standard EuroISDN (Q.931) access has replaced it. + +See also: [DASS1](#dass1), [Q.931](q.md#q931), [PRI](p.md#pri), [DPNSS](#dpnss). + +## Data modelling +Analysing the data a business uses: identifying the entities, their attributes and the relationships between them. + +See also: [RDBMS](r.md#rdbms), [Model](m.md#model). + +## Data Plane +The part of a network device that forwards packets, using tables built by the control plane. Also called the forwarding plane or [User Plane](u.md#user-plane). + +See also: [Control Plane](c.md#control-plane), [Management Plane](m.md#management-plane). + +## Database Lead +On projects with large or critical databases, the person responsible for the design and construction of the database parts of the system. + +## Data Warehouse +A central store of current and historical data, integrated from several operational systems and organised for reporting and analysis rather than for transactions. + +![Data Warehouse](images/15007957.png) + +[[CAPTION:Figure]] Data Warehouse. + +![Data Warehouse](images/15007965.png) + +[[CAPTION:Figure]] Data Warehouse. + +See also: [ETL](e.md#etl), [RDBMS](r.md#rdbms). + +## DCE +**Data Circuit-terminating Equipment.** The device that connects a terminal or computer (the DTE) to a transmission line, such as a modem. + +![DCE](images/15007971.png) + +[[CAPTION:Figure]] DCE. + +See also: [DTE](#dte). + +## DDK +**Driver Development Kit.** The tools, libraries, headers and documentation needed to write device drivers for an operating system. + +See also: [Driver](#driver), [SDK](s.md#sdk). + +## DDOS +**Distributed Denial of Service.** An attack that floods a target with traffic from many sources at once, often a botnet, to make it unavailable to legitimate users. Because the traffic comes from everywhere, blocking one source does not stop it. + +![DDOS](images/15007977.png) + +[[CAPTION:Figure]] DDOS. + +See also: [Firewall](f.md#firewall), [IPS](i.md#ips). + +## Decomposition Estimation +Estimating each part of a system after breaking it down. See [Bottom-Up Estimation](b.md#bottom-up-estimation). + +## Defect +A flaw in a system, process or artefact that makes it fail to meet its specification, a standard or a reasonable expectation. + +See also: [Bug](b.md#bug), [Defect Report](#defect-report), [Defect Management](#defect-management). + +## Defect Management +Recording, prioritising, tracking and closing defects so that none are lost. + +See also: [Defect](#defect), [Issue tracker](i.md#issue-tracker). + +## Defect Report +A record asking for an artefact to be changed because it does not conform to a requirement or standard, or because it fails. + +See also: [Defect](#defect). + +## Definition of Done +**DoD.** A list, agreed by the team, of what must be true before a piece of work, such as a user story, counts as finished. It usually includes tests passing, review and documentation. + +See also: [Definition of Ready](#definition-of-ready), [User Stories](u.md#user-stories). + +## Definition of Ready +**DoR.** Criteria a backlog item must meet before the team takes it into an iteration, such as clear acceptance criteria and no blocking dependencies. Many teams base it on INVEST. + +See also: [Definition of Done](#definition-of-done), [INVEST](i.md#invest), [Backlog Grooming](b.md#backlog-grooming). + +## Deliverable +An artefact, or set of artefacts, that a project or piece of work hands over as its output. + +See also: [Artefact](a.md#artefact), [Work Product](w.md#work-product). + +## Delphi +Embarcadero's (originally Borland's) development environment for Object Pascal, an object-oriented descendant of Pascal. The language itself is often called Delphi too. + +See also: [Pascal](p.md#pascal). + +## Deployment +Installing a system and making it ready for use, including any integration and configuration for the particular site. + +See also: [Deployment Plan](#deployment-plan), [Release](r.md#release). + +## Deployment Lead +On projects with complex deployment needs, the person who plans and oversees deployment, and often operations and maintenance. + +## Deployment Plan +A plan for releasing a system, covering the target platforms, the configuration to release and the steps to follow. + +See also: [Deployment](#deployment), [Release Management](r.md#release-management). + +## DER +**Distinguished Encoding Rules.** A restricted form of BER that allows exactly one encoding for each ASN.1 value. It is used where encodings must be unique, above all for data that is signed, such as X.509 certificates. + +See also: [BER](b.md#ber), [CER](c.md#cer), [ASN.1](a.md#asn1), [X.509](x.md#x509). + +## DES +**Data Encryption Standard.** A symmetric block cipher from IBM, published as a US federal standard in 1977, with a 64-bit block and a 56-bit key. The short key made it breakable by brute force by the late 1990s, and AES replaced it. + +![DES](images/15007983.png) + +[[CAPTION:Figure]] DES. + +See also: [Triple DES](t.md#triple-des), [AES](a.md#aes). + +## Design +Software design: working out the structure of a system and the models and plans for implementing its requirements. + +See also: [High Level Design](h.md#high-level-design), [Low Level Design](l.md#low-level-design), [Architecture](a.md#architecture). + +## Design Lead +The person responsible for a system's architecture and for overseeing design work. + +## Design Pattern +A general, reusable solution to a problem that keeps coming up in software design. A pattern is a description to adapt, not finished code. The Gang of Four book catalogued the best-known object-oriented patterns. + +See also: [GoF](g.md#gof), [Pattern](p.md#pattern), [Antipattern](a.md#antipattern). + +## Design to Tools +Deciding what goes into a system by what existing tools support directly, rather than by what users need. + +## Desk Check +An informal review in which the author asks one or more colleagues to read an artefact and look for defects. + +See also: [Review](r.md#review), [Walkthrough](w.md#walkthrough). + +## Detailed Design +Design work below the architecture, covering parts of both high-level and low-level design. + +See also: [High Level Design](h.md#high-level-design), [Low Level Design](l.md#low-level-design). + +## Detailed Schedule +A fine-grained schedule of tasks, effort, people and dependencies, usually built bottom-up from a work plan. It often covers only the next few weeks and rolls forward. + +See also: [Business Schedule](b.md#business-schedule), [Rolling Wave Planning](r.md#rolling-wave-planning), [Project Headlights](p.md#project-headlights). + +## Developer +A software engineer, especially when doing design and construction. + +## Developer Integration Test +**DIT.** An integration test that a developer runs in the system test environment after a project build, before releasing new functionality. + +See also: [Integration Test](i.md#integration-test). + +## Development Environment +The hardware, software and tools in which developers build and test software. + +See also: [Local Development Environment](l.md#local-development-environment), [IDE](i.md#ide). + +## Development process +The set of activities a team follows to develop software. + +See also: [Lifecycle](l.md#lifecycle), [SDLC](s.md#sdlc). + +## DevOps +A way of working that brings development and operations together, sharing responsibility for delivering and running software. It relies on automation of builds, tests, deployment and monitoring. + +See also: [Continuous Integration](c.md#continuous-integration), [Continuous Deployment](c.md#continuous-deployment). + +## DHCP +**Dynamic Host Configuration Protocol.** The protocol a device uses to get an IP address and other network settings, such as the gateway and DNS servers, from a server when it joins a network. + +See also: [BOOTP](b.md#bootp), [DNS](#dns). + +## Diagram +A graphical representation of a system, process or other information. + +## DLE +**Digital Local Exchange.** A digital telephone exchange that connects subscribers' lines, either directly or through remote concentrators. In the UK these were System X and AXE 10 exchanges. + +![DLE](images/15007989.png) + +[[CAPTION:Figure]] DLE. + +![DLE](images/15007995.png) + +[[CAPTION:Figure]] DLE. + +See also: [SystemX](s.md#systemx), [MDF](m.md#mdf), [Local Loop](l.md#local-loop). + +## DLP +**Data Link Protocol.** Any protocol at the data link layer (layer 2) of the OSI model, such as Ethernet, Token Ring, FDDI or PPP. DLP also commonly stands for data loss prevention. + +See also: [L2](l.md#l2). + +## DMZ +**Demilitarised Zone.** A network segment that sits between the internet and an organisation's internal network, holding the services that outsiders need to reach, such as web and mail servers. Firewalls on both sides limit what each zone can reach. + +![DMZ](images/15008001.png) + +[[CAPTION:Figure]] DMZ. + +See also: [Firewall](f.md#firewall). + +## DN +**Directory Number.** A telephone number: the digits that identify a subscriber's line or device and route calls to it. + +See also: [E.164](e.md#e164). + +## DNB +A Dun & Bradstreet business file: a company's credit and identity record from Dun & Bradstreet. + +## DNS +**Domain Name System.** The internet's distributed, hierarchical naming system. Its main job is to translate names such as `example.com` into IP addresses, but it holds many other kinds of record. + +See also: [Domain Name](#domain-name), [FQDN](f.md#fqdn), [DNS - A](#dns-a), [DNS Spoofing](#dns-spoofing). + +## DNS - A {#dns-a} +**Address record.** A DNS record that maps a name to an IPv4 address. + +See also: [DNS - AAAA](#dns-aaaa), [DNS](#dns). + +## DNS - AAAA {#dns-aaaa} +A DNS record that maps a name to an IPv6 address. + +See also: [DNS - A](#dns-a), [IPv6](i.md#ipv6). + +## DNS - CNAME {#dns-cname} +**Canonical name record.** A DNS record that makes one name an alias for another; the lookup carries on with the target name. + +See also: [DNS - DNAME](#dns-dname). + +## DNS - DNAME {#dns-dname} +A DNS record that makes a name, and every name under it, an alias for another domain. A CNAME aliases only the exact name. + +See also: [DNS - CNAME](#dns-cname). + +## DNS - SRV {#dns-srv} +**Service record.** A DNS record that gives the host and port for a named service, such as SIP or XMPP, together with priority and weight. + +See also: [SIP](s.md#sip), [XMPP](x.md#xmpp). + +## DNS - TXT {#dns-txt} +A DNS record that holds arbitrary text. It now mostly carries machine-readable data, such as SPF, DKIM and DMARC policies for email and domain ownership proofs. + +## DNS Spoofing +An attack that puts false records into a DNS resolver's cache, so that it sends users to the wrong address, usually one the attacker controls. Also called DNS cache poisoning. + +See also: [DNS](#dns), [ARP spoofing](a.md#arp-spoofing). + +## DoD +See [Definition of Done](#definition-of-done). + +## DoR +See [Definition of Ready](#definition-of-ready). + +## Docker +A platform for building and running software in containers: packages that hold an application and its dependencies and run in isolation on a shared operating system kernel. + +See also: [VM](v.md#vm). + +## Document Collaboration +Tools that let several people work on the same document at once, from different places, to reach one final version. Wikis, Confluence and SharePoint are examples. + +See also: [Wiki](w.md#wiki). + +## Domain Name +A name registered in the DNS, such as `example.co.uk`, that identifies an area of administrative control on the internet. Names form a hierarchy under the unnamed root: top-level domains, then second-level domains, then subdomains. + +![Domain Name](images/15008007.png) + +[[CAPTION:Figure]] Domain Name. + +![Domain Name](images/15008013.png) + +[[CAPTION:Figure]] Domain Name. + +See also: [DNS](#dns), [FQDN](f.md#fqdn). + +## Dot1Q +**IEEE 802.1Q.** The standard for VLANs on Ethernet. It adds a 4-byte tag to each frame with a 12-bit VLAN identifier and a 3-bit priority (802.1p). + +See also: [VLAN](v.md#vlan), [Dot1Q trunking](#dot1q-trunking), [PCP](p.md#pcp). + +## Dot1Q trunking +Carrying several VLANs over one link between switches. Each frame is tagged with its VLAN ID as it enters the trunk. The switch at the other end uses the tag to deliver it to ports in the same VLAN. + +![Dot1Q trunking](images/15008019.png) + +[[CAPTION:Figure]] Dot1Q trunking. + +![Dot1Q trunking](images/15008025.png) + +[[CAPTION:Figure]] Dot1Q trunking. + +![Dot1Q trunking](images/15008031.png) + +[[CAPTION:Figure]] Dot1Q trunking. + +```mermaid +flowchart LR + A10[Port, VLAN 10] --> S1[Switch 1] + A20[Port, VLAN 20] --> S1 + S1 == "trunk: frames tagged 10 or 20" ==> S2[Switch 2] + S2 --> B10[Port, VLAN 10] + S2 --> B20[Port, VLAN 20] +``` + +[[CAPTION:Figure]] Two VLANs sharing an 802.1Q trunk. + +See also: [Dot1Q](#dot1q), [VLAN](v.md#vlan). + +## Downstream +In a project, the later activities and artefacts, such as coding, testing and deployment. In networking, the direction towards the customer. + +See also: [Upstream](u.md#upstream). + +## DP +**Distribution Point.** In a UK access network, the point near the customer, often on a telegraph pole, where the drop wire to the premises joins the cable from the exchange. In fibre networks, the point where a fibre splits towards several premises. + +![DP](images/15008037.png) + +[[CAPTION:Figure]] DP. + +See also: [Local Loop](l.md#local-loop), [PCP](p.md#pcp). + +## DPI +**Deep Packet Inspection.** Examining the contents of packets, beyond the IP and transport headers, to classify, block, shape or log traffic. Firewalls, intrusion prevention and traffic management use it. + +See also: [Firewall](f.md#firewall), [IPS](i.md#ips), [Packet sniffing](p.md#packet-sniffing). + +## DPNSS +**Digital Private Network Signalling System.** A BT-defined protocol for linking PBXs over digital private circuits, with a common set of features across vendors. QSIG is the international equivalent. + +See also: [QSIG](q.md#qsig), [DASS2](#dass2). + +## DQ +**Data Quality.** How accurate, complete, consistent and timely data is for its purpose. + +## Draft +An artefact under version control but not yet under change control. + +See also: [Baseline](b.md#baseline). + +## Driver +Software that lets an operating system and applications use a particular device, hiding the details of the hardware behind a standard interface. + +See also: [DDK](#ddk), [Kernel](k.md#kernel). + +## DSCP +**Differentiated Services Code Point.** A 6-bit field in the IP header that marks a packet's traffic class, so routers can give it the right treatment under DiffServ quality of service. + +See also: [QoS](q.md#qos), [CoS](c.md#cos). + +## DSDM +**Dynamic Systems Development Method.** An agile framework from the 1990s, built on rapid application development, that fixes time and cost and varies scope. It is now called the DSDM agile project framework. + +See also: [Agile development](a.md#agile-development), [RAD](r.md#rad). + +## DSL +**Digital Subscriber Line.** A family of technologies that carry digital data over copper telephone lines, using frequencies above voice. ADSL and VDSL are the most common. + +![DSL](images/15008043.png) + +[[CAPTION:Figure]] DSL. + +See also: [ADSL](a.md#adsl), [SDSL](s.md#sdsl), [VDSL](v.md#vdsl), [DSLAM](#dslam). + +## DSLAM +**Digital Subscriber Line Access Multiplexer.** The equipment in an exchange or street cabinet that terminates many customers' DSL lines and combines their traffic onto one high-speed link. + +![DSLAM](images/15008049.png) + +[[CAPTION:Figure]] DSLAM. + +See also: [DSL](#dsl), [MSAN](m.md#msan), [BRAS](b.md#bras). + +## DTE +**Data Terminal Equipment.** The device at the end of a data link that is the source or destination of data, such as a computer or terminal. It connects to the line through a DCE. + +![DTE](images/15007971.png) + +[[CAPTION:Figure]] DTE. + +```mermaid +flowchart LR + A[DTE: computer] --- B[DCE: modem] --- C((Line)) --- D[DCE: modem] --- E[DTE: computer] +``` + +[[CAPTION:Figure]] DTEs connected through DCEs. + +See also: [DCE](#dce). + +## DTLS +**Datagram Transport Layer Security.** A version of TLS for datagram transports such as UDP, giving the same confidentiality, integrity and authentication. It is used by CoAP and LwM2M in IoT and by WebRTC. The current version is DTLS 1.3, in RFC 9147. + +See also: [TLS](t.md#tls), [UDP](u.md#udp), [LwM2M](l.md#lwm2m). + +## DTMF +**Dual-Tone Multi-Frequency.** Telephone signalling in which each key sends two tones at once, one from a low group and one from a high group. It is standardised as ITU-T Q.23, and BT calls it MF4. + +![DTMF](images/15008055.png) + +[[CAPTION:Figure]] DTMF. + +See also: [IVR](i.md#ivr), [PSTN](p.md#pstn). + +## DTT +**Digital Terrestrial Television.** Television broadcast digitally from ground-based transmitters, such as Freeview in the UK. It fits several channels into the spectrum one analogue channel used. + +See also: [DTV](#dtv), [COFDM](c.md#cofdm). + +## DTV +**Digital Television.** Television sent as digital signals rather than analogue, by terrestrial, satellite, cable or IP networks. + +See also: [DTT](#dtt), [IPTV](i.md#iptv). + +## Duplex +A link that can carry communication in both directions. In full duplex both sides can send at once; in half duplex they take turns. A simplex link carries one direction only. + +![Duplex](images/15008061.png) + +[[CAPTION:Figure]] Duplex. + +![Duplex](images/15008067.png) + +[[CAPTION:Figure]] Duplex. + +![Duplex](images/15008073.png) + +[[CAPTION:Figure]] Duplex. + +![Duplex](images/15008079.png) + +[[CAPTION:Figure]] Duplex. + +```mermaid +flowchart LR + subgraph Simplex + S1[A] --> S2[B] + end + subgraph "Half duplex: one way at a time" + H1[A] <-.-> H2[B] + end + subgraph "Full duplex: both ways at once" + F1[A] --> F2[B] + F2 --> F1 + end +``` + +[[CAPTION:Figure]] Simplex, half duplex and full duplex. + +See also: [Full duplex](f.md#full-duplex), [Half Duplex](h.md#half-duplex), [Simplex](s.md#simplex). + +## DVSR +**Dynamically Verified Static Routing.** A Redback (later Ericsson) routing feature: a static route that the router adds to its table only while regular pings show the next hop is reachable. + +See also: [Router](r.md#router). diff --git a/content/glossary/e.md b/content/glossary/e.md new file mode 100644 index 0000000..ee60466 --- /dev/null +++ b/content/glossary/e.md @@ -0,0 +1,253 @@ +--- +title: E +--- + +# E + +## E NodeB +**Evolved Node B (eNodeB, eNB).** The LTE base station. Unlike a UMTS Node B, it has no separate controller: it handles radio resource management and handover itself, and connects directly to the core. + +![E NodeB](images/14614584.png) + +[[CAPTION:Figure]] E NodeB. + +See also: [E-UTRAN](#e-utran), [Node B](n.md#node-b), [MME](m.md#mme), [LTE (4G)](l.md#lte-4g). + +## E-UTRAN +**Evolved UMTS Terrestrial Radio Access Network.** The LTE radio access network: eNodeBs and the E-UTRA air interface, which uses OFDMA downstream and SC-FDMA upstream. + +![E-UTRAN](images/15008132.png) + +[[CAPTION:Figure]] E-UTRAN. + +See also: [E NodeB](#e-nodeb), [UTRAN](u.md#utran), [LTE (4G)](l.md#lte-4g). + +## E.164 {#e164} +The ITU-T recommendation for the international public telephone numbering plan. An E.164 number has at most 15 digits, starting with the country code, and is written with a leading `+`. + +See also: [DN](d.md#dn). + +## EAD +**Ethernet Access Direct.** Openreach's point-to-point fibre Ethernet circuit, used to connect business sites and for backhaul, at speeds up to 10 Gbit/s. + +![EAD](images/15008088.png) + +[[CAPTION:Figure]] EAD. + +See also: [EoFTTC](#eofttc), [Leased Line](l.md#leased-line), [Backhaul](b.md#backhaul). + +## EAP +**Extensible Authentication Protocol.** An authentication framework, defined in RFC 3748, that carries many authentication methods, such as EAP-TLS and PEAP. IEEE 802.1X uses it for network access control on Wi-Fi and wired networks. + +![EAP](images/15008096.png) + +[[CAPTION:Figure]] EAP. + +See also: [PEAP](p.md#peap), [WPA2](w.md#wpa2), [Radius](r.md#radius). + +## Earned Value +**EV.** A measure of progress that compares the value of the work completed with the work planned and its actual cost, to show schedule and cost variance. + +See also: [Earned Value Management](#earned-value-management). + +## Earned Value Management +**EVM.** Managing a project using earned value measures. + +See also: [Earned Value](#earned-value). + +## Echo +A delayed reflection of a signal. On telephone calls it comes from hybrids that convert between 2-wire and 4-wire circuits, or from a loudspeaker feeding back into a microphone. Delay makes it more noticeable. + +See also: [Echo cancellation](#echo-cancellation), [Sidetone](s.md#sidetone). + +## Echo cancellation +Removing echo from a call by modelling the echo path and subtracting an estimate of the echo from the incoming signal. It replaced echo suppression, which muted the return path while the far end was talking. Line echo cancellers handle network echo; acoustic echo cancellers handle loudspeaker-to-microphone echo. + +See also: [Echo](#echo), [Echo Cancellation - G.168](#echo-cancellation-g168). + +## Echo Cancellation - G.168 {#echo-cancellation-g168} +ITU-T Recommendation G.168, which specifies digital network echo cancellers. It covers their performance, how tones disable them for fax and data, and the laboratory tests to check them. + +See also: [Echo cancellation](#echo-cancellation). + +## EDB +**Emergency Database.** In UK telephony, the database that tells emergency services the address behind the number calling 999. + +## EDGE +**Enhanced Data rates for GSM Evolution.** An upgrade to GSM data, also called EGPRS, that uses better modulation and coding to roughly triple GPRS speeds on the same radio channels. + +See also: [GPRS](g.md#gprs), [GSM](g.md#gsm), [GERAN](g.md#geran). + +## Editor +In an inspection, the person who makes the fixes to the artefact afterwards, usually its author. + +See also: [Inspection](i.md#inspection). + +## EFM +**Ethernet in the First Mile.** IEEE 802.3ah, which defines Ethernet over the access link between provider and customer, on copper or fibre, with its own management functions. + +See also: [Ethernet over twisted pair](#ethernet-over-twisted-pair), [EoFTTC](#eofttc). + +## Egress +Traffic leaving a network or a device, as opposed to ingress traffic coming in. + +![Egress](images/15008102.png) + +[[CAPTION:Figure]] Egress. + +See also: [Ingress](i.md#ingress). + +## EIA +**Electronic Industries Alliance.** A former US standards and trade body for electronics manufacturers, known for standards such as RS-232. It closed in 2011 and its work passed to other bodies, including the TIA. + +See also: [TIA](t.md#tia). + +## EIGRP +**Enhanced Interior Gateway Routing Protocol.** Cisco's advanced distance-vector routing protocol, which sends only incremental updates. It replaced IGRP in 1993 and was published as RFC 7868 in 2016. + +See also: [IGRP](i.md#igrp), [OSPF](o.md#ospf), [IGP](i.md#igp). + +## Embedded systems programming +Writing software for an embedded system, a computer built into a device, using the interfaces and limited resources the device provides. + +See also: [RTOS](r.md#rtos), [MCU](m.md#mcu), [C Language](c.md#c-language). + +## Engineering Discussion +A meeting to frame a problem and look for solutions. If materials go out beforehand for people to read, it is closer to a preview. + +See also: [Preview](p.md#preview). + +## Engineering Management +**EM.** Planning, staffing, tracking and controlling software projects, together with managing the teams and the organisation. + +## Engineering Process +Software engineering process: defining how software engineering work is done, and improving it. + +See also: [Process](p.md#process). + +## EoFTTC +**Ethernet over FTTC.** A business Ethernet service over fibre to the cabinet: copper runs from the premises to the street cabinet, and fibre from there to the exchange. It gives symmetrical speeds of tens of megabits per second at lower cost than a fibre leased line. + +![EoFTTC](images/15008108.png) + +[[CAPTION:Figure]] EoFTTC. + +See also: [EAD](#ead), [VDSL](v.md#vdsl), [EFM](#efm). + +## EoS +**Ethernet over SDH or SONET.** Protocols that carry Ethernet over SDH or SONET networks. They usually use the generic framing procedure (GFP) and virtual concatenation to fit the Ethernet rate into SDH paths. + +See also: [SDH](s.md#sdh), [SONET](s.md#sonet). + +## ePDG +**Evolved Packet Data Gateway.** The LTE core node that lets devices on untrusted non-3GPP access, such as public Wi-Fi, reach the mobile core through an IPsec tunnel. Wi-Fi calling relies on it. + +See also: [PGW](p.md#pgw), [IPSec](i.md#ipsec), [ANDSF](a.md#andsf). + +## Epic +A large user story, too big for one iteration, that is split into smaller stories. + +See also: [User Stories](u.md#user-stories), [Story Splitting](s.md#story-splitting). + +## EPWS +**Ethernet Private Wire Service.** A point-to-point Ethernet service between two customer sites, carried over the provider's network. + +See also: [EVPL](#evpl), [VPWS](v.md#vpws). + +## ESB +**Enterprise Service Bus.** Middleware in a service-oriented architecture that connects applications through a shared bus. It routes, transforms and mediates messages between them, so that each application integrates with the bus rather than with every other application. + +![ESB](images/15008114.png) + +[[CAPTION:Figure]] ESB. + +See also: [SOA](s.md#soa), [Microservice](m.md#microservice). + +## Estimate +The result of estimating: the predicted size, effort, cost or duration, with the inputs, assumptions and method behind it. It can be a full project estimate or a one-line task estimate. + +See also: [Estimation](#estimation), [Project Estimate](p.md#project-estimate), [Task Estimate](t.md#task-estimate). + +## Estimation +Working out the likely size, effort, cost, schedule or quality of a project. Several techniques used together give a better estimate than one, and estimates should be redone as the project goes on. + +See also: [Estimate](#estimate), [Cone of Uncertainty](c.md#cone-of-uncertainty), [Planning Poker](p.md#planning-poker). + +## Ethernet over twisted pair +Ethernet physical layers that use twisted-pair copper cable. 10BASE-T, 100BASE-TX, 1000BASE-T and 10GBASE-T are the main ones. Single-pair variants, such as 10BASE-T1L, serve industrial and automotive uses. + +![Ethernet over twisted pair](images/15138870.png) + +[[CAPTION:Figure]] Ethernet over twisted pair. + +![Ethernet over twisted pair](images/14614590.png) + +[[CAPTION:Figure]] Ethernet over twisted pair. + +![Ethernet over twisted pair](images/15008120.png) + +[[CAPTION:Figure]] Ethernet over twisted pair. + +See also: [Cat 5](c.md#cat-5), [Cat 6](c.md#cat-6), [Twisted pair](t.md#twisted-pair). + +## Ethical Hacking +Testing an organisation's security with its permission, using attackers' methods to find weaknesses before they do. Ethical hackers are also called white hats. + +See also: [Pen testing](p.md#pen-testing). + +## ETL +**Extract, Transform, Load.** The three steps for moving data into a data warehouse: read data from source systems, convert and clean it into the target form, and write it into the target. + +![ETL](images/15008126.png) + +[[CAPTION:Figure]] ETL. + +See also: [Data Warehouse](d.md#data-warehouse). + +## ETSI +**European Telecommunications Standards Institute.** The not-for-profit body, based in Sophia Antipolis in France, that sets European telecoms standards. Its work includes GSM, DECT and TETRA, and it is a founding partner of 3GPP. + +See also: [ITU](i.md#itu), [GSM](g.md#gsm). + +## EVC +**Ethernet Virtual Connection.** In Metro Ethernet Forum (MEF) terms, the association between two or more customer interfaces (UNIs) across a provider's network. It is a virtual pipe that defines which sites can exchange frames. + +See also: [EVPL](#evpl), [UNI](u.md#uni). + +## Evolutionary Delivery Lifecycle +A lifecycle that combines evolutionary prototyping with staged delivery. + +See also: [Evolutionary Prototyping Lifecycle](#evolutionary-prototyping-lifecycle), [Staged Delivery Lifecycle](s.md#staged-delivery-lifecycle). + +## Evolutionary Prototyping Lifecycle +A lifecycle in which a system grows from an initial concept through repeated prototypes until it is ready to deliver. + +See also: [Lifecycle](l.md#lifecycle), [Iterative Development](i.md#iterative-development). + +## EVPL +**Ethernet Virtual Private Line.** An MEF E-Line service that carries several point-to-point virtual connections over one customer interface, separated by VLAN. Its sibling, Ethernet Private Line (EPL), gives one transparent connection per port. + +See also: [EVC](#evc), [EPWS](#epws). + +## Expert Judgement +Estimating from the experience of people who know the work. It works best when combined with other techniques, such as analogy and decomposition. + +See also: [Estimation](#estimation), [Analogy Estimation](a.md#analogy-estimation). + +## Explicit Change Control +Change control of artefacts, such as requirements and plans, that a change control board manages directly. + +See also: [Implicit Change Control](i.md#implicit-change-control), [Change Control Board](c.md#change-control-board). + +## Explicit Risk Management +See [Extrinsic Risk Management](#extrinsic-risk-management). + +## Exploratory Testing +Testing in which the tester learns about the software, designs tests and runs them all at once, rather than following a script. + +See also: [Informal Testing](i.md#informal-testing). + +## Extrinsic Risk Management +Formal risk management added to a project, such as keeping a top-ten risk list and reviewing it regularly. + +See also: [Intrinsic Risk Management](i.md#intrinsic-risk-management), [Risk Management](r.md#risk-management). diff --git a/content/glossary/f.md b/content/glossary/f.md new file mode 100644 index 0000000..8464fb9 --- /dev/null +++ b/content/glossary/f.md @@ -0,0 +1,179 @@ +--- +title: F +--- + +# F + +## Facilitation +Running a meeting or workshop so the group reaches its goal, without taking part in the content. The person who does it is the facilitator. + +## FCS +**Frame Check Sequence.** An error-detecting code, usually a CRC, added to the end of a frame. A receiver that finds a mismatch discards the frame; protocols higher up, such as TCP, recover the loss. + +See also: [CRC](c.md#crc). + +## FDDI +**Fibre Distributed Data Interface.** A 100 Mbit/s token-passing LAN standard over a dual fibre ring. Fast Ethernet made it obsolete. + +See also: [Token Ring](t.md#token-ring), [100 Mbps (Fast Ethernet)](numbers.md#100-mbps-fast-ethernet). + +## Feature +A piece of a system's functionality, seen from the user's side. The term also refers to a project's scope, measured in features. + +See also: [Feature Test](#feature-test), [Feature creep](#feature-creep). + +## Feature creep +The tendency for requirements to keep growing during a project, adding features that were never planned, at a cost to schedule and quality. + +See also: [Change Control](c.md#change-control). + +## Feature Test +A test of one area of system functionality once its construction is finished. + +## FemtoCell +A small, low-power mobile base station for a home or small office, connected to the operator over the customer's broadband. It improves indoor coverage for a handful of phones. + +![FemtoCell](images/15008172.png) + +[[CAPTION:Figure]] FemtoCell. + +See also: [Mobile Cell](m.md#mobile-cell), [E NodeB](e.md#e-nodeb). + +## FIPS +**Federal Information Processing Standards.** US federal government standards for computer systems, issued by NIST, covering areas such as cryptography where no suitable industry standard exists. + +See also: [FIPS 140-2](#fips-140-2), [NIST](n.md#nist). + +## FIPS 140-2 +The US standard, from 2001, of security requirements for cryptographic modules. It defines four security levels and covers module design, ports, roles, physical security and key management. FIPS 140-3 replaced it in 2019, and 140-2 certificates are being retired. + +See also: [FIPS](#fips), [NIST](n.md#nist). + +## Firewall +A network security device or software that allows or blocks traffic by rules, usually between a trusted internal network and an untrusted one. Network firewalls protect whole networks; host-based firewalls protect one machine. + +![Firewall](images/15008180.png) + +[[CAPTION:Figure]] Firewall. + +![Firewall](images/15008186.png) + +[[CAPTION:Figure]] Firewall. + +See also: [DMZ](d.md#dmz), [IPS](i.md#ips), [DPI](d.md#dpi). + +## FNA +**Full Network Authentication.** A broadband login method in which the network authenticates the line itself, by its network identifier, through RADIUS, rather than a username and password typed by the customer. + +See also: [Radius](r.md#radius), [PPPoE](p.md#pppoe). + +## Formal Testing +Testing that follows documented test cases. + +See also: [Informal Testing](i.md#informal-testing), [Test Case](t.md#test-case). + +## Fortran +The first widely used high-level language, developed at IBM in the 1950s for scientific computing. Modern Fortran is still used for numerical work and supercomputing. + +See also: [COBOL](c.md#cobol). + +## Forwarding plane +See [Data Plane](d.md#data-plane). + +## FOTA +**Firmware Over-The-Air.** Updating device firmware remotely over a wireless link. A safe design verifies a signature on each image and keeps two partitions, so a failed update rolls back. It also rolls updates out to the fleet in stages. + +See also: [Root of Trust](r.md#root-of-trust), [LwM2M](l.md#lwm2m). + +## FQDN +**Fully Qualified Domain Name.** A domain name that names every level of the DNS hierarchy down from the root, such as `www.example.com.`, so it can mean only one thing. + +![FQDN](images/15008192.png) + +[[CAPTION:Figure]] FQDN. + +See also: [Domain Name](d.md#domain-name), [DNS](d.md#dns). + +## Frame Relay +A packet-switched WAN technology, widely sold by carriers in the 1990s, that connects sites through virtual circuits over a shared network. It was cheaper than leased lines; MPLS VPNs have since replaced it. + +![Frame Relay](images/15008198.png) + +[[CAPTION:Figure]] Frame Relay. + +See also: [X.25](x.md#x25), [PVC](p.md#pvc), [L3VPN](l.md#l3vpn). + +## Frequent Releases +Putting the product into users' hands often, so the team gets feedback early and acts on it. + +See also: [Continuous Deployment](c.md#continuous-deployment), [Release](r.md#release). + +## Frequently Asked Question +**FAQ.** A list of common questions and their answers. + +## FSK +**Frequency-Shift Keying.** Modulation that sends digital data by switching the carrier between frequencies; in binary FSK, one frequency for 1 and another for 0. Caller ID and early modems use it. + +![FSK](images/15008204.png) + +[[CAPTION:Figure]] FSK. + +See also: [QAM](q.md#qam). + +## FTP +**File Transfer Protocol.** A protocol for transferring files between a client and a server, using separate control and data connections. It sends credentials in clear text, so FTPS or SFTP should be used instead. + +See also: [FTPS](#ftps), [SFTP](s.md#sftp). + +## FTPS +**FTP Secure.** FTP with TLS encryption added. It is a different protocol from SFTP, which runs over SSH. + +See also: [FTP](#ftp), [SFTP](s.md#sftp), [TLS](t.md#tls). + +## Full duplex +Communication in which both ends can send and receive at the same time, like a telephone call. + +See also: [Duplex](d.md#duplex), [Half Duplex](h.md#half-duplex). + +## Function Point +A measure of software size based on the functionality it gives users, counted from its inputs, outputs, enquiries, files and interfaces. + +See also: [Lines of Code](l.md#lines-of-code), [Estimation](e.md#estimation). + +## Functional Requirement +Something a system must do to meet the business requirements. Also called a [What Requirement](w.md#what-requirement). + +See also: [Non-functional Requirement](n.md#non-functional-requirement), [Business Requirement](b.md#business-requirement). + +## Functional Testing +Testing a system's behaviour against what is expected, without regard to how it works inside. Also called black-box testing. + +See also: [Black-Box Testing](b.md#black-box-testing). + +## Fuzzy Front End +The early, ill-defined stage of a project, from the first idea to the approval of the project charter. + +See also: [Project Charter](p.md#project-charter). + +## Fuzzy Logic +In estimation, sorting items into rough size bands, such as small, medium and large, each standing for an overlapping range of values. In computing generally, fuzzy logic means reasoning with degrees of truth between 0 and 1. + +See also: [T-Shirt Sizing](t.md#t-shirt-sizing), [Analogy Estimation](a.md#analogy-estimation). + +## FXO +**Foreign Exchange Office.** The port on a device, such as a phone, fax or VoIP gateway, that plugs into an analogue telephone line and receives dial tone and ringing from it. + +![FXO](images/15008210.png) + +[[CAPTION:Figure]] FXO. + +See also: [FXS](#fxs). + +## FXS +**Foreign Exchange Station.** The port that provides an analogue telephone line: it supplies power, dial tone and ringing to the phone connected to it. An exchange line or an analogue adapter has FXS ports. + +![FXS](images/15008210.png) + +[[CAPTION:Figure]] FXS. + +See also: [FXO](#fxo). diff --git a/content/glossary/g-k.md b/content/glossary/g-k.md deleted file mode 100644 index ce0dea5..0000000 --- a/content/glossary/g-k.md +++ /dev/null @@ -1,711 +0,0 @@ ---- -title: Glossary, G to K ---- - -# G to K - -## G - -### G.711 {#g711} -The ITU-T standard, from 1972, for coding telephone speech as 64 kbit/s PCM: 8,000 samples a second, each companded to 8 bits. It has two variants, A-law and µ-law. It is the baseline codec that every VoIP system supports. - -See also: [A-law](a-c.md#a-law), [Mu-law](l-n.md#mu-law), [PCM](o-r.md#pcm), [G.711.1](#g7111). - -### G.711.1 {#g7111} -A wideband extension of G.711, from 2008, that adds layers on top of a G.711 core. It runs at 64, 80 or 96 kbit/s, and a G.711-only receiver can still decode the core. - -See also: [G.711](#g711), [G.722](#g722). - -### G.719 {#g719} -An ITU-T codec for full-band audio (20 Hz to 20 kHz) at 32 to 128 kbit/s, developed by Polycom and Ericsson for high-quality conferencing. It handles music as well as speech. - -See also: [G.722](#g722), [Audio Codecs](a-c.md#audio-codecs). - -### G.722 {#g722} -An ITU-T wideband speech codec from 1988, covering 50 to 7,000 Hz at 48 to 64 kbit/s using sub-band ADPCM. It is widely used for HD voice on desk phones. G.722.1 and G.722.2 (AMR-WB) are different codecs despite the names. - -See also: [G.711](#g711), [AMR-WB](a-c.md#amr-wb). - -### G.723.1 {#g7231} -An ITU-T low-rate speech codec, at 5.3 or 6.3 kbit/s, with 30 ms frames. It was common in early VoIP and H.323 systems. It is unrelated to the older G.723. - -See also: [G.726](#g726), [H.323](#h323). - -### G.726 {#g726} -An ITU-T ADPCM speech codec at 16, 24, 32 or 40 kbit/s, which replaced G.721 and G.723. The 32 kbit/s rate, half that of G.711, is the most used, for example in DECT cordless phones. - -See also: [G.711](#g711), [PCM](o-r.md#pcm). - -### G.Fast {#gfast} -An ITU-T DSL standard (G.9700 and G.9701) for very short copper lines, up to a few hundred metres, giving several hundred megabits per second. It is used from a distribution point or cabinet fed by fibre. - -See also: [VDSL](s-z.md#vdsl), [DSL](d-f.md#dsl), [DP](d-f.md#dp). - -### Gantt chart -A bar chart of a project schedule, with tasks down the side and time along the top. It shows when each task starts and ends, and how the tasks depend on each other. - -See also: [PERT chart](o-r.md#pert-chart), [CPM](a-c.md#cpm). - -### Gap analysis -Comparing where things are now with where they need to be, to find what must change to get there. - -### Gate -A set of criteria that must be met before work can pass from one phase or stage to the next. - -See also: [Gate Checklist](#gate-checklist), [Phase](o-r.md#phase). - -### Gate Checklist -A checklist used to check that the criteria for a gate have been met. - -See also: [Gate](#gate). - -### GBE / GE {#gbe-ge} -Abbreviations for [Gigabit Ethernet](#gigabit-ethernet). - -### GCHQ -**Government Communications Headquarters.** The UK's signals intelligence and security agency, based in Cheltenham. It grew out of the Government Code and Cypher School of Bletchley Park, and runs the National Cyber Security Centre. - -See also: [NCSC](l-n.md#ncsc), [CESG](a-c.md#cesg). - -### Geocast -Delivering a message to all the nodes in a geographical area, rather than to a list of addresses. It is a form of multicast used in some mobile ad hoc and vehicle networks. - -See also: [Multicast](l-n.md#multicast), [MANET](l-n.md#manet). - -### GERAN -**GSM EDGE Radio Access Network.** The radio access network of GSM and EDGE: the base stations, their controllers and the interfaces between them. 3GPP maintains its standards. - -See also: [BTS](a-c.md#bts), [BSC](a-c.md#bsc), [EDGE](d-f.md#edge), [UTRAN](s-z.md#utran). - -### GGSN -**Gateway GPRS Support Node.** The GPRS and UMTS core node that connects the mobile packet network to external networks such as the internet. It allocates the device's IP address and anchors the session as the user moves. In LTE the PGW does this job. - -See also: [SGSN](s-z.md#sgsn), [GPRS](#gprs), [APN](a-c.md#apn), [PGW](o-r.md#pgw). - -### Gherkin -The plain-language format used by Cucumber and similar tools to write behaviour-driven tests. Scenarios are written as `Given`, `When` and `Then` steps under a `Feature`, so that business people and developers can read the same tests. - -See also: [BDD](a-c.md#bdd), [Given When Then](#given-when-then). - -### Gigabit Ethernet -Ethernet at 1 Gbit/s, defined in IEEE 802.3. It replaced Fast Ethernet as the usual speed for wired networks from the early 2000s. - -See also: [GBE / GE](#gbe-ge), [100 Mbps (Fast Ethernet)](s-z.md#100-mbps-fast-ethernet), [Ethernet over twisted pair](d-f.md#ethernet-over-twisted-pair). - -### Git -A distributed version control system, created by Linus Torvalds in 2005 for the Linux kernel. Every clone holds the full history, so most work is local and fast, and branching and merging are cheap. - -See also: [Version Control](s-z.md#version-control), [Subversion](s-z.md#subversion). - -### Given When Then -A template for writing acceptance criteria and tests: given some context, when an action happens, then expect these results. - -See also: [Gherkin](#gherkin), [BDD](a-c.md#bdd). - -### GMSC -**Gateway Mobile Switching Centre.** An MSC function that routes calls between the mobile network and other networks. For an incoming call it asks the HLR where the subscriber is. - -See also: [MSC](l-n.md#msc), [HLR](#hlr). - -### GNU -**GNU's Not Unix.** The free software operating system started by Richard Stallman in 1983, made up of tools, libraries and compilers. Combined with the Linux kernel, it forms most Linux distributions. - -See also: [GPL](#gpl), [Linux](l-n.md#linux), [Copyleft](a-c.md#copyleft). - -### Go -A statically typed, compiled language created at Google in 2009 by Robert Griesemer, Rob Pike and Ken Thompson. It has garbage collection, built-in concurrency with goroutines and channels, and fast builds. It is also called Golang. - -### Goals -The results a project, process or activity is meant to achieve. Also called objectives. - -See also: [Objectives](o-r.md#objectives). - -### GoF -**Gang of Four.** The four authors (Gamma, Helm, Johnson and Vlissides) of *Design Patterns: Elements of Reusable Object-Oriented Software* (1994). The book catalogues 23 object-oriented design patterns. - -See also: [Design Pattern](d-f.md#design-pattern). - -### Google Cloud Platform -Google's cloud computing services, offering infrastructure, platform and serverless computing, data analytics and machine learning on the infrastructure Google uses itself. - -See also: [AWS](a-c.md#aws), [Azure](a-c.md#azure). - -### GPL -<!-- vale PyDoc.Spelling = NO --> -**GNU General Public License.** -<!-- vale PyDoc.Spelling = YES --> -The most widely used copyleft free software licence, written by Richard Stallman. It lets anyone run, study, share and change the software, as long as they share copies and derived works under the same licence. - -See also: [Copyleft](a-c.md#copyleft), [GNU](#gnu). - -### GPRS -**General Packet Radio Service.** The packet data service added to GSM, giving always-on connections charged by data volume rather than by time. It is best-effort, with speeds of tens of kilobits per second. - -See also: [GSM](#gsm), [EDGE](d-f.md#edge), [SGSN](s-z.md#sgsn), [GGSN](#ggsn). - -### GSM -**Global System for Mobile communications.** The second-generation (2G) digital mobile standard, developed by ETSI and first launched in 1991. It was designed for circuit-switched voice and SMS, and GPRS and EDGE later added data. - -See also: [GPRS](#gprs), [EDGE](d-f.md#edge), [UMTS](s-z.md#umts), [SIM](s-z.md#sim). - -### GSM AMR -The AMR speech codec as used in GSM. See [AMR-NB](a-c.md#amr-nb). - -### GSM-FR -**GSM Full Rate.** The first GSM speech codec (GSM 06.10), at 13 kbit/s. Its quality is poor by modern standards; Enhanced Full Rate and AMR replaced it. - -See also: [GSM AMR](#gsm-amr), [AMR-NB](a-c.md#amr-nb). - -### GSM-R -**GSM for Railways.** A version of GSM for railway voice and data, including the signalling of the European Rail Traffic Management System. It is designed to work at train speeds up to 500 km/h. - -See also: [GSM](#gsm). - -### GSN -**GPRS Support Node.** A node in the GPRS core network: either a serving node (SGSN) or a gateway node (GGSN). - -See also: [SGSN](s-z.md#sgsn), [GGSN](#ggsn). - -## H - -### H.248 {#h248} -**Gateway Control Protocol (Megaco).** The ITU-T and IETF protocol with which a media gateway controller or softswitch controls media gateways, which convert between telephone circuits and IP media streams. It does the same job as MGCP. - -See also: [MGCP](l-n.md#mgcp), [MGW](l-n.md#mgw), [Softswitch](s-z.md#softswitch). - -### H.323 {#h323} -An ITU-T family of protocols for voice and video calls over packet networks, covering signalling, media and bandwidth control. It was the main VoIP standard before SIP replaced it. - -See also: [SIP](s-z.md#sip), [Q.931](o-r.md#q931), [RAS](o-r.md#ras). - -### Half Duplex -Communication in which both ends can send, but only one at a time, like a push-to-talk radio. - -See also: [Duplex](d-f.md#duplex), [Full duplex](d-f.md#full-duplex), [Simplex](s-z.md#simplex). - -### Happy Path -See [Nominal Path](l-n.md#nominal-path). - -### Hash Function -A function that maps data of any size to a fixed-size value. A cryptographic hash, such as SHA-256, is also one-way and collision-resistant. That makes it useful for integrity checks, digital signatures and storing passwords. - -See also: [SHA](s-z.md#sha), [MD5](l-n.md#md5), [CRC](a-c.md#crc). - -### HDF -**Handover Distribution Frame.** In UK local loop unbundling, the frame in the exchange where the copper pairs are handed over from the main distribution frame to another operator's equipment. - -See also: [MDF](l-n.md#mdf), [Local Loop](l-n.md#local-loop). - -### HDLC -**High-Level Data Link Control.** An ISO bit-oriented, synchronous data link protocol. Many other link protocols, including PPP framing, LAPB in X.25 and LAPD in ISDN, are based on it. - -See also: [PPP](o-r.md#ppp), [X.25](s-z.md#x25), [Frame Relay](d-f.md#frame-relay). - -### HDSL -**High-bit-rate Digital Subscriber Line.** A symmetrical DSL that carries a 2 Mbit/s (E1) or 1.5 Mbit/s (T1) circuit over two or three copper pairs. It was used to deliver leased lines without new cable. - -See also: [SDSL](s-z.md#sdsl), [DSL](d-f.md#dsl). - -### Heartbeat Retrospective -A retrospective held at a regular interval, such as the end of each iteration, in which the team reflects on recent events and agrees improvements. - -See also: [Sprint retrospective](s-z.md#sprint-retrospective). - -### HiFi Prototype -See [High Fidelity Prototype](#high-fidelity-prototype). - -### High Fidelity Prototype -A prototype that looks and behaves much like the finished system, usually built with real development tools. - -See also: [Low Fidelity Prototype](l-n.md#low-fidelity-prototype), [User Interface Prototype](s-z.md#user-interface-prototype). - -### High Level Design -**HLD.** The design level between the architecture and the low-level design. It defines the components of the solution, their interfaces and how they interact, without going into how each is built. - -See also: [Low Level Design](l-n.md#low-level-design), [Architecture](a-c.md#architecture). - -### HLD -See [High Level Design](#high-level-design). - -### HLR -**Home Location Register.** The central database of a GSM or UMTS network, holding each subscriber's details: their IMSI, phone numbers, services and current location. In LTE and IMS, the HSS replaced it. - -See also: [VLR](s-z.md#vlr), [HSS](#hss), [IMSI](#imsi), [MSC](l-n.md#msc). - -### Hotfix -A small, urgent change, sometimes called a patch, that fixes a bug in released software outside the normal release cycle. - -See also: [Release](o-r.md#release). - -### How Requirement -Usually a synonym for [Non-functional Requirement](l-n.md#non-functional-requirement); sometimes used for design constraints. - -### How Well Requirement -See [Non-functional Requirement](l-n.md#non-functional-requirement). - -### HSRP -**Hot Standby Router Protocol.** Cisco's protocol for a fault-tolerant default gateway. Two or more routers share a virtual IP address; one is active and the others take over if it fails. VRRP is the open-standard equivalent. - -See also: [Router](o-r.md#router). - -### HSS -**Home Subscriber Server.** The master subscriber database for IMS and LTE, holding user profiles and authentication data and tracking where users are registered. It takes over the jobs of the HLR and the authentication centre. - -See also: [HLR](#hlr), [IMS](#ims), [CSCF](a-c.md#cscf), [MME](l-n.md#mme). - -### HTTP -**Hypertext Transfer Protocol.** The request-response protocol of the web. A client asks for a resource by URL with a method such as `GET` or `POST`, and the server returns a status and content. HTTP/2 and HTTP/3 keep the same model with more efficient transports. - -See also: [HTTPS](#https), [REST](o-r.md#rest), [URL](s-z.md#url). - -### HTTPS -HTTP over TLS. It encrypts the traffic and lets the client check the server's identity through its certificate, protecting against eavesdropping and tampering. - -See also: [HTTP](#http), [TLS](s-z.md#tls), [CA](a-c.md#ca). - -### Hub -A network device that repeats every frame it receives to all its other ports, so all connected devices share one collision domain. It works at layer 1, and switches have replaced it. - -See also: [Switch](s-z.md#switch), [L1](l-n.md#l1). - -### Human factors -The study of how people behave, physically and mentally, with products, systems and environments, and how to design for them. - -See also: [User Experience](s-z.md#user-experience). - -### Hungarian notation -A naming convention in which a prefix on an identifier shows its kind or type, such as `cApples` for a count or `szName` for a zero-terminated string. Most modern style guides advise against encoding the type in this way. - -See also: [Camel Case](a-c.md#camel-case), [Snake Case](s-z.md#snake-case). - -## I - -### IA -**Information Assurance.** Managing the risks to information and the systems that handle it, so that it keeps its confidentiality, integrity, availability, authenticity and non-repudiation. - -See also: [CESG](a-c.md#cesg), [NCSC](l-n.md#ncsc). - -### IaaS -**Infrastructure as a Service.** Cloud computing that rents virtual machines, storage and networks, which the customer manages from the operating system up. - -See also: [PaaS](o-r.md#paas), [SaaS](s-z.md#saas), [NaaS](l-n.md#naas). - -### ICE -**Interactive Connectivity Establishment.** A technique, defined in RFC 8445, for finding the most direct working path between two peers behind NATs and firewalls. It tries candidate addresses gathered with STUN and TURN. SIP and WebRTC media use it. - -See also: [STUN](s-z.md#stun), [NAT](l-n.md#nat). - -### ICMP -**Internet Control Message Protocol.** The IP protocol that carries error and diagnostic messages, such as "destination unreachable" and the echo request and reply used by `ping`. - -See also: [ICMPv6](#icmpv6), [Internet Protocol Suite](#internet-protocol-suite). - -### ICMPv6 -ICMP for IPv6, defined in RFC 4443. As well as errors and `ping`, it carries Neighbour Discovery, which replaces ARP, and Multicast Listener Discovery, which replaces IGMP. - -See also: [ICMP](#icmp), [IPv6](#ipv6). - -### IDE -**Integrated Development Environment.** An application that combines a code editor, build tools and a debugger, and often much more, for writing software. - -See also: [Integrated Development Environment](#integrated-development-environment). - -### IDQ -**Informatica Data Quality.** Informatica's tool for profiling, cleansing and standardising enterprise data. - -See also: [DQ](d-f.md#dq). - -### IEC -**International Electrotechnical Commission.** The international body that publishes standards for electrical and electronic technologies. It works with ISO on information technology standards, such as the ISO/IEC series. - -See also: [ISO](#iso). - -### IEEE -**Institute of Electrical and Electronics Engineers.** The world's largest professional association of engineers. It publishes many standards, including the 802 family for LANs: Ethernet (802.3) and Wi-Fi (802.11). - -See also: [802.11](s-z.md#80211), [802.3 Physical Layer](s-z.md#8023-physical-layer). - -### IEEE 802.1ah {#ieee-8021ah} -**Provider Backbone Bridges (PBB), or MAC-in-MAC.** An IEEE standard that wraps customer Ethernet frames in a provider's own Ethernet header. Providers can then scale carrier Ethernet without learning customer MAC addresses or running out of VLAN IDs. - -See also: [IEEE 802.1Qay](#ieee-8021qay), [VLAN](s-z.md#vlan). - -### IEEE 802.1Qay {#ieee-8021qay} -**Provider Backbone Bridge Traffic Engineering (PBB-TE).** A carrier Ethernet standard that builds on 802.1ah MAC-in-MAC. It removes flooding and spanning tree and has the operator set up paths explicitly, giving predictable, SDH-like behaviour. - -See also: [IEEE 802.1ah](#ieee-8021ah), [SDH](s-z.md#sdh). - -### IETF -**Internet Engineering Task Force.** The open standards body that develops internet protocols, such as TCP/IP, HTTP and TLS, and publishes them as RFCs. - -See also: [RFC](o-r.md#rfc). - -### IGMP -**Internet Group Management Protocol.** The IPv4 protocol with which hosts tell their local routers which multicast groups they want to receive. IPTV uses it to join channels. - -See also: [Multicast](l-n.md#multicast), [IPTV](#iptv). - -### IGP -**Interior Gateway Protocol.** A routing protocol used within one autonomous system, such as OSPF, IS-IS, RIP or EIGRP. - -See also: [BGP](a-c.md#bgp), [OSPF](o-r.md#ospf), [RIP](o-r.md#rip). - -### IGRP -**Interior Gateway Routing Protocol.** Cisco's original distance-vector routing protocol, which used a composite metric of bandwidth and delay. EIGRP replaced it. - -See also: [EIGRP](d-f.md#eigrp), [RIP](o-r.md#rip). - -### IIS -**Internet Information Services.** Microsoft's web server for Windows Server. - -### IKE -**Internet Key Exchange.** The protocol that sets up security associations for IPsec. The peers authenticate each other and agree keys using Diffie-Hellman. IKEv2 is the current version. - -See also: [IPSec](#ipsec), [VPN](s-z.md#vpn). - -### iLBC -**Internet Low Bitrate Codec.** A royalty-free narrowband speech codec at about 13 to 15 kbit/s, designed to cope well with packet loss. It was part of WebRTC. - -See also: [OPUS](o-r.md#opus), [Audio Codecs](a-c.md#audio-codecs). - -### iManager -Huawei's network management software for its access equipment, such as MSANs and DSLAMs. - -### IMAP -**Internet Message Access Protocol.** The protocol email clients use to read and manage mail held on a server, so that several devices see the same mailbox. It uses port 143, or 993 over TLS. - -See also: [POP](o-r.md#pop). - -### Implementation -A synonym for construction, deployment or both, depending on context. - -See also: [Construction](a-c.md#construction), [Deployment](d-f.md#deployment). - -### Implementation Lead -A synonym for the construction lead, the deployment lead or both. - -### Implicit Change Control -Control of artefacts, such as source code, that the change control board does not manage directly, but which change only when the controlled artefacts they derive from change. - -See also: [Explicit Change Control](d-f.md#explicit-change-control). - -### Implicit Risk Management -See [Intrinsic Risk Management](#intrinsic-risk-management). - -### IMS -**IP Multimedia Subsystem.** The 3GPP architecture for delivering voice, video and messaging over IP using SIP. It is the core behind VoLTE, Wi-Fi calling and many fixed-line VoIP networks. - -See also: [CSCF](a-c.md#cscf), [HSS](#hss), [SIP](s-z.md#sip), [VoLTE](s-z.md#volte). - -### IMSI -**International Mobile Subscriber Identity.** The unique number, of up to 15 digits, that identifies a mobile subscriber, stored on the SIM. It is made up of the mobile country code, the network code and the subscriber number. Networks send it as rarely as possible, using temporary identities instead, to stop tracking. - -See also: [SIM](s-z.md#sim), [HLR](#hlr), [MNC](l-n.md#mnc). - -### IN -**Intelligent Network.** The ITU-T architecture (Q.1200 series) that moves service logic out of telephone switches into separate service control points. Freephone, number translation and prepaid services were built on it. - -See also: [INAP](#inap), [CAMEL](a-c.md#camel), [SS7](s-z.md#ss7). - -### INAP -**Intelligent Network Application Part.** The SS7 protocol, over TCAP, between switches and service control points in an Intelligent Network. - -See also: [IN](#in), [TCAP](s-z.md#tcap), [CAP](a-c.md#cap). - -### Inch-Pebble -See [Miniature Milestone](l-n.md#miniature-milestone). - -### Incremental Development -Building a product in increments, where each version is usable and adds user-visible functionality to the one before. - -See also: [Iterative Development](#iterative-development). - -### Informal Review -Any review other than an inspection, such as a walkthrough or desk check. - -See also: [Inspection](#inspection), [Walkthrough](s-z.md#walkthrough), [Desk Check](d-f.md#desk-check). - -### Informal Testing -Testing based on the tester's judgement, without documented test cases. - -See also: [Formal Testing](d-f.md#formal-testing), [Exploratory Testing](d-f.md#exploratory-testing). - -### Information architecture -How information in a system or site is organised, structured and labelled so that people can find and understand it. - -See also: [Information design](#information-design). - -### Information design -Planning the specific information to give a particular audience, and how to present it, to meet given goals. - -See also: [Information architecture](#information-architecture). - -### Information Radiators -Large, visible displays, such as task boards or burndown charts, placed where the whole team can see the current state of work at a glance. - -See also: [Task Board](s-z.md#task-board), [Kanban Board](#kanban-board), [Burndown Chart](a-c.md#burndown-chart). - -### Ingress -Traffic entering a network or a device, as opposed to egress traffic leaving it. - -See also: [Egress](d-f.md#egress). - -### Inspection -A formal review with defined roles (moderator, author, reader, inspectors and scribe), preparation, a logging meeting, rework and follow-up. - -See also: [Moderator](l-n.md#moderator), [Inspector](#inspector), [Reading Inspection](o-r.md#reading-inspection). - -### Inspector -A reviewer in an inspection. - -See also: [Inspection](#inspection). - -### Institute of Electrical and Electronics Engineers -See [IEEE](#ieee). - -### Integrated Development Environment -See [IDE](#ide). - -### Integration -Combining software components and making them work together. - -See also: [Integration Test](#integration-test), [Continuous Integration](a-c.md#continuous-integration). - -### Integration Test -A test that checks that components work together correctly after they are combined or changed. - -See also: [Unit Testing](s-z.md#unit-testing), [System Test](s-z.md#system-test). - -### Internet Protocol Suite -The protocols of the internet, often called TCP/IP after its two main protocols. It has four layers: link, internet (IP), transport (TCP, UDP) and application (such as HTTP or DNS). - -See also: [TCP](s-z.md#tcp), [UDP](s-z.md#udp), [IPv4](#ipv4), [IPv6](#ipv6). - -### Interpreted Language -A language whose usual implementations run the source, or bytecode compiled from it, through an interpreter rather than compiling it ahead of time to machine code. Python and JavaScript are examples. The distinction is about implementations rather than languages. - -See also: [Compiled Language](a-c.md#compiled-language). - -### Intrinsic Risk Management -Managing risk through how a project is run, for example by choosing a lifecycle and practices that expose and reduce risks as a matter of course. - -See also: [Extrinsic Risk Management](d-f.md#extrinsic-risk-management). - -### INVEST -A checklist for good user stories: Independent, Negotiable, Valuable, Estimable, Small and Testable. - -See also: [User Stories](s-z.md#user-stories), [Definition of Ready](d-f.md#definition-of-ready). - -### IOS -**Internetwork Operating System.** The operating system on most Cisco routers and switches, known for its command-line interface. - -### IPS -**Intrusion Prevention System.** A device or software that watches network or host activity for attacks and policy breaches and blocks them. An intrusion detection system (IDS) only reports them. - -See also: [Firewall](d-f.md#firewall), [DPI](d-f.md#dpi). - -### IPSC -**IP Stream Connect.** A BT Wholesale product that handed over broadband customers' traffic to an internet service provider at a central point. - -See also: [ISP](#isp), [L2TP](l-n.md#l2tp). - -### IPSec -**Internet Protocol Security.** A suite of protocols that authenticate and encrypt IP packets, between hosts, between gateways or between a host and a gateway. It is widely used for VPNs; IKE sets up its keys. - -See also: [IKE](#ike), [VPN](s-z.md#vpn), [ePDG](d-f.md#epdg). - -### IPTV -**Internet Protocol Television.** Television delivered over a managed IP network, usually by multicast to a set-top box, rather than by broadcast or the open internet. - -See also: [IGMP](#igmp), [STB](s-z.md#stb), [VOD](s-z.md#vod). - -### IPv4 -**Internet Protocol version 4.** The original version of IP, from 1981 (RFC 791), with 32-bit addresses. It still carries much of the internet's traffic, stretched by NAT because its address space is too small. - -See also: [IPv6](#ipv6), [NAT](l-n.md#nat). - -### IPv6 -**Internet Protocol version 6.** The successor to IPv4, with 128-bit addresses, a simpler header and built-in autoconfiguration. It became an internet standard in 2017 (RFC 8200). - -See also: [IPv4](#ipv4), [ICMPv6](#icmpv6), [DNS - AAAA](d-f.md#dns-aaaa). - -### ISDN -**Integrated Services Digital Network.** A set of standards for digital voice and data over the telephone network, using 64 kbit/s channels with separate signalling. It was sold as basic rate (BRI) and primary rate (PRI) access. - -See also: [BRI](a-c.md#bri), [PRI](o-r.md#pri), [Q.931](o-r.md#q931), [ISDN2](#isdn2), [ISDN30](#isdn30). - -### ISDN2 -BT's name for ISDN basic rate access: two 64 kbit/s channels. See [BRI](a-c.md#bri). - -### ISDN30 -BT's name for ISDN primary rate access: up to 30 channels on a 2 Mbit/s line. See [PRI](o-r.md#pri). - -### ISO -<!-- vale PyDoc.Spelling = NO --> -**International Organization for Standardization.** -<!-- vale PyDoc.Spelling = YES --> -The international body made up of national standards bodies, such as BSI and ANSI, that publishes standards in almost every field. - -See also: [IEC](#iec), [ANSI](a-c.md#ansi). - -### ISP -**Internet Service Provider.** A company that provides internet access, and often related services such as email, hosting and domain registration. - -### Issue -Something that needs resolving and could affect the project if left. In this sense it is not already covered by the plan, a change request, a defect or a risk. - -See also: [Issue Management](#issue-management), [Issue tracker](#issue-tracker). - -### Issue Management -Recording, tracking and resolving a project's issues. - -See also: [Issue](#issue). - -### Issue tracker -Software that records issues, such as bugs, tasks or support requests, and tracks them from report to resolution. - -See also: [Defect Management](d-f.md#defect-management). - -### ISUP -**ISDN User Part.** The SS7 protocol that sets up and clears telephone calls between exchanges, carrying details such as the called number. It is defined in the ITU-T Q.76x series. - -See also: [SS7](s-z.md#ss7), [IUP](#iup), [TUP](s-z.md#tup). - -### ITIL -A widely used framework of good practice for IT service management, covering how services are designed, delivered, run and improved. The name came from the Information Technology Infrastructure Library. - -See also: [CMDB](a-c.md#cmdb), [CCTA](a-c.md#ccta). - -### ITU -**International Telecommunication Union.** The United Nations agency for information and communication technologies. It coordinates the radio spectrum and satellite orbits, and sets telecoms standards through ITU-T. - -See also: [ITU-T](#itu-t), [ETSI](d-f.md#etsi). - -### ITU-T -The ITU's standardisation sector, which publishes telecoms standards as Recommendations, such as G.711 and H.323. It was called the CCITT until 1993. - -See also: [ITU](#itu), [CCITT](a-c.md#ccitt). - -### IUP -**Interconnect User Part.** The UK national version of SS7 call signalling between operators' networks, formerly called BTNUP. - -See also: [ISUP](#isup), [BTNUP](a-c.md#btnup). - -### IVR -**Interactive Voice Response.** A telephone system that talks to callers with recorded or generated speech and responds to their key presses or spoken answers, typically to route calls or serve simple requests. - -See also: [DTMF](d-f.md#dtmf). - -### Item -In a backlog or tracker, one unit of work being managed, such as a change request, defect, risk or issue. - -See also: [Work Item](s-z.md#work-item). - -### Iteration -A fixed period, or timebox, in which a team develops and delivers a working increment. - -See also: [Sprints](s-z.md#sprints), [Timebox](s-z.md#timebox). - -### Iterative Development -Developing by repeating cycles of work and deliberately revisiting earlier work in the light of feedback, rather than doing each step once. - -See also: [Incremental Development](#incremental-development), [Refactoring](o-r.md#refactoring). - -## J - -### Java -A class-based, object-oriented language released by Sun Microsystems in 1995. It compiles to bytecode that runs on the Java virtual machine, so the same code runs on any platform that has one. - -See also: [JavaScript](#javascript), [C#](a-c.md#c_2). - -### JavaScript -The scripting language of the web, run by every browser and, through Node.js, on servers. It is dynamic and supports object-oriented, functional and event-driven styles. It is standardised as ECMAScript. - -See also: [JSON](#json), [Java](#java). - -### Jitter -Variation in the timing of a signal or in the delay of packets. In packet networks it is also called packet delay variation, and it affects real-time traffic such as voice and video. - -See also: [Jitter buffer](#jitter-buffer), [Latency](l-n.md#latency). - -### Jitter buffer -A buffer at a receiver that holds arriving voice or video packets for a short time, so they can be played out at an even rate despite jitter. Adaptive buffers change their size to match conditions, trading delay against loss. - -See also: [Jitter](#jitter), [RTP](o-r.md#rtp). - -### Joint Application Development -**JAD.** A facilitated workshop in which customer representatives and developers draft requirements together. - -See also: [RAD](o-r.md#rad). - -### JSON -**JavaScript Object Notation.** A lightweight text format for data made of objects (name and value pairs) and arrays, defined in RFC 8259. It is the usual format for web APIs. - -See also: [JSON-RPC](#json-rpc), [YAML](s-z.md#yaml), [XML](s-z.md#xml), [CBOR](a-c.md#cbor). - -### JSON-RPC -A simple remote procedure call protocol that encodes calls and results in JSON. - -See also: [XML-RPC](s-z.md#xml-rpc), [RPC](o-r.md#rpc). - -## K - -### K & R {#k-r} -**Kernighan and Ritchie.** Brian Kernighan and Dennis Ritchie of Bell Labs, authors of *The C Programming Language* (1978). Ritchie created C and, with Ken Thompson, Unix. - -See also: [K & R C](#k-r-c), [C Language](a-c.md#c-language). - -### K & R C {#k-r-c} -The version of C described in the first edition of Kernighan and Ritchie's book, which served as its informal specification before the ANSI standard of 1989. - -See also: [K & R](#k-r), [C Language](a-c.md#c-language). - -### K & R indent style {#k-r-indent-style} -The brace style used in Kernighan and Ritchie's book. A function's opening brace goes on its own line, while blocks inside the function open on the same line as their control statement. - -See also: [Construction Standard](a-c.md#construction-standard). - -### Kanban -A method for managing knowledge work by making the workflow visible, limiting work in progress and improving the flow step by step. - -See also: [Kanban Board](#kanban-board), [Lean](l-n.md#lean). - -### Kanban Board -A board with a column for each stage of a workflow, on which cards representing work items move from left to right. - -See also: [Kanban](#kanban), [Task Board](s-z.md#task-board). - -### KDL -A document language with XML-like semantics and a syntax that looks like a series of commands. It is used for configuration and data, like JSON, YAML or XML. - -See also: [YAML](s-z.md#yaml), [TOML](s-z.md#toml). - -### Kerberos -A network authentication protocol, developed at MIT, in which a trusted key distribution centre issues time-limited tickets. Clients and services then prove their identities to each other without sending passwords. Windows Active Directory uses it. - -See also: [AD](a-c.md#ad), [LDAP](l-n.md#ldap). - -### Kernel -The core of an operating system, which runs with full control of the hardware. It manages processes, memory, devices and system calls, and keeps applications apart from each other. - -See also: [Linux](l-n.md#linux), [Driver](d-f.md#driver). - -### KISS Principle -**Keep it simple, stupid.** The principle that most systems work best when kept simple, and that unnecessary complexity should be avoided. - -See also: [Rule of Simplicity](o-r.md#rule-of-simplicity), [Simple Design](s-z.md#simple-design). - -### Knowledge Management -**KM.** Capturing, sharing and reusing an organisation's knowledge so that people can find what they need. - -See also: [Wiki](s-z.md#wiki). - -### KPI -**Key Performance Indicator.** A measure of how well an organisation or activity is meeting an important goal. - -See also: [Metric](l-n.md#metric), [SLA](s-z.md#sla). - -### Ksh -The KornShell, a Unix shell written by David Korn at Bell Labs in the early 1980s. It is compatible with the Bourne shell and adds features from the C shell. - -See also: [Sh](s-z.md#sh), [Bash](a-c.md#bash), [Csh](a-c.md#csh). diff --git a/content/glossary/g.md b/content/glossary/g.md new file mode 100644 index 0000000..f4bcc6d --- /dev/null +++ b/content/glossary/g.md @@ -0,0 +1,201 @@ +--- +title: G +--- + +# G + +## G.711 {#g711} +The ITU-T standard, from 1972, for coding telephone speech as 64 kbit/s PCM: 8,000 samples a second, each companded to 8 bits. It has two variants, A-law and µ-law. It is the baseline codec that every VoIP system supports. + +![G.711](images/15008219.png) + +[[CAPTION:Figure]] G.711. + +See also: [A-law](a.md#a-law), [Mu-law](m.md#mu-law), [PCM](p.md#pcm), [G.711.1](#g7111). + +## G.711.1 {#g7111} +A wideband extension of G.711, from 2008, that adds layers on top of a G.711 core. It runs at 64, 80 or 96 kbit/s, and a G.711-only receiver can still decode the core. + +See also: [G.711](#g711), [G.722](#g722). + +## G.719 {#g719} +An ITU-T codec for full-band audio (20 Hz to 20 kHz) at 32 to 128 kbit/s, developed by Polycom and Ericsson for high-quality conferencing. It handles music as well as speech. + +See also: [G.722](#g722), [Audio Codecs](a.md#audio-codecs). + +## G.722 {#g722} +An ITU-T wideband speech codec from 1988, covering 50 to 7,000 Hz at 48 to 64 kbit/s using sub-band ADPCM. It is widely used for HD voice on desk phones. G.722.1 and G.722.2 (AMR-WB) are different codecs despite the names. + +![G.722](images/15008219.png) + +[[CAPTION:Figure]] G.722. + +See also: [G.711](#g711), [AMR-WB](a.md#amr-wb). + +## G.723.1 {#g7231} +An ITU-T low-rate speech codec, at 5.3 or 6.3 kbit/s, with 30 ms frames. It was common in early VoIP and H.323 systems. It is unrelated to the older G.723. + +See also: [G.726](#g726), [H.323](h.md#h323). + +## G.726 {#g726} +An ITU-T ADPCM speech codec at 16, 24, 32 or 40 kbit/s, which replaced G.721 and G.723. The 32 kbit/s rate, half that of G.711, is the most used, for example in DECT cordless phones. + +See also: [G.711](#g711), [PCM](p.md#pcm). + +## G.Fast {#gfast} +An ITU-T DSL standard (G.9700 and G.9701) for very short copper lines, up to a few hundred metres, giving several hundred megabits per second. It is used from a distribution point or cabinet fed by fibre. + +![G.Fast](images/15008227.png) + +[[CAPTION:Figure]] G.Fast. + +![G.Fast](images/15008233.png) + +[[CAPTION:Figure]] G.Fast. + +![G.Fast](images/15008239.png) + +[[CAPTION:Figure]] G.Fast. + +See also: [VDSL](v.md#vdsl), [DSL](d.md#dsl), [DP](d.md#dp). + +## Gantt chart +A bar chart of a project schedule, with tasks down the side and time along the top. It shows when each task starts and ends, and how the tasks depend on each other. + +![Gantt chart](images/15008245.png) + +[[CAPTION:Figure]] Gantt chart. + +See also: [PERT chart](p.md#pert-chart), [CPM](c.md#cpm). + +## Gap analysis +Comparing where things are now with where they need to be, to find what must change to get there. + +## Gate +A set of criteria that must be met before work can pass from one phase or stage to the next. + +See also: [Gate Checklist](#gate-checklist), [Phase](p.md#phase). + +## Gate Checklist +A checklist used to check that the criteria for a gate have been met. + +See also: [Gate](#gate). + +## GBE / GE {#gbe-ge} +Abbreviations for [Gigabit Ethernet](#gigabit-ethernet). + +## GCHQ +**Government Communications Headquarters.** The UK's signals intelligence and security agency, based in Cheltenham. It grew out of the Government Code and Cypher School of Bletchley Park, and runs the National Cyber Security Centre. + +See also: [NCSC](n.md#ncsc), [CESG](c.md#cesg). + +## Geocast +Delivering a message to all the nodes in a geographical area, rather than to a list of addresses. It is a form of multicast used in some mobile ad hoc and vehicle networks. + +![Geocast](images/15008251.png) + +[[CAPTION:Figure]] Geocast. + +See also: [Multicast](m.md#multicast), [MANET](m.md#manet). + +## GERAN +**GSM EDGE Radio Access Network.** The radio access network of GSM and EDGE: the base stations, their controllers and the interfaces between them. 3GPP maintains its standards. + +![GERAN](images/15008257.png) + +[[CAPTION:Figure]] GERAN. + +See also: [BTS](b.md#bts), [BSC](b.md#bsc), [EDGE](e.md#edge), [UTRAN](u.md#utran). + +## GGSN +**Gateway GPRS Support Node.** The GPRS and UMTS core node that connects the mobile packet network to external networks such as the internet. It allocates the device's IP address and anchors the session as the user moves. In LTE the PGW does this job. + +See also: [SGSN](s.md#sgsn), [GPRS](#gprs), [APN](a.md#apn), [PGW](p.md#pgw). + +## Gherkin +The plain-language format used by Cucumber and similar tools to write behaviour-driven tests. Scenarios are written as `Given`, `When` and `Then` steps under a `Feature`, so that business people and developers can read the same tests. + +See also: [BDD](b.md#bdd), [Given When Then](#given-when-then). + +## Gigabit Ethernet +Ethernet at 1 Gbit/s, defined in IEEE 802.3. It replaced Fast Ethernet as the usual speed for wired networks from the early 2000s. + +See also: [GBE / GE](#gbe-ge), [100 Mbps (Fast Ethernet)](numbers.md#100-mbps-fast-ethernet), [Ethernet over twisted pair](e.md#ethernet-over-twisted-pair). + +## Git +A distributed version control system, created by Linus Torvalds in 2005 for the Linux kernel. Every clone holds the full history, so most work is local and fast, and branching and merging are cheap. + +See also: [Version Control](v.md#version-control), [Subversion](s.md#subversion). + +## Given When Then +A template for writing acceptance criteria and tests: given some context, when an action happens, then expect these results. + +See also: [Gherkin](#gherkin), [BDD](b.md#bdd). + +## GMSC +**Gateway Mobile Switching Centre.** An MSC function that routes calls between the mobile network and other networks. For an incoming call it asks the HLR where the subscriber is. + +See also: [MSC](m.md#msc), [HLR](h.md#hlr). + +## GNU +**GNU's Not Unix.** The free software operating system started by Richard Stallman in 1983, made up of tools, libraries and compilers. Combined with the Linux kernel, it forms most Linux distributions. + +See also: [GPL](#gpl), [Linux](l.md#linux), [Copyleft](c.md#copyleft). + +## Go +A statically typed, compiled language created at Google in 2009 by Robert Griesemer, Rob Pike and Ken Thompson. It has garbage collection, built-in concurrency with goroutines and channels, and fast builds. It is also called Golang. + +## Goals +The results a project, process or activity is meant to achieve. Also called objectives. + +See also: [Objectives](o.md#objectives). + +## GoF +**Gang of Four.** The four authors (Gamma, Helm, Johnson and Vlissides) of *Design Patterns: Elements of Reusable Object-Oriented Software* (1994). The book catalogues 23 object-oriented design patterns. + +See also: [Design Pattern](d.md#design-pattern). + +## Google Cloud Platform +Google's cloud computing services, offering infrastructure, platform and serverless computing, data analytics and machine learning on the infrastructure Google uses itself. + +See also: [AWS](a.md#aws), [Azure](a.md#azure). + +## GPL +<!-- vale PyDoc.Spelling = NO --> +**GNU General Public License.** +<!-- vale PyDoc.Spelling = YES --> +The most widely used copyleft free software licence, written by Richard Stallman. It lets anyone run, study, share and change the software, as long as they share copies and derived works under the same licence. + +See also: [Copyleft](c.md#copyleft), [GNU](#gnu). + +## GPRS +**General Packet Radio Service.** The packet data service added to GSM, giving always-on connections charged by data volume rather than by time. It is best-effort, with speeds of tens of kilobits per second. + +![GPRS](images/15008263.png) + +[[CAPTION:Figure]] GPRS. + +See also: [GSM](#gsm), [EDGE](e.md#edge), [SGSN](s.md#sgsn), [GGSN](#ggsn). + +## GSM +**Global System for Mobile communications.** The second-generation (2G) digital mobile standard, developed by ETSI and first launched in 1991. It was designed for circuit-switched voice and SMS, and GPRS and EDGE later added data. + +See also: [GPRS](#gprs), [EDGE](e.md#edge), [UMTS](u.md#umts), [SIM](s.md#sim). + +## GSM AMR +The AMR speech codec as used in GSM. See [AMR-NB](a.md#amr-nb). + +## GSM-FR +**GSM Full Rate.** The first GSM speech codec (GSM 06.10), at 13 kbit/s. Its quality is poor by modern standards; Enhanced Full Rate and AMR replaced it. + +See also: [GSM AMR](#gsm-amr), [AMR-NB](a.md#amr-nb). + +## GSM-R +**GSM for Railways.** A version of GSM for railway voice and data, including the signalling of the European Rail Traffic Management System. It is designed to work at train speeds up to 500 km/h. + +See also: [GSM](#gsm). + +## GSN +**GPRS Support Node.** A node in the GPRS core network: either a serving node (SGSN) or a gateway node (GGSN). + +See also: [SGSN](s.md#sgsn), [GGSN](#ggsn). diff --git a/content/glossary/h.md b/content/glossary/h.md new file mode 100644 index 0000000..e076d00 --- /dev/null +++ b/content/glossary/h.md @@ -0,0 +1,143 @@ +--- +title: H +--- + +# H + +## H.248 {#h248} +**Gateway Control Protocol (Megaco).** The ITU-T and IETF protocol with which a media gateway controller or softswitch controls media gateways, which convert between telephone circuits and IP media streams. It does the same job as MGCP. + +![H.248](images/15008304.png) + +[[CAPTION:Figure]] H.248. + +See also: [MGCP](m.md#mgcp), [MGW](m.md#mgw), [Softswitch](s.md#softswitch). + +## H.323 {#h323} +An ITU-T family of protocols for voice and video calls over packet networks, covering signalling, media and bandwidth control. It was the main VoIP standard before SIP replaced it. + +![H.323](images/15008312.png) + +[[CAPTION:Figure]] H.323. + +See also: [SIP](s.md#sip), [Q.931](q.md#q931), [RAS](r.md#ras). + +## Half Duplex +Communication in which both ends can send, but only one at a time, like a push-to-talk radio. + +See also: [Duplex](d.md#duplex), [Full duplex](f.md#full-duplex), [Simplex](s.md#simplex). + +## Happy Path +See [Nominal Path](n.md#nominal-path). + +## Hash Function +A function that maps data of any size to a fixed-size value. A cryptographic hash, such as SHA-256, is also one-way and collision-resistant. That makes it useful for integrity checks, digital signatures and storing passwords. + +![Hash Function](images/15008318.png) + +[[CAPTION:Figure]] Hash Function. + +See also: [SHA](s.md#sha), [MD5](m.md#md5), [CRC](c.md#crc). + +## HDF +**Handover Distribution Frame.** In UK local loop unbundling, the frame in the exchange where the copper pairs are handed over from the main distribution frame to another operator's equipment. + +![HDF](images/15008324.png) + +[[CAPTION:Figure]] HDF. + +See also: [MDF](m.md#mdf), [Local Loop](l.md#local-loop). + +## HDLC +**High-Level Data Link Control.** An ISO bit-oriented, synchronous data link protocol. Many other link protocols, including PPP framing, LAPB in X.25 and LAPD in ISDN, are based on it. + +![HDLC](images/15008330.png) + +[[CAPTION:Figure]] HDLC. + +See also: [PPP](p.md#ppp), [X.25](x.md#x25), [Frame Relay](f.md#frame-relay). + +## HDSL +**High-bit-rate Digital Subscriber Line.** A symmetrical DSL that carries a 2 Mbit/s (E1) or 1.5 Mbit/s (T1) circuit over two or three copper pairs. It was used to deliver leased lines without new cable. + +See also: [SDSL](s.md#sdsl), [DSL](d.md#dsl). + +## Heartbeat Retrospective +A retrospective held at a regular interval, such as the end of each iteration, in which the team reflects on recent events and agrees improvements. + +See also: [Sprint retrospective](s.md#sprint-retrospective). + +## HiFi Prototype +See [High Fidelity Prototype](#high-fidelity-prototype). + +## High Fidelity Prototype +A prototype that looks and behaves much like the finished system, usually built with real development tools. + +See also: [Low Fidelity Prototype](l.md#low-fidelity-prototype), [User Interface Prototype](u.md#user-interface-prototype). + +## High Level Design +**HLD.** The design level between the architecture and the low-level design. It defines the components of the solution, their interfaces and how they interact, without going into how each is built. + +See also: [Low Level Design](l.md#low-level-design), [Architecture](a.md#architecture). + +## HLD +See [High Level Design](#high-level-design). + +## HLR +**Home Location Register.** The central database of a GSM or UMTS network, holding each subscriber's details: their IMSI, phone numbers, services and current location. In LTE and IMS, the HSS replaced it. + +See also: [VLR](v.md#vlr), [HSS](#hss), [IMSI](i.md#imsi), [MSC](m.md#msc). + +## Hotfix +A small, urgent change, sometimes called a patch, that fixes a bug in released software outside the normal release cycle. + +See also: [Release](r.md#release). + +## How Requirement +Usually a synonym for [Non-functional Requirement](n.md#non-functional-requirement); sometimes used for design constraints. + +## How Well Requirement +See [Non-functional Requirement](n.md#non-functional-requirement). + +## HSRP +**Hot Standby Router Protocol.** Cisco's protocol for a fault-tolerant default gateway. Two or more routers share a virtual IP address; one is active and the others take over if it fails. VRRP is the open-standard equivalent. + +![HSRP](images/15008336.png) + +[[CAPTION:Figure]] HSRP. + +See also: [Router](r.md#router). + +## HSS +**Home Subscriber Server.** The master subscriber database for IMS and LTE, holding user profiles and authentication data and tracking where users are registered. It takes over the jobs of the HLR and the authentication centre. + +See also: [HLR](#hlr), [IMS](i.md#ims), [CSCF](c.md#cscf), [MME](m.md#mme). + +## HTTP +**Hypertext Transfer Protocol.** The request-response protocol of the web. A client asks for a resource by URL with a method such as `GET` or `POST`, and the server returns a status and content. HTTP/2 and HTTP/3 keep the same model with more efficient transports. + +See also: [HTTPS](#https), [REST](r.md#rest), [URL](u.md#url). + +## HTTPS +HTTP over TLS. It encrypts the traffic and lets the client check the server's identity through its certificate, protecting against eavesdropping and tampering. + +See also: [HTTP](#http), [TLS](t.md#tls), [CA](c.md#ca). + +## Hub +A network device that repeats every frame it receives to all its other ports, so all connected devices share one collision domain. It works at layer 1, and switches have replaced it. + +![Hub](images/15008342.png) + +[[CAPTION:Figure]] Hub. + +See also: [Switch](s.md#switch), [L1](l.md#l1). + +## Human factors +The study of how people behave, physically and mentally, with products, systems and environments, and how to design for them. + +See also: [User Experience](u.md#user-experience). + +## Hungarian notation +A naming convention in which a prefix on an identifier shows its kind or type, such as `cApples` for a count or `szName` for a zero-terminated string. Most modern style guides advise against encoding the type in this way. + +See also: [Camel Case](c.md#camel-case), [Snake Case](s.md#snake-case). diff --git a/content/glossary/i.md b/content/glossary/i.md new file mode 100644 index 0000000..d949d3c --- /dev/null +++ b/content/glossary/i.md @@ -0,0 +1,392 @@ +--- +title: I +--- + +# I + +## IA +**Information Assurance.** Managing the risks to information and the systems that handle it, so that it keeps its confidentiality, integrity, availability, authenticity and non-repudiation. + +See also: [CESG](c.md#cesg), [NCSC](n.md#ncsc). + +## IaaS +**Infrastructure as a Service.** Cloud computing that rents virtual machines, storage and networks, which the customer manages from the operating system up. + +![IaaS](images/15008352.png) + +[[CAPTION:Figure]] IaaS. + +See also: [PaaS](p.md#paas), [SaaS](s.md#saas), [NaaS](n.md#naas). + +## ICE +**Interactive Connectivity Establishment.** A technique, defined in RFC 8445, for finding the most direct working path between two peers behind NATs and firewalls. It tries candidate addresses gathered with STUN and TURN. SIP and WebRTC media use it. + +See also: [STUN](s.md#stun), [NAT](n.md#nat). + +## ICMP +**Internet Control Message Protocol.** The IP protocol that carries error and diagnostic messages, such as "destination unreachable" and the echo request and reply used by `ping`. + +See also: [ICMPv6](#icmpv6), [Internet Protocol Suite](#internet-protocol-suite). + +## ICMPv6 +ICMP for IPv6, defined in RFC 4443. As well as errors and `ping`, it carries Neighbour Discovery, which replaces ARP, and Multicast Listener Discovery, which replaces IGMP. + +See also: [ICMP](#icmp), [IPv6](#ipv6). + +## IDE +**Integrated Development Environment.** An application that combines a code editor, build tools and a debugger, and often much more, for writing software. + +See also: [Integrated Development Environment](#integrated-development-environment). + +## IDQ +**Informatica Data Quality.** Informatica's tool for profiling, cleansing and standardising enterprise data. + +See also: [DQ](d.md#dq). + +## IEC +**International Electrotechnical Commission.** The international body that publishes standards for electrical and electronic technologies. It works with ISO on information technology standards, such as the ISO/IEC series. + +See also: [ISO](#iso). + +## IEEE +**Institute of Electrical and Electronics Engineers.** The world's largest professional association of engineers. It publishes many standards, including the 802 family for LANs: Ethernet (802.3) and Wi-Fi (802.11). + +See also: [802.11](numbers.md#80211), [802.3 Physical Layer](numbers.md#8023-physical-layer). + +## IEEE 802.1ah {#ieee-8021ah} +**Provider Backbone Bridges (PBB), or MAC-in-MAC.** An IEEE standard that wraps customer Ethernet frames in a provider's own Ethernet header. Providers can then scale carrier Ethernet without learning customer MAC addresses or running out of VLAN IDs. + +See also: [IEEE 802.1Qay](#ieee-8021qay), [VLAN](v.md#vlan). + +## IEEE 802.1Qay {#ieee-8021qay} +**Provider Backbone Bridge Traffic Engineering (PBB-TE).** A carrier Ethernet standard that builds on 802.1ah MAC-in-MAC. It removes flooding and spanning tree and has the operator set up paths explicitly, giving predictable, SDH-like behaviour. + +See also: [IEEE 802.1ah](#ieee-8021ah), [SDH](s.md#sdh). + +## IETF +**Internet Engineering Task Force.** The open standards body that develops internet protocols, such as TCP/IP, HTTP and TLS, and publishes them as RFCs. + +See also: [RFC](r.md#rfc). + +## IGMP +**Internet Group Management Protocol.** The IPv4 protocol with which hosts tell their local routers which multicast groups they want to receive. IPTV uses it to join channels. + +![IGMP](images/15008360.png) + +[[CAPTION:Figure]] IGMP. + +See also: [Multicast](m.md#multicast), [IPTV](#iptv). + +## IGP +**Interior Gateway Protocol.** A routing protocol used within one autonomous system, such as OSPF, IS-IS, RIP or EIGRP. + +See also: [BGP](b.md#bgp), [OSPF](o.md#ospf), [RIP](r.md#rip). + +## IGRP +**Interior Gateway Routing Protocol.** Cisco's original distance-vector routing protocol, which used a composite metric of bandwidth and delay. EIGRP replaced it. + +See also: [EIGRP](e.md#eigrp), [RIP](r.md#rip). + +## IIS +**Internet Information Services.** Microsoft's web server for Windows Server. + +## IKE +**Internet Key Exchange.** The protocol that sets up security associations for IPsec. The peers authenticate each other and agree keys using Diffie-Hellman. IKEv2 is the current version. + +![IKE](images/15008366.png) + +[[CAPTION:Figure]] IKE. + +See also: [IPSec](#ipsec), [VPN](v.md#vpn). + +## iLBC +**Internet Low Bitrate Codec.** A royalty-free narrowband speech codec at about 13 to 15 kbit/s, designed to cope well with packet loss. It was part of WebRTC. + +See also: [OPUS](o.md#opus), [Audio Codecs](a.md#audio-codecs). + +## iManager +Huawei's network management software for its access equipment, such as MSANs and DSLAMs. + +## IMAP +**Internet Message Access Protocol.** The protocol email clients use to read and manage mail held on a server, so that several devices see the same mailbox. It uses port 143, or 993 over TLS. + +See also: [POP](p.md#pop). + +## Implementation +A synonym for construction, deployment or both, depending on context. + +See also: [Construction](c.md#construction), [Deployment](d.md#deployment). + +## Implementation Lead +A synonym for the construction lead, the deployment lead or both. + +## Implicit Change Control +Control of artefacts, such as source code, that the change control board does not manage directly, but which change only when the controlled artefacts they derive from change. + +See also: [Explicit Change Control](e.md#explicit-change-control). + +## Implicit Risk Management +See [Intrinsic Risk Management](#intrinsic-risk-management). + +## IMS +**IP Multimedia Subsystem.** The 3GPP architecture for delivering voice, video and messaging over IP using SIP. It is the core behind VoLTE, Wi-Fi calling and many fixed-line VoIP networks. + +![IMS](images/15204369.png) + +[[CAPTION:Figure]] IMS. + +See also: [CSCF](c.md#cscf), [HSS](h.md#hss), [SIP](s.md#sip), [VoLTE](v.md#volte). + +## IMSI +**International Mobile Subscriber Identity.** The unique number, of up to 15 digits, that identifies a mobile subscriber, stored on the SIM. It is made up of the mobile country code, the network code and the subscriber number. Networks send it as rarely as possible, using temporary identities instead, to stop tracking. + +See also: [SIM](s.md#sim), [HLR](h.md#hlr), [MNC](m.md#mnc). + +## IN +**Intelligent Network.** The ITU-T architecture (Q.1200 series) that moves service logic out of telephone switches into separate service control points. Freephone, number translation and prepaid services were built on it. + +See also: [INAP](#inap), [CAMEL](c.md#camel), [SS7](s.md#ss7). + +## INAP +**Intelligent Network Application Part.** The SS7 protocol, over TCAP, between switches and service control points in an Intelligent Network. + +See also: [IN](#in), [TCAP](t.md#tcap), [CAP](c.md#cap). + +## Inch-Pebble +See [Miniature Milestone](m.md#miniature-milestone). + +## Incremental Development +Building a product in increments, where each version is usable and adds user-visible functionality to the one before. + +See also: [Iterative Development](#iterative-development). + +## Informal Review +Any review other than an inspection, such as a walkthrough or desk check. + +See also: [Inspection](#inspection), [Walkthrough](w.md#walkthrough), [Desk Check](d.md#desk-check). + +## Informal Testing +Testing based on the tester's judgement, without documented test cases. + +See also: [Formal Testing](f.md#formal-testing), [Exploratory Testing](e.md#exploratory-testing). + +## Information architecture +How information in a system or site is organised, structured and labelled so that people can find and understand it. + +See also: [Information design](#information-design). + +## Information design +Planning the specific information to give a particular audience, and how to present it, to meet given goals. + +See also: [Information architecture](#information-architecture). + +## Information Radiators +Large, visible displays, such as task boards or burndown charts, placed where the whole team can see the current state of work at a glance. + +![Information Radiators](images/15008372.png) + +[[CAPTION:Figure]] Information Radiators. + +See also: [Task Board](t.md#task-board), [Kanban Board](k.md#kanban-board), [Burndown Chart](b.md#burndown-chart). + +## Ingress +Traffic entering a network or a device, as opposed to egress traffic leaving it. + +![Ingress](images/15302712.png) + +[[CAPTION:Figure]] Ingress. + +See also: [Egress](e.md#egress). + +## Inspection +A formal review with defined roles (moderator, author, reader, inspectors and scribe), preparation, a logging meeting, rework and follow-up. + +See also: [Moderator](m.md#moderator), [Inspector](#inspector), [Reading Inspection](r.md#reading-inspection). + +## Inspector +A reviewer in an inspection. + +See also: [Inspection](#inspection). + +## Institute of Electrical and Electronics Engineers +See [IEEE](#ieee). + +## Integrated Development Environment +See [IDE](#ide). + +## Integration +Combining software components and making them work together. + +See also: [Integration Test](#integration-test), [Continuous Integration](c.md#continuous-integration). + +## Integration Test +A test that checks that components work together correctly after they are combined or changed. + +See also: [Unit Testing](u.md#unit-testing), [System Test](s.md#system-test). + +## Internet Protocol Suite +The protocols of the internet, often called TCP/IP after its two main protocols. It has four layers: link, internet (IP), transport (TCP, UDP) and application (such as HTTP or DNS). + +![Internet Protocol Suite](images/15008378.png) + +[[CAPTION:Figure]] Internet Protocol Suite. + +See also: [TCP](t.md#tcp), [UDP](u.md#udp), [IPv4](#ipv4), [IPv6](#ipv6). + +## Interpreted Language +A language whose usual implementations run the source, or bytecode compiled from it, through an interpreter rather than compiling it ahead of time to machine code. Python and JavaScript are examples. The distinction is about implementations rather than languages. + +See also: [Compiled Language](c.md#compiled-language). + +## Intrinsic Risk Management +Managing risk through how a project is run, for example by choosing a lifecycle and practices that expose and reduce risks as a matter of course. + +See also: [Extrinsic Risk Management](e.md#extrinsic-risk-management). + +## INVEST +A checklist for good user stories: Independent, Negotiable, Valuable, Estimable, Small and Testable. + +See also: [User Stories](u.md#user-stories), [Definition of Ready](d.md#definition-of-ready). + +## IOS +**Internetwork Operating System.** The operating system on most Cisco routers and switches, known for its command-line interface. + +## IPS +**Intrusion Prevention System.** A device or software that watches network or host activity for attacks and policy breaches and blocks them. An intrusion detection system (IDS) only reports them. + +See also: [Firewall](f.md#firewall), [DPI](d.md#dpi). + +## IPSC +**IP Stream Connect.** A BT Wholesale product that handed over broadband customers' traffic to an internet service provider at a central point. + +![IPSC](images/15008384.png) + +[[CAPTION:Figure]] IPSC. + +See also: [ISP](#isp), [L2TP](l.md#l2tp). + +## IPSec +**Internet Protocol Security.** A suite of protocols that authenticate and encrypt IP packets, between hosts, between gateways or between a host and a gateway. It is widely used for VPNs; IKE sets up its keys. + +See also: [IKE](#ike), [VPN](v.md#vpn), [ePDG](e.md#epdg). + +## IPTV +**Internet Protocol Television.** Television delivered over a managed IP network, usually by multicast to a set-top box, rather than by broadcast or the open internet. + +See also: [IGMP](#igmp), [STB](s.md#stb), [VOD](v.md#vod). + +## IPv4 +**Internet Protocol version 4.** The original version of IP, from 1981 (RFC 791), with 32-bit addresses. It still carries much of the internet's traffic, stretched by NAT because its address space is too small. + +![IPv4](images/15008390.png) + +[[CAPTION:Figure]] IPv4. + +See also: [IPv6](#ipv6), [NAT](n.md#nat). + +## IPv6 +**Internet Protocol version 6.** The successor to IPv4, with 128-bit addresses, a simpler header and built-in autoconfiguration. It became an internet standard in 2017 (RFC 8200). + +![IPv6](images/15008396.png) + +[[CAPTION:Figure]] IPv6. + +![IPv6](images/15008402.png) + +[[CAPTION:Figure]] IPv6. + +See also: [IPv4](#ipv4), [ICMPv6](#icmpv6), [DNS - AAAA](d.md#dns-aaaa). + +## ISDN +**Integrated Services Digital Network.** A set of standards for digital voice and data over the telephone network, using 64 kbit/s channels with separate signalling. It was sold as basic rate (BRI) and primary rate (PRI) access. + +![ISDN](images/15008408.png) + +[[CAPTION:Figure]] ISDN. + +See also: [BRI](b.md#bri), [PRI](p.md#pri), [Q.931](q.md#q931), [ISDN2](#isdn2), [ISDN30](#isdn30). + +## ISDN2 +BT's name for ISDN basic rate access: two 64 kbit/s channels. See [BRI](b.md#bri). + +## ISDN30 +BT's name for ISDN primary rate access: up to 30 channels on a 2 Mbit/s line. See [PRI](p.md#pri). + +## ISO +<!-- vale PyDoc.Spelling = NO --> +**International Organization for Standardization.** +<!-- vale PyDoc.Spelling = YES --> +The international body made up of national standards bodies, such as BSI and ANSI, that publishes standards in almost every field. + +See also: [IEC](#iec), [ANSI](a.md#ansi). + +## ISP +**Internet Service Provider.** A company that provides internet access, and often related services such as email, hosting and domain registration. + +![ISP](images/15008414.png) + +[[CAPTION:Figure]] ISP. + +## Issue +Something that needs resolving and could affect the project if left. In this sense it is not already covered by the plan, a change request, a defect or a risk. + +See also: [Issue Management](#issue-management), [Issue tracker](#issue-tracker). + +## Issue Management +Recording, tracking and resolving a project's issues. + +See also: [Issue](#issue). + +## Issue tracker +Software that records issues, such as bugs, tasks or support requests, and tracks them from report to resolution. + +See also: [Defect Management](d.md#defect-management). + +## ISUP +**ISDN User Part.** The SS7 protocol that sets up and clears telephone calls between exchanges, carrying details such as the called number. It is defined in the ITU-T Q.76x series. + +![ISUP](images/15008420.png) + +[[CAPTION:Figure]] ISUP. + +See also: [SS7](s.md#ss7), [IUP](#iup), [TUP](t.md#tup). + +## ITIL +A widely used framework of good practice for IT service management, covering how services are designed, delivered, run and improved. The name came from the Information Technology Infrastructure Library. + +See also: [CMDB](c.md#cmdb), [CCTA](c.md#ccta). + +## ITU +**International Telecommunication Union.** The United Nations agency for information and communication technologies. It coordinates the radio spectrum and satellite orbits, and sets telecoms standards through ITU-T. + +See also: [ITU-T](#itu-t), [ETSI](e.md#etsi). + +## ITU-T +The ITU's standardisation sector, which publishes telecoms standards as Recommendations, such as G.711 and H.323. It was called the CCITT until 1993. + +See also: [ITU](#itu), [CCITT](c.md#ccitt). + +## IUP +**Interconnect User Part.** The UK national version of SS7 call signalling between operators' networks, formerly called BTNUP. + +See also: [ISUP](#isup), [BTNUP](b.md#btnup). + +## IVR +**Interactive Voice Response.** A telephone system that talks to callers with recorded or generated speech and responds to their key presses or spoken answers, typically to route calls or serve simple requests. + +See also: [DTMF](d.md#dtmf). + +## Item +In a backlog or tracker, one unit of work being managed, such as a change request, defect, risk or issue. + +See also: [Work Item](w.md#work-item). + +## Iteration +A fixed period, or timebox, in which a team develops and delivers a working increment. + +See also: [Sprints](s.md#sprints), [Timebox](t.md#timebox). + +## Iterative Development +Developing by repeating cycles of work and deliberately revisiting earlier work in the light of feedback, rather than doing each step once. + +See also: [Incremental Development](#incremental-development), [Refactoring](r.md#refactoring). diff --git a/content/glossary/images/14123067.png b/content/glossary/images/14123067.png new file mode 100644 index 0000000..82cb674 Binary files /dev/null and b/content/glossary/images/14123067.png differ diff --git a/content/glossary/images/14123073.png b/content/glossary/images/14123073.png new file mode 100644 index 0000000..82cb674 Binary files /dev/null and b/content/glossary/images/14123073.png differ diff --git a/content/glossary/images/14123086.png b/content/glossary/images/14123086.png new file mode 100644 index 0000000..4fac398 Binary files /dev/null and b/content/glossary/images/14123086.png differ diff --git a/content/glossary/images/14614584.png b/content/glossary/images/14614584.png new file mode 100644 index 0000000..a596ae1 Binary files /dev/null and b/content/glossary/images/14614584.png differ diff --git a/content/glossary/images/14614590.png b/content/glossary/images/14614590.png new file mode 100644 index 0000000..73f5517 Binary files /dev/null and b/content/glossary/images/14614590.png differ diff --git a/content/glossary/images/14647331.svg b/content/glossary/images/14647331.svg new file mode 100644 index 0000000..4e55740 --- /dev/null +++ b/content/glossary/images/14647331.svg @@ -0,0 +1,6404 @@ +<?xml version='1.0' encoding='UTF-8' standalone='yes'?> +<!-- Created with gnuclad 0.2.4 --> + +<svg + xmlns='http://www.w3.org/2000/svg' + xmlns:svg='http://www.w3.org/2000/svg' + xmlns:xlink='http://www.w3.org/1999/xlink' + xmlns:inkscape='http://www.inkscape.org/namespaces/inkscape' + xmlns:sodipodi='http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd' + xmlns:dc='http://purl.org/dc/elements/1.1/' + xmlns:cc='http://creativecommons.org/ns#' + xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' + version='1.1' + width='3120' + height='12378' +> + +<title>Linux Distributions Timeline + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Burapha + + MNIS + + Stampede + + VectorLinux + + WHAX + + SLAMPP + + DNALinux + + Wolvix + + Slax + + Plamo + + Stux + + STUX + + DARKSTAR + + Minislack + + Zenwalk + + Slack 390 + + Kate + + KateOS + + MOPSLinux + + AgiliaLinux + + Slamd64 + + pQui + + RIP + + Bluewhite64 + + Wifislax + + Draco + + Wifiway + + Kongoni + + Salix + + Slackel + + S.u.S.E + + Porteus + + SlaXBMC + + Linvo + + PLATYPUX + + SMS + + Superb Mini Server + + SL4P + + ARMedslack + + Slackware ARM + + Absolute + + Voltalinux + + Slackintosh + + NetSecL + + Frugalware + + NimbleX + + HostGIS + + GoblinX + + Imagineos + + AUSTRUMI + + Kwort + + DeLicate + + DeLi + + ConnochaetOS + + Topologilinux + + Splack Linux + + Tiny + + Slackware + + SLS + + Libranet + + Gibraltar + + Progeny + + 64Studio + + DeMuDi + + Lihuen + + gnuLinEx + + BlackRhino + + ASLinux + + Guadalinex + + DeadCD + + Olive + + Underground Desktop + + grml + + Arco-Debian + + Arc-Live + + Resulinux + + Voyage Linux + + LliureX + + Elive + + Finnix + + 2XOS + + Epidemic + + Librassoc + + siduction + + sidux + + aptosid + + Webconverger + + GALPon MiniNo + + Estrella Roja + + Metamorphose + + Canaima + + Matriux + + DoudouLinux + + Linux Mint Debian Edition + + GNUGuitarINUX + + Liquid Lemur + + Snowlinux + + TurnKey + + Xamin + + LinuxBBQ + + Point Linux + + SparkyLinux + + Rebellin + + Volumio + + armbian + + Tanglu + + AV Linux + + MX Linux + + Q4OS + + Refracta + + Devuan + + deepin + + Parrot + + Corel + + Xandros + + Netrunner + + PrimTux + + OSMC + + SELKS + + Endless OS + + RoboLinux + + ToriOS + + Ye Olde SteamOSe + + Stephenson's Rocket + + SteamOS + + Rescatux + + Kali + + Univention Corporate Server + + Raspbmc + + Raspyfi + + Moode + + Raspbian + + Raspberry Pi OS + + YunoHost + + SolusOS + + OpenMediaVault + + Red Ribbon + + SalineOS + + Progress + + UberStudent + + Semplice + + Tails + + Untangle + + Inquisitor + + Proxmox + + PelicanHPC + + BOSS + + PureOS + + VyOS + + Vyatta + + Brocade Vyatta vRouter + + NepaLinux + + OS2005 + + Maemo + + Meego Harmattan + + Tuquito + + VENENUX + + Trisquel + + BlankOn + + MoLinux + + Ulteo + + Asturix + + Bardinux + + Netrunner + + DEFT + + Kubuntu + + Gobuntu + + Runtu + + Voyager + + GalliumOS + + PC OS + + OS4 + + Black Lab + + Peach OSI + + Xinutop + + Xubuntu + + PUD + + xPUD + + Hiweed Linux + + Linux Deepin + + Deepin + + deepin + + Muslim Edition + + Sabily + + MAX + + Ubuntu Studio + + Moblin + + Geubuntu + + OpenGEU + + Kiwi + + Eeebuntu + + Aurora OS + + Zebuntu + + ZevenOS + + Ultimate Edition + + MoonOS + + eBox + + Zentyal + + Newtoos + + Vibuntu + + VINUX + + Qimo + + OSGeo-Live + + Peppermint OS + + LXLE + + Lubuntu + + Ylmf OS + + StartOS + + Element + + Jolicloud + + SecUntu + + BackBox + + Pinguy OS + + elementary OS + + Pear OS + + Descent OS + + Ubuntu-Pack + + Ubuntu GNOME + + Ubuntu Kylin + + Ubuntu MATE + + KDE Neon + + Pop OS + + Arabian + + Budgie-Remix + + Ubuntu Budgie + + Snakeoil OS + + Ubuntu Touch + + Linux Lite + + Santoku + + TorBOX + + Whonix + + Linkat + + Sn0wL1nuX + + LuninuX OS + + Bodhi + + Dream Studio + + Tango Studio + + AriOS + + GnackTrack + + Backtrack + + wattOS + + Monomaxos + + AV + + AV Linux + + XBMC Live + + XBMCbuntu + + CAINE + + Zorin OS + + Leeenux + + Ubuntu eee + + Easy Peasy + + Ubuntu Rescue Remix + + CrunchBang plus plus + + CrunchBang + + Bunsen Labs + + Fluxbuntu + + Greenie + + Ubuntulite + + U-lite + + Mythbuntu + + Madbox + + Uberyl + + ComFusion + + gNewSense + + Commodore OS + + Linux Mint + + Christian Edition + + nUbuntu + + Edubuntu + + Ubuntu + + Hiweed Linux + + Clonezilla Live + + DRBL + + K-DEMar + + kademar + + antiX + + MEPIS + + SimplyMEPIS + + Rxart + + Freespire + + Lindows + + Linspire + + Skolelinux + + DebianEdu + + Quantian + + Damn Vulnerable Linux + + Damn Small Linux + + KnoppMyth + + Danix + + Parsix + + Kanotix + + MAX + + Kaella + + ParallelKnoppix + + Musix + + NeoDizinha + + Patinho Faminto + + Symphony OS + + Whoppix + + B2D + + Auditor Security Collection + + Backtrack + + Feather + + Medialinux + + Mediainlinux + + ArtistX + + INSERT + + Aquamorph + + Shift Linux + + Hiwix + + Hiweed Desktop + + Dreamlinux + + Morphix + + Kalango + + Dizinha + + Poseidon + + Kurumin + + Knoppix + + Omoikane Arma + + Debian + + Redmond + + Lycoris Desktop/LX + + Caldera OpenLinux + + SCO OpenLinux + + Conectiva + + PLD + + Turbolinux + + Fermi + + e-smith + + SME Server + + Yellow Dog + + Red Flag + + Kondara + + ELX + + Finnix + + Asianux + + Miracle + + Best + + SOT + + LBA + + ClarkConnect + + ClearOS + + Independence + + Aurora + + Bayanihan + + Aurox + + AnNyung + + SuliX + + PUIAS + + Springdale Linux + + White Box + + Scientific + + Tao + + Oracle Enterprise + + Oracle Linux + + EuroLinux + + WGS Linux Pro + + ServOS + + Endian + + StartCom + + CERN + + Elastix + + Rockstor + + Baruwa + + Asterisk at Home + + trixbox + + NethServer + + BlueOnyx + + CentOS + + Momonga + + Ekaaty + + Vixta + + Simplis + + Xange + + eZeY + + Tizen + + Sailfish OS + + Mer + + MeeGo + + Moblin 2 + + Hanthana + + Synergy + + Kororaa + + Korora + + Viperr + + Chapeau + + NST + + Maui OS + + Hawaii OS + + Parsidora + + Fuduntu + + Qubes OS + + VortexBox + + Ojuba + + Amahi + + MythDora + + Fedora Core + + Fedora + + ATmission + + Berry + + BLAG + + EnGarde + + K12 + + ASPLinux + + Rocks + + Pingo + + BU Linux + + Trustix + + Linpus + + Immunix + + Vine + + Eridani + + KRUD + + ALT + + SAM + + SAMity + + OpenSLS + + Annvix + + Mageia + + Demolinux + + OpenMandriva Lx + + ROSA + + Unity Linux + + blackPanther + + Granular + + TinyMe + + Phinx + + PCLinuxOS + + Mandrake + + Mandriva + + Eurielec + + Buhawi + + LinuxPPC + + Red Hat + + Red Hat Enterprise + + SUSE Linux Enterprise Server + + Caixa Mágica + + SCO Linux + + United Linux + + Sun JDS + + GeckoLinux + + Astaro + + EasyNAS + + Novell Linux Desktop + + SUSE Linux Enterprise Desktop + + Stresslinux + + S.u.S.E Linux + + SuSE Linux + + SUSE Linux + + openSUSE + + Jurix + + Gentoox + + Knopperdisk + + RR4 + + Sabayon + + Pardus + + Calculate + + SystemRescueCD + + Nova + + Liberté + + CoreOS + + Container Linux + + wtfplay-live + + Ututo-e + + Porteus Kiosk + + Daphile + + NayuOS + + CloudReady + + Flint OS + + Chromium OS + + Funtoo + + Toorox + + Papug + + Kororaa + + Pentoo + + VidaLinux + + VLOS + + Ututo + + Ututo XS + + Enoch + + Gentoo + + Chakra + + KahelOS + + Arch Linux ARM + + Parabola + + Bluestar Linux + + PoliArch + + BBQLinux + + AudioPhile Linux + + ARCHLabs + + Artix + + Arch Linux 32 + + AL-AMLUG + + Archie + + SystemRescueCD + + SystemRescue + + ArchMerge + + ArcoLinux + + Hyperbola + + Obarun + + BlackArch + + Cinnarch + + Antergos + + Bridge + + Netrunner Rolling + + Manjaro + + ArchBang + + CTKarchLive + + CTKArch + + LinHES + + Arch Linux + + MCC Interim + + TAMU + + Yggdrasil + + DLD + + LST + + Bogus + + Xdenu + + Linux-FT + + Mini + + Trans-Ameritech + + Unifix + + Linux Universe + + Craftworks + + DILINUX + + DOSLINUX + + mkLinux + + Monkey + + LEAF + + Linux Router Project + + Weaver + + Nitix + + uClinux + + T2 + + ROCK + + FREESCO + + Project Ballantin + + tomsrtbt + + Coyote + + eIT easyLinux + + Peanut + + aLinux + + BluePoint + + NuTyX + + AryaLinux + + ZENIX + + KaarPux + + Linux From Scratch + + IPFire + + IPCop + + SmoothWall GPL + + SmoothWall Express + + CRUX + + Beehive + + Midori + + Leka Rescue Floppy + + Openwall + + dyne:bolic + + Lede Project + + OpenWRT + + Ark + + LPS + + TENS + + Thinstation + + NetStation + + Source Mage + + Lunar Linux + + Sorcerer + + Tinfoil Hat + + LinuxConsole + + GoboLinux + + Yoper + + UHU + + GeeXboX + + Macpup + + Simplicity + + TEENpup + + Legacy OS + + Sage Live CD + + Puppy + + Quirky + + Devil + + NixOS + + GuixSD + + Guix System + + QiLinux + + Natures Linux + + Openfiler + + Octoz + + Hedinux + + Foresight + + Specifix + + rPath + + Paldo + + BrazilFW + + Jarro Negro + + Ophcrack + + postmarketOS + + Alpine Linux + + Everest + + Qomo + + Zeroshell + + Parted Magic + + Tiny SliTaz + + SliTaz + + openmamba + + Syllable Server + + Ångström + + PLoP + + VoidLinux + + Exherbo + + dCore + + piCorePlayer + + Tiny Core + + Dragora + + LuneOS + + webOS + + LibreELEC + + Lakka + + OpenELEC + + 0Linux + + 4M + + noop + + Bedrock + + AOSC OS + + KaOS + + Minimal Linux Live + + Pisi Linux + + Evolve OS + + Solus + + Clear Linux + + RancherOS + + Super Grub2 Disk + + OviOS + + Asteroid OS + + januslinux + + Ataraxia Linux + + Android-x86 + + OmniROM + + LineageOS + + Android-IA + + CyanogenMod + + AOSP + + + + + + + 1992 + 1992 + 1993 + 1993 + 1994 + 1994 + 1995 + 1995 + 1996 + 1996 + 1997 + 1997 + 1998 + 1998 + 1999 + 1999 + 2000 + 2000 + 2001 + 2001 + 2002 + 2002 + 2003 + 2003 + 2004 + 2004 + 2005 + 2005 + 2006 + 2006 + 2007 + 2007 + 2008 + 2008 + 2009 + 2009 + 2010 + 2010 + 2011 + 2011 + 2012 + 2012 + 2013 + 2013 + 2014 + 2014 + 2015 + 2015 + 2016 + 2016 + 2017 + 2017 + 2018 + 2018 + 2019 + 2019 + 2020 + 2020 + 2021 + 2021 + 2022 + 2022 + + + + + + + Linux Distributions Timeline + Version 21.10 + + © Andreas Lundqvist, Donjan Rodic, Mohammed A. Mustafa + © Muhammad Herdiansyah, Fabio Loli + https://github.com/FabioLolix/LinuxTimeline + Original source: futurist.se/gldt + Published under the GNU Free Documentation License + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Influence, developer switching + Rebasing, substantial code flow, project overtaking + Developer & code sharing, project merging + + + + + + + + + diff --git a/content/glossary/images/14876769.png b/content/glossary/images/14876769.png new file mode 100644 index 0000000..a596ae1 Binary files /dev/null and b/content/glossary/images/14876769.png differ diff --git a/content/glossary/images/14909510.png b/content/glossary/images/14909510.png new file mode 100644 index 0000000..b4b8bcc Binary files /dev/null and b/content/glossary/images/14909510.png differ diff --git a/content/glossary/images/14909529.png b/content/glossary/images/14909529.png new file mode 100644 index 0000000..6571871 Binary files /dev/null and b/content/glossary/images/14909529.png differ diff --git a/content/glossary/images/15007749.png b/content/glossary/images/15007749.png new file mode 100644 index 0000000..e4d42f4 Binary files /dev/null and b/content/glossary/images/15007749.png differ diff --git a/content/glossary/images/15007760.png b/content/glossary/images/15007760.png new file mode 100644 index 0000000..5f858bb Binary files /dev/null and b/content/glossary/images/15007760.png differ diff --git a/content/glossary/images/15007768.png b/content/glossary/images/15007768.png new file mode 100644 index 0000000..23ebac6 Binary files /dev/null and b/content/glossary/images/15007768.png differ diff --git a/content/glossary/images/15007774.png b/content/glossary/images/15007774.png new file mode 100644 index 0000000..f8aaeef Binary files /dev/null and b/content/glossary/images/15007774.png differ diff --git a/content/glossary/images/15007780.png b/content/glossary/images/15007780.png new file mode 100644 index 0000000..316799a Binary files /dev/null and b/content/glossary/images/15007780.png differ diff --git a/content/glossary/images/15007786.png b/content/glossary/images/15007786.png new file mode 100644 index 0000000..3eec70c Binary files /dev/null and b/content/glossary/images/15007786.png differ diff --git a/content/glossary/images/15007792.png b/content/glossary/images/15007792.png new file mode 100644 index 0000000..11a9e75 Binary files /dev/null and b/content/glossary/images/15007792.png differ diff --git a/content/glossary/images/15007798.png b/content/glossary/images/15007798.png new file mode 100644 index 0000000..6ab5eb0 Binary files /dev/null and b/content/glossary/images/15007798.png differ diff --git a/content/glossary/images/15007804.png b/content/glossary/images/15007804.png new file mode 100644 index 0000000..3019078 Binary files /dev/null and b/content/glossary/images/15007804.png differ diff --git a/content/glossary/images/15007813.png b/content/glossary/images/15007813.png new file mode 100644 index 0000000..085654d Binary files /dev/null and b/content/glossary/images/15007813.png differ diff --git a/content/glossary/images/15007821.png b/content/glossary/images/15007821.png new file mode 100644 index 0000000..87dd010 Binary files /dev/null and b/content/glossary/images/15007821.png differ diff --git a/content/glossary/images/15007827.png b/content/glossary/images/15007827.png new file mode 100644 index 0000000..5f9cfc8 Binary files /dev/null and b/content/glossary/images/15007827.png differ diff --git a/content/glossary/images/15007833.png b/content/glossary/images/15007833.png new file mode 100644 index 0000000..1886406 Binary files /dev/null and b/content/glossary/images/15007833.png differ diff --git a/content/glossary/images/15007839.png b/content/glossary/images/15007839.png new file mode 100644 index 0000000..1cd10f3 Binary files /dev/null and b/content/glossary/images/15007839.png differ diff --git a/content/glossary/images/15007845.png b/content/glossary/images/15007845.png new file mode 100644 index 0000000..804b57d Binary files /dev/null and b/content/glossary/images/15007845.png differ diff --git a/content/glossary/images/15007851.png b/content/glossary/images/15007851.png new file mode 100644 index 0000000..74ed047 Binary files /dev/null and b/content/glossary/images/15007851.png differ diff --git a/content/glossary/images/15007857.png b/content/glossary/images/15007857.png new file mode 100644 index 0000000..cd32db8 Binary files /dev/null and b/content/glossary/images/15007857.png differ diff --git a/content/glossary/images/15007863.png b/content/glossary/images/15007863.png new file mode 100644 index 0000000..b0e4a1b Binary files /dev/null and b/content/glossary/images/15007863.png differ diff --git a/content/glossary/images/15007869.png b/content/glossary/images/15007869.png new file mode 100644 index 0000000..5bb8bb3 Binary files /dev/null and b/content/glossary/images/15007869.png differ diff --git a/content/glossary/images/15007880.png b/content/glossary/images/15007880.png new file mode 100644 index 0000000..b76ea3b Binary files /dev/null and b/content/glossary/images/15007880.png differ diff --git a/content/glossary/images/15007888.png b/content/glossary/images/15007888.png new file mode 100644 index 0000000..28fdfec Binary files /dev/null and b/content/glossary/images/15007888.png differ diff --git a/content/glossary/images/15007894.png b/content/glossary/images/15007894.png new file mode 100644 index 0000000..120e94e Binary files /dev/null and b/content/glossary/images/15007894.png differ diff --git a/content/glossary/images/15007900.png b/content/glossary/images/15007900.png new file mode 100644 index 0000000..aa2590f Binary files /dev/null and b/content/glossary/images/15007900.png differ diff --git a/content/glossary/images/15007906.png b/content/glossary/images/15007906.png new file mode 100644 index 0000000..d5e9856 Binary files /dev/null and b/content/glossary/images/15007906.png differ diff --git a/content/glossary/images/15007912.png b/content/glossary/images/15007912.png new file mode 100644 index 0000000..f1a7a99 Binary files /dev/null and b/content/glossary/images/15007912.png differ diff --git a/content/glossary/images/15007918.png b/content/glossary/images/15007918.png new file mode 100644 index 0000000..3066cb1 Binary files /dev/null and b/content/glossary/images/15007918.png differ diff --git a/content/glossary/images/15007924.png b/content/glossary/images/15007924.png new file mode 100644 index 0000000..6ba6eb8 Binary files /dev/null and b/content/glossary/images/15007924.png differ diff --git a/content/glossary/images/15007930.png b/content/glossary/images/15007930.png new file mode 100644 index 0000000..cad4ee9 Binary files /dev/null and b/content/glossary/images/15007930.png differ diff --git a/content/glossary/images/15007936.png b/content/glossary/images/15007936.png new file mode 100644 index 0000000..74f9f76 Binary files /dev/null and b/content/glossary/images/15007936.png differ diff --git a/content/glossary/images/15007942.png b/content/glossary/images/15007942.png new file mode 100644 index 0000000..87335ad Binary files /dev/null and b/content/glossary/images/15007942.png differ diff --git a/content/glossary/images/15007948.png b/content/glossary/images/15007948.png new file mode 100644 index 0000000..5907dd3 Binary files /dev/null and b/content/glossary/images/15007948.png differ diff --git a/content/glossary/images/15007957.png b/content/glossary/images/15007957.png new file mode 100644 index 0000000..b4b1ee1 Binary files /dev/null and b/content/glossary/images/15007957.png differ diff --git a/content/glossary/images/15007965.png b/content/glossary/images/15007965.png new file mode 100644 index 0000000..5171fe0 Binary files /dev/null and b/content/glossary/images/15007965.png differ diff --git a/content/glossary/images/15007971.png b/content/glossary/images/15007971.png new file mode 100644 index 0000000..c1f888d Binary files /dev/null and b/content/glossary/images/15007971.png differ diff --git a/content/glossary/images/15007977.png b/content/glossary/images/15007977.png new file mode 100644 index 0000000..89e6430 Binary files /dev/null and b/content/glossary/images/15007977.png differ diff --git a/content/glossary/images/15007983.png b/content/glossary/images/15007983.png new file mode 100644 index 0000000..94a4362 Binary files /dev/null and b/content/glossary/images/15007983.png differ diff --git a/content/glossary/images/15007989.png b/content/glossary/images/15007989.png new file mode 100644 index 0000000..62532b1 Binary files /dev/null and b/content/glossary/images/15007989.png differ diff --git a/content/glossary/images/15007995.png b/content/glossary/images/15007995.png new file mode 100644 index 0000000..ceb51ae Binary files /dev/null and b/content/glossary/images/15007995.png differ diff --git a/content/glossary/images/15008001.png b/content/glossary/images/15008001.png new file mode 100644 index 0000000..d82b24b Binary files /dev/null and b/content/glossary/images/15008001.png differ diff --git a/content/glossary/images/15008007.png b/content/glossary/images/15008007.png new file mode 100644 index 0000000..02e2d20 Binary files /dev/null and b/content/glossary/images/15008007.png differ diff --git a/content/glossary/images/15008013.png b/content/glossary/images/15008013.png new file mode 100644 index 0000000..15b6b0d Binary files /dev/null and b/content/glossary/images/15008013.png differ diff --git a/content/glossary/images/15008019.png b/content/glossary/images/15008019.png new file mode 100644 index 0000000..d8aba4d Binary files /dev/null and b/content/glossary/images/15008019.png differ diff --git a/content/glossary/images/15008025.png b/content/glossary/images/15008025.png new file mode 100644 index 0000000..0bce2f5 Binary files /dev/null and b/content/glossary/images/15008025.png differ diff --git a/content/glossary/images/15008031.png b/content/glossary/images/15008031.png new file mode 100644 index 0000000..40682e9 Binary files /dev/null and b/content/glossary/images/15008031.png differ diff --git a/content/glossary/images/15008037.png b/content/glossary/images/15008037.png new file mode 100644 index 0000000..a80e09b Binary files /dev/null and b/content/glossary/images/15008037.png differ diff --git a/content/glossary/images/15008043.png b/content/glossary/images/15008043.png new file mode 100644 index 0000000..d4cd277 Binary files /dev/null and b/content/glossary/images/15008043.png differ diff --git a/content/glossary/images/15008049.png b/content/glossary/images/15008049.png new file mode 100644 index 0000000..2c6fdce Binary files /dev/null and b/content/glossary/images/15008049.png differ diff --git a/content/glossary/images/15008055.png b/content/glossary/images/15008055.png new file mode 100644 index 0000000..182ee13 Binary files /dev/null and b/content/glossary/images/15008055.png differ diff --git a/content/glossary/images/15008061.png b/content/glossary/images/15008061.png new file mode 100644 index 0000000..33d31fc Binary files /dev/null and b/content/glossary/images/15008061.png differ diff --git a/content/glossary/images/15008067.png b/content/glossary/images/15008067.png new file mode 100644 index 0000000..7e73ca9 Binary files /dev/null and b/content/glossary/images/15008067.png differ diff --git a/content/glossary/images/15008073.png b/content/glossary/images/15008073.png new file mode 100644 index 0000000..4d96f3e Binary files /dev/null and b/content/glossary/images/15008073.png differ diff --git a/content/glossary/images/15008079.png b/content/glossary/images/15008079.png new file mode 100644 index 0000000..fb273ad Binary files /dev/null and b/content/glossary/images/15008079.png differ diff --git a/content/glossary/images/15008088.png b/content/glossary/images/15008088.png new file mode 100644 index 0000000..86ebd2e Binary files /dev/null and b/content/glossary/images/15008088.png differ diff --git a/content/glossary/images/15008096.png b/content/glossary/images/15008096.png new file mode 100644 index 0000000..5dc4ee0 Binary files /dev/null and b/content/glossary/images/15008096.png differ diff --git a/content/glossary/images/15008102.png b/content/glossary/images/15008102.png new file mode 100644 index 0000000..d8791ed Binary files /dev/null and b/content/glossary/images/15008102.png differ diff --git a/content/glossary/images/15008108.png b/content/glossary/images/15008108.png new file mode 100644 index 0000000..38f7f71 Binary files /dev/null and b/content/glossary/images/15008108.png differ diff --git a/content/glossary/images/15008114.png b/content/glossary/images/15008114.png new file mode 100644 index 0000000..65e7961 Binary files /dev/null and b/content/glossary/images/15008114.png differ diff --git a/content/glossary/images/15008120.png b/content/glossary/images/15008120.png new file mode 100644 index 0000000..c4fe611 Binary files /dev/null and b/content/glossary/images/15008120.png differ diff --git a/content/glossary/images/15008126.png b/content/glossary/images/15008126.png new file mode 100644 index 0000000..1c1b87d Binary files /dev/null and b/content/glossary/images/15008126.png differ diff --git a/content/glossary/images/15008132.png b/content/glossary/images/15008132.png new file mode 100644 index 0000000..1739025 Binary files /dev/null and b/content/glossary/images/15008132.png differ diff --git a/content/glossary/images/15008172.png b/content/glossary/images/15008172.png new file mode 100644 index 0000000..cb352f4 Binary files /dev/null and b/content/glossary/images/15008172.png differ diff --git a/content/glossary/images/15008180.png b/content/glossary/images/15008180.png new file mode 100644 index 0000000..376e05f Binary files /dev/null and b/content/glossary/images/15008180.png differ diff --git a/content/glossary/images/15008186.png b/content/glossary/images/15008186.png new file mode 100644 index 0000000..c83c350 Binary files /dev/null and b/content/glossary/images/15008186.png differ diff --git a/content/glossary/images/15008192.png b/content/glossary/images/15008192.png new file mode 100644 index 0000000..aa5fa6c Binary files /dev/null and b/content/glossary/images/15008192.png differ diff --git a/content/glossary/images/15008198.png b/content/glossary/images/15008198.png new file mode 100644 index 0000000..cbc0206 Binary files /dev/null and b/content/glossary/images/15008198.png differ diff --git a/content/glossary/images/15008204.png b/content/glossary/images/15008204.png new file mode 100644 index 0000000..b1548b4 Binary files /dev/null and b/content/glossary/images/15008204.png differ diff --git a/content/glossary/images/15008210.png b/content/glossary/images/15008210.png new file mode 100644 index 0000000..0486fa1 Binary files /dev/null and b/content/glossary/images/15008210.png differ diff --git a/content/glossary/images/15008219.png b/content/glossary/images/15008219.png new file mode 100644 index 0000000..01f99d1 Binary files /dev/null and b/content/glossary/images/15008219.png differ diff --git a/content/glossary/images/15008227.png b/content/glossary/images/15008227.png new file mode 100644 index 0000000..2b97b09 Binary files /dev/null and b/content/glossary/images/15008227.png differ diff --git a/content/glossary/images/15008233.png b/content/glossary/images/15008233.png new file mode 100644 index 0000000..141849c Binary files /dev/null and b/content/glossary/images/15008233.png differ diff --git a/content/glossary/images/15008239.png b/content/glossary/images/15008239.png new file mode 100644 index 0000000..5c50e48 Binary files /dev/null and b/content/glossary/images/15008239.png differ diff --git a/content/glossary/images/15008245.png b/content/glossary/images/15008245.png new file mode 100644 index 0000000..a613738 Binary files /dev/null and b/content/glossary/images/15008245.png differ diff --git a/content/glossary/images/15008251.png b/content/glossary/images/15008251.png new file mode 100644 index 0000000..5107ee8 Binary files /dev/null and b/content/glossary/images/15008251.png differ diff --git a/content/glossary/images/15008257.png b/content/glossary/images/15008257.png new file mode 100644 index 0000000..06ce6ea Binary files /dev/null and b/content/glossary/images/15008257.png differ diff --git a/content/glossary/images/15008263.png b/content/glossary/images/15008263.png new file mode 100644 index 0000000..0ab2ca9 Binary files /dev/null and b/content/glossary/images/15008263.png differ diff --git a/content/glossary/images/15008304.png b/content/glossary/images/15008304.png new file mode 100644 index 0000000..6041c33 Binary files /dev/null and b/content/glossary/images/15008304.png differ diff --git a/content/glossary/images/15008312.png b/content/glossary/images/15008312.png new file mode 100644 index 0000000..0b89a7d Binary files /dev/null and b/content/glossary/images/15008312.png differ diff --git a/content/glossary/images/15008318.png b/content/glossary/images/15008318.png new file mode 100644 index 0000000..93b4c20 Binary files /dev/null and b/content/glossary/images/15008318.png differ diff --git a/content/glossary/images/15008324.png b/content/glossary/images/15008324.png new file mode 100644 index 0000000..8a1d674 Binary files /dev/null and b/content/glossary/images/15008324.png differ diff --git a/content/glossary/images/15008330.png b/content/glossary/images/15008330.png new file mode 100644 index 0000000..6fc571b Binary files /dev/null and b/content/glossary/images/15008330.png differ diff --git a/content/glossary/images/15008336.png b/content/glossary/images/15008336.png new file mode 100644 index 0000000..71323a9 Binary files /dev/null and b/content/glossary/images/15008336.png differ diff --git a/content/glossary/images/15008342.png b/content/glossary/images/15008342.png new file mode 100644 index 0000000..726f907 Binary files /dev/null and b/content/glossary/images/15008342.png differ diff --git a/content/glossary/images/15008352.png b/content/glossary/images/15008352.png new file mode 100644 index 0000000..8bc328d Binary files /dev/null and b/content/glossary/images/15008352.png differ diff --git a/content/glossary/images/15008360.png b/content/glossary/images/15008360.png new file mode 100644 index 0000000..86e7018 Binary files /dev/null and b/content/glossary/images/15008360.png differ diff --git a/content/glossary/images/15008366.png b/content/glossary/images/15008366.png new file mode 100644 index 0000000..55e8583 Binary files /dev/null and b/content/glossary/images/15008366.png differ diff --git a/content/glossary/images/15008372.png b/content/glossary/images/15008372.png new file mode 100644 index 0000000..68fe990 Binary files /dev/null and b/content/glossary/images/15008372.png differ diff --git a/content/glossary/images/15008378.png b/content/glossary/images/15008378.png new file mode 100644 index 0000000..a7bcf54 Binary files /dev/null and b/content/glossary/images/15008378.png differ diff --git a/content/glossary/images/15008384.png b/content/glossary/images/15008384.png new file mode 100644 index 0000000..ee983b6 Binary files /dev/null and b/content/glossary/images/15008384.png differ diff --git a/content/glossary/images/15008390.png b/content/glossary/images/15008390.png new file mode 100644 index 0000000..468651f Binary files /dev/null and b/content/glossary/images/15008390.png differ diff --git a/content/glossary/images/15008396.png b/content/glossary/images/15008396.png new file mode 100644 index 0000000..07a541c Binary files /dev/null and b/content/glossary/images/15008396.png differ diff --git a/content/glossary/images/15008402.png b/content/glossary/images/15008402.png new file mode 100644 index 0000000..202ff51 Binary files /dev/null and b/content/glossary/images/15008402.png differ diff --git a/content/glossary/images/15008408.png b/content/glossary/images/15008408.png new file mode 100644 index 0000000..eed8d14 Binary files /dev/null and b/content/glossary/images/15008408.png differ diff --git a/content/glossary/images/15008414.png b/content/glossary/images/15008414.png new file mode 100644 index 0000000..1174190 Binary files /dev/null and b/content/glossary/images/15008414.png differ diff --git a/content/glossary/images/15008420.png b/content/glossary/images/15008420.png new file mode 100644 index 0000000..d965607 Binary files /dev/null and b/content/glossary/images/15008420.png differ diff --git a/content/glossary/images/15008430.png b/content/glossary/images/15008430.png new file mode 100644 index 0000000..fda9074 Binary files /dev/null and b/content/glossary/images/15008430.png differ diff --git a/content/glossary/images/15008438.png b/content/glossary/images/15008438.png new file mode 100644 index 0000000..58fbff7 Binary files /dev/null and b/content/glossary/images/15008438.png differ diff --git a/content/glossary/images/15008447.png b/content/glossary/images/15008447.png new file mode 100644 index 0000000..4e79bd5 Binary files /dev/null and b/content/glossary/images/15008447.png differ diff --git a/content/glossary/images/15008455.png b/content/glossary/images/15008455.png new file mode 100644 index 0000000..dc19d2b Binary files /dev/null and b/content/glossary/images/15008455.png differ diff --git a/content/glossary/images/15008464.png b/content/glossary/images/15008464.png new file mode 100644 index 0000000..259d114 Binary files /dev/null and b/content/glossary/images/15008464.png differ diff --git a/content/glossary/images/15008478.png b/content/glossary/images/15008478.png new file mode 100644 index 0000000..d08596e Binary files /dev/null and b/content/glossary/images/15008478.png differ diff --git a/content/glossary/images/15008484.png b/content/glossary/images/15008484.png new file mode 100644 index 0000000..103b061 Binary files /dev/null and b/content/glossary/images/15008484.png differ diff --git a/content/glossary/images/15008490.png b/content/glossary/images/15008490.png new file mode 100644 index 0000000..eddc026 Binary files /dev/null and b/content/glossary/images/15008490.png differ diff --git a/content/glossary/images/15008496.png b/content/glossary/images/15008496.png new file mode 100644 index 0000000..7a808b9 Binary files /dev/null and b/content/glossary/images/15008496.png differ diff --git a/content/glossary/images/15008502.png b/content/glossary/images/15008502.png new file mode 100644 index 0000000..5f13915 Binary files /dev/null and b/content/glossary/images/15008502.png differ diff --git a/content/glossary/images/15008508.png b/content/glossary/images/15008508.png new file mode 100644 index 0000000..9e4cf09 Binary files /dev/null and b/content/glossary/images/15008508.png differ diff --git a/content/glossary/images/15008514.png b/content/glossary/images/15008514.png new file mode 100644 index 0000000..2ce03e4 Binary files /dev/null and b/content/glossary/images/15008514.png differ diff --git a/content/glossary/images/15008520.png b/content/glossary/images/15008520.png new file mode 100644 index 0000000..eba2e2a Binary files /dev/null and b/content/glossary/images/15008520.png differ diff --git a/content/glossary/images/15008526.png b/content/glossary/images/15008526.png new file mode 100644 index 0000000..dc2246b Binary files /dev/null and b/content/glossary/images/15008526.png differ diff --git a/content/glossary/images/15008537.png b/content/glossary/images/15008537.png new file mode 100644 index 0000000..00efecb Binary files /dev/null and b/content/glossary/images/15008537.png differ diff --git a/content/glossary/images/15008545.png b/content/glossary/images/15008545.png new file mode 100644 index 0000000..a06ecd0 Binary files /dev/null and b/content/glossary/images/15008545.png differ diff --git a/content/glossary/images/15008551.png b/content/glossary/images/15008551.png new file mode 100644 index 0000000..24e6b27 Binary files /dev/null and b/content/glossary/images/15008551.png differ diff --git a/content/glossary/images/15008557.png b/content/glossary/images/15008557.png new file mode 100644 index 0000000..1c00359 Binary files /dev/null and b/content/glossary/images/15008557.png differ diff --git a/content/glossary/images/15008563.png b/content/glossary/images/15008563.png new file mode 100644 index 0000000..d121148 Binary files /dev/null and b/content/glossary/images/15008563.png differ diff --git a/content/glossary/images/15008569.png b/content/glossary/images/15008569.png new file mode 100644 index 0000000..2688c05 Binary files /dev/null and b/content/glossary/images/15008569.png differ diff --git a/content/glossary/images/15008575.png b/content/glossary/images/15008575.png new file mode 100644 index 0000000..1aeab86 Binary files /dev/null and b/content/glossary/images/15008575.png differ diff --git a/content/glossary/images/15008581.png b/content/glossary/images/15008581.png new file mode 100644 index 0000000..ad75b26 Binary files /dev/null and b/content/glossary/images/15008581.png differ diff --git a/content/glossary/images/15008587.png b/content/glossary/images/15008587.png new file mode 100644 index 0000000..97e9fd7 Binary files /dev/null and b/content/glossary/images/15008587.png differ diff --git a/content/glossary/images/15008593.png b/content/glossary/images/15008593.png new file mode 100644 index 0000000..4baf9bc Binary files /dev/null and b/content/glossary/images/15008593.png differ diff --git a/content/glossary/images/15008599.png b/content/glossary/images/15008599.png new file mode 100644 index 0000000..791cdff Binary files /dev/null and b/content/glossary/images/15008599.png differ diff --git a/content/glossary/images/15008605.png b/content/glossary/images/15008605.png new file mode 100644 index 0000000..5ff7282 Binary files /dev/null and b/content/glossary/images/15008605.png differ diff --git a/content/glossary/images/15008611.png b/content/glossary/images/15008611.png new file mode 100644 index 0000000..1195bc6 Binary files /dev/null and b/content/glossary/images/15008611.png differ diff --git a/content/glossary/images/15008617.png b/content/glossary/images/15008617.png new file mode 100644 index 0000000..0186d64 Binary files /dev/null and b/content/glossary/images/15008617.png differ diff --git a/content/glossary/images/15008623.png b/content/glossary/images/15008623.png new file mode 100644 index 0000000..05311c5 Binary files /dev/null and b/content/glossary/images/15008623.png differ diff --git a/content/glossary/images/15008635.png b/content/glossary/images/15008635.png new file mode 100644 index 0000000..2384aa5 Binary files /dev/null and b/content/glossary/images/15008635.png differ diff --git a/content/glossary/images/15008641.png b/content/glossary/images/15008641.png new file mode 100644 index 0000000..a1f040f Binary files /dev/null and b/content/glossary/images/15008641.png differ diff --git a/content/glossary/images/15008647.png b/content/glossary/images/15008647.png new file mode 100644 index 0000000..f9578a4 Binary files /dev/null and b/content/glossary/images/15008647.png differ diff --git a/content/glossary/images/15008661.png b/content/glossary/images/15008661.png new file mode 100644 index 0000000..47c85dc Binary files /dev/null and b/content/glossary/images/15008661.png differ diff --git a/content/glossary/images/15008669.png b/content/glossary/images/15008669.png new file mode 100644 index 0000000..9c31b53 Binary files /dev/null and b/content/glossary/images/15008669.png differ diff --git a/content/glossary/images/15008675.png b/content/glossary/images/15008675.png new file mode 100644 index 0000000..87320c0 Binary files /dev/null and b/content/glossary/images/15008675.png differ diff --git a/content/glossary/images/15008681.png b/content/glossary/images/15008681.png new file mode 100644 index 0000000..ff40b29 Binary files /dev/null and b/content/glossary/images/15008681.png differ diff --git a/content/glossary/images/15008687.png b/content/glossary/images/15008687.png new file mode 100644 index 0000000..20fdbe6 Binary files /dev/null and b/content/glossary/images/15008687.png differ diff --git a/content/glossary/images/15008693.png b/content/glossary/images/15008693.png new file mode 100644 index 0000000..10653e6 Binary files /dev/null and b/content/glossary/images/15008693.png differ diff --git a/content/glossary/images/15008699.png b/content/glossary/images/15008699.png new file mode 100644 index 0000000..c2dab53 Binary files /dev/null and b/content/glossary/images/15008699.png differ diff --git a/content/glossary/images/15008705.png b/content/glossary/images/15008705.png new file mode 100644 index 0000000..7a4055a Binary files /dev/null and b/content/glossary/images/15008705.png differ diff --git a/content/glossary/images/15008711.png b/content/glossary/images/15008711.png new file mode 100644 index 0000000..9d97966 Binary files /dev/null and b/content/glossary/images/15008711.png differ diff --git a/content/glossary/images/15008720.png b/content/glossary/images/15008720.png new file mode 100644 index 0000000..8c98d55 Binary files /dev/null and b/content/glossary/images/15008720.png differ diff --git a/content/glossary/images/15008728.png b/content/glossary/images/15008728.png new file mode 100644 index 0000000..d46a30b Binary files /dev/null and b/content/glossary/images/15008728.png differ diff --git a/content/glossary/images/15008734.png b/content/glossary/images/15008734.png new file mode 100644 index 0000000..0ef5bac Binary files /dev/null and b/content/glossary/images/15008734.png differ diff --git a/content/glossary/images/15008740.png b/content/glossary/images/15008740.png new file mode 100644 index 0000000..bb94ae0 Binary files /dev/null and b/content/glossary/images/15008740.png differ diff --git a/content/glossary/images/15008750.png b/content/glossary/images/15008750.png new file mode 100644 index 0000000..6e3b5cd Binary files /dev/null and b/content/glossary/images/15008750.png differ diff --git a/content/glossary/images/15008758.png b/content/glossary/images/15008758.png new file mode 100644 index 0000000..5e50006 Binary files /dev/null and b/content/glossary/images/15008758.png differ diff --git a/content/glossary/images/15008764.png b/content/glossary/images/15008764.png new file mode 100644 index 0000000..436ff59 Binary files /dev/null and b/content/glossary/images/15008764.png differ diff --git a/content/glossary/images/15008770.png b/content/glossary/images/15008770.png new file mode 100644 index 0000000..d1e80e0 Binary files /dev/null and b/content/glossary/images/15008770.png differ diff --git a/content/glossary/images/15008776.png b/content/glossary/images/15008776.png new file mode 100644 index 0000000..d109dc7 Binary files /dev/null and b/content/glossary/images/15008776.png differ diff --git a/content/glossary/images/15008782.png b/content/glossary/images/15008782.png new file mode 100644 index 0000000..6c5867c Binary files /dev/null and b/content/glossary/images/15008782.png differ diff --git a/content/glossary/images/15008788.png b/content/glossary/images/15008788.png new file mode 100644 index 0000000..8102c6e Binary files /dev/null and b/content/glossary/images/15008788.png differ diff --git a/content/glossary/images/15008794.png b/content/glossary/images/15008794.png new file mode 100644 index 0000000..afda031 Binary files /dev/null and b/content/glossary/images/15008794.png differ diff --git a/content/glossary/images/15008800.png b/content/glossary/images/15008800.png new file mode 100644 index 0000000..37a76ff Binary files /dev/null and b/content/glossary/images/15008800.png differ diff --git a/content/glossary/images/15008806.png b/content/glossary/images/15008806.png new file mode 100644 index 0000000..bd51b25 Binary files /dev/null and b/content/glossary/images/15008806.png differ diff --git a/content/glossary/images/15008812.png b/content/glossary/images/15008812.png new file mode 100644 index 0000000..91749bf Binary files /dev/null and b/content/glossary/images/15008812.png differ diff --git a/content/glossary/images/15008818.png b/content/glossary/images/15008818.png new file mode 100644 index 0000000..720efac Binary files /dev/null and b/content/glossary/images/15008818.png differ diff --git a/content/glossary/images/15008824.png b/content/glossary/images/15008824.png new file mode 100644 index 0000000..953d02a Binary files /dev/null and b/content/glossary/images/15008824.png differ diff --git a/content/glossary/images/15008830.png b/content/glossary/images/15008830.png new file mode 100644 index 0000000..daeaf1a Binary files /dev/null and b/content/glossary/images/15008830.png differ diff --git a/content/glossary/images/15008836.png b/content/glossary/images/15008836.png new file mode 100644 index 0000000..2647db7 Binary files /dev/null and b/content/glossary/images/15008836.png differ diff --git a/content/glossary/images/15008851.png b/content/glossary/images/15008851.png new file mode 100644 index 0000000..491d164 Binary files /dev/null and b/content/glossary/images/15008851.png differ diff --git a/content/glossary/images/15008859.png b/content/glossary/images/15008859.png new file mode 100644 index 0000000..a986307 Binary files /dev/null and b/content/glossary/images/15008859.png differ diff --git a/content/glossary/images/15008865.png b/content/glossary/images/15008865.png new file mode 100644 index 0000000..15bef79 Binary files /dev/null and b/content/glossary/images/15008865.png differ diff --git a/content/glossary/images/15008871.png b/content/glossary/images/15008871.png new file mode 100644 index 0000000..b9cc3bb Binary files /dev/null and b/content/glossary/images/15008871.png differ diff --git a/content/glossary/images/15008877.png b/content/glossary/images/15008877.png new file mode 100644 index 0000000..04aaf05 Binary files /dev/null and b/content/glossary/images/15008877.png differ diff --git a/content/glossary/images/15008883.png b/content/glossary/images/15008883.png new file mode 100644 index 0000000..cb1c32b Binary files /dev/null and b/content/glossary/images/15008883.png differ diff --git a/content/glossary/images/15008889.png b/content/glossary/images/15008889.png new file mode 100644 index 0000000..26becac Binary files /dev/null and b/content/glossary/images/15008889.png differ diff --git a/content/glossary/images/15008895.png b/content/glossary/images/15008895.png new file mode 100644 index 0000000..68fb439 Binary files /dev/null and b/content/glossary/images/15008895.png differ diff --git a/content/glossary/images/15008901.png b/content/glossary/images/15008901.png new file mode 100644 index 0000000..c92d3cb Binary files /dev/null and b/content/glossary/images/15008901.png differ diff --git a/content/glossary/images/15008907.png b/content/glossary/images/15008907.png new file mode 100644 index 0000000..2df564c Binary files /dev/null and b/content/glossary/images/15008907.png differ diff --git a/content/glossary/images/15008913.png b/content/glossary/images/15008913.png new file mode 100644 index 0000000..c73f85f Binary files /dev/null and b/content/glossary/images/15008913.png differ diff --git a/content/glossary/images/15008919.png b/content/glossary/images/15008919.png new file mode 100644 index 0000000..5b151e2 Binary files /dev/null and b/content/glossary/images/15008919.png differ diff --git a/content/glossary/images/15008934.png b/content/glossary/images/15008934.png new file mode 100644 index 0000000..a4a414d Binary files /dev/null and b/content/glossary/images/15008934.png differ diff --git a/content/glossary/images/15008942.png b/content/glossary/images/15008942.png new file mode 100644 index 0000000..b7b8f5f Binary files /dev/null and b/content/glossary/images/15008942.png differ diff --git a/content/glossary/images/15008948.png b/content/glossary/images/15008948.png new file mode 100644 index 0000000..db389ae Binary files /dev/null and b/content/glossary/images/15008948.png differ diff --git a/content/glossary/images/15008954.png b/content/glossary/images/15008954.png new file mode 100644 index 0000000..c68e787 Binary files /dev/null and b/content/glossary/images/15008954.png differ diff --git a/content/glossary/images/15008960.png b/content/glossary/images/15008960.png new file mode 100644 index 0000000..9f1a9d2 Binary files /dev/null and b/content/glossary/images/15008960.png differ diff --git a/content/glossary/images/15008966.png b/content/glossary/images/15008966.png new file mode 100644 index 0000000..dad37bf Binary files /dev/null and b/content/glossary/images/15008966.png differ diff --git a/content/glossary/images/15008972.png b/content/glossary/images/15008972.png new file mode 100644 index 0000000..22e946b Binary files /dev/null and b/content/glossary/images/15008972.png differ diff --git a/content/glossary/images/15008978.png b/content/glossary/images/15008978.png new file mode 100644 index 0000000..3ba01d0 Binary files /dev/null and b/content/glossary/images/15008978.png differ diff --git a/content/glossary/images/15008984.png b/content/glossary/images/15008984.png new file mode 100644 index 0000000..8f7192a Binary files /dev/null and b/content/glossary/images/15008984.png differ diff --git a/content/glossary/images/15008990.png b/content/glossary/images/15008990.png new file mode 100644 index 0000000..ae750da Binary files /dev/null and b/content/glossary/images/15008990.png differ diff --git a/content/glossary/images/15008996.png b/content/glossary/images/15008996.png new file mode 100644 index 0000000..227ed9d Binary files /dev/null and b/content/glossary/images/15008996.png differ diff --git a/content/glossary/images/15009002.png b/content/glossary/images/15009002.png new file mode 100644 index 0000000..c17cc95 Binary files /dev/null and b/content/glossary/images/15009002.png differ diff --git a/content/glossary/images/15009008.png b/content/glossary/images/15009008.png new file mode 100644 index 0000000..60ef5ab Binary files /dev/null and b/content/glossary/images/15009008.png differ diff --git a/content/glossary/images/15009014.png b/content/glossary/images/15009014.png new file mode 100644 index 0000000..c3b61c9 Binary files /dev/null and b/content/glossary/images/15009014.png differ diff --git a/content/glossary/images/15009020.png b/content/glossary/images/15009020.png new file mode 100644 index 0000000..4fc02ec Binary files /dev/null and b/content/glossary/images/15009020.png differ diff --git a/content/glossary/images/15009026.png b/content/glossary/images/15009026.png new file mode 100644 index 0000000..c02372d Binary files /dev/null and b/content/glossary/images/15009026.png differ diff --git a/content/glossary/images/15009032.png b/content/glossary/images/15009032.png new file mode 100644 index 0000000..1412a8e Binary files /dev/null and b/content/glossary/images/15009032.png differ diff --git a/content/glossary/images/15009042.png b/content/glossary/images/15009042.png new file mode 100644 index 0000000..887d0f5 Binary files /dev/null and b/content/glossary/images/15009042.png differ diff --git a/content/glossary/images/15009050.png b/content/glossary/images/15009050.png new file mode 100644 index 0000000..ff228a4 Binary files /dev/null and b/content/glossary/images/15009050.png differ diff --git a/content/glossary/images/15009056.png b/content/glossary/images/15009056.png new file mode 100644 index 0000000..eb0afee Binary files /dev/null and b/content/glossary/images/15009056.png differ diff --git a/content/glossary/images/15009062.png b/content/glossary/images/15009062.png new file mode 100644 index 0000000..c7af138 Binary files /dev/null and b/content/glossary/images/15009062.png differ diff --git a/content/glossary/images/15009068.png b/content/glossary/images/15009068.png new file mode 100644 index 0000000..46f945e Binary files /dev/null and b/content/glossary/images/15009068.png differ diff --git a/content/glossary/images/15009074.png b/content/glossary/images/15009074.png new file mode 100644 index 0000000..9d4552b Binary files /dev/null and b/content/glossary/images/15009074.png differ diff --git a/content/glossary/images/15009084.png b/content/glossary/images/15009084.png new file mode 100644 index 0000000..f3cb5a7 Binary files /dev/null and b/content/glossary/images/15009084.png differ diff --git a/content/glossary/images/15009092.png b/content/glossary/images/15009092.png new file mode 100644 index 0000000..c2cecbe Binary files /dev/null and b/content/glossary/images/15009092.png differ diff --git a/content/glossary/images/15009098.png b/content/glossary/images/15009098.png new file mode 100644 index 0000000..16104f9 Binary files /dev/null and b/content/glossary/images/15009098.png differ diff --git a/content/glossary/images/15009104.png b/content/glossary/images/15009104.png new file mode 100644 index 0000000..bacc238 Binary files /dev/null and b/content/glossary/images/15009104.png differ diff --git a/content/glossary/images/15009110.png b/content/glossary/images/15009110.png new file mode 100644 index 0000000..61f1486 Binary files /dev/null and b/content/glossary/images/15009110.png differ diff --git a/content/glossary/images/15009116.png b/content/glossary/images/15009116.png new file mode 100644 index 0000000..d0e27e9 Binary files /dev/null and b/content/glossary/images/15009116.png differ diff --git a/content/glossary/images/15009122.png b/content/glossary/images/15009122.png new file mode 100644 index 0000000..2caf81a Binary files /dev/null and b/content/glossary/images/15009122.png differ diff --git a/content/glossary/images/15009128.png b/content/glossary/images/15009128.png new file mode 100644 index 0000000..4ec7ef3 Binary files /dev/null and b/content/glossary/images/15009128.png differ diff --git a/content/glossary/images/15009134.png b/content/glossary/images/15009134.png new file mode 100644 index 0000000..0cb8a72 Binary files /dev/null and b/content/glossary/images/15009134.png differ diff --git a/content/glossary/images/15009151.png b/content/glossary/images/15009151.png new file mode 100644 index 0000000..ac18e00 Binary files /dev/null and b/content/glossary/images/15009151.png differ diff --git a/content/glossary/images/15009159.png b/content/glossary/images/15009159.png new file mode 100644 index 0000000..f0201b9 Binary files /dev/null and b/content/glossary/images/15009159.png differ diff --git a/content/glossary/images/15009165.png b/content/glossary/images/15009165.png new file mode 100644 index 0000000..45affd2 Binary files /dev/null and b/content/glossary/images/15009165.png differ diff --git a/content/glossary/images/15009171.png b/content/glossary/images/15009171.png new file mode 100644 index 0000000..d9ac82b Binary files /dev/null and b/content/glossary/images/15009171.png differ diff --git a/content/glossary/images/15009177.png b/content/glossary/images/15009177.png new file mode 100644 index 0000000..377d128 Binary files /dev/null and b/content/glossary/images/15009177.png differ diff --git a/content/glossary/images/15009183.png b/content/glossary/images/15009183.png new file mode 100644 index 0000000..adacc6a Binary files /dev/null and b/content/glossary/images/15009183.png differ diff --git a/content/glossary/images/15009189.png b/content/glossary/images/15009189.png new file mode 100644 index 0000000..39a0e49 Binary files /dev/null and b/content/glossary/images/15009189.png differ diff --git a/content/glossary/images/15009199.png b/content/glossary/images/15009199.png new file mode 100644 index 0000000..16b15e7 Binary files /dev/null and b/content/glossary/images/15009199.png differ diff --git a/content/glossary/images/15009207.png b/content/glossary/images/15009207.png new file mode 100644 index 0000000..8343169 Binary files /dev/null and b/content/glossary/images/15009207.png differ diff --git a/content/glossary/images/15009213.png b/content/glossary/images/15009213.png new file mode 100644 index 0000000..47380cf Binary files /dev/null and b/content/glossary/images/15009213.png differ diff --git a/content/glossary/images/15009219.png b/content/glossary/images/15009219.png new file mode 100644 index 0000000..2d9a124 Binary files /dev/null and b/content/glossary/images/15009219.png differ diff --git a/content/glossary/images/15009225.png b/content/glossary/images/15009225.png new file mode 100644 index 0000000..6244f83 Binary files /dev/null and b/content/glossary/images/15009225.png differ diff --git a/content/glossary/images/15009231.png b/content/glossary/images/15009231.png new file mode 100644 index 0000000..cc9b694 Binary files /dev/null and b/content/glossary/images/15009231.png differ diff --git a/content/glossary/images/15009237.png b/content/glossary/images/15009237.png new file mode 100644 index 0000000..533ae79 Binary files /dev/null and b/content/glossary/images/15009237.png differ diff --git a/content/glossary/images/15009243.png b/content/glossary/images/15009243.png new file mode 100644 index 0000000..0c95c56 Binary files /dev/null and b/content/glossary/images/15009243.png differ diff --git a/content/glossary/images/15009253.png b/content/glossary/images/15009253.png new file mode 100644 index 0000000..4573409 Binary files /dev/null and b/content/glossary/images/15009253.png differ diff --git a/content/glossary/images/15138870.png b/content/glossary/images/15138870.png new file mode 100644 index 0000000..c7e9723 Binary files /dev/null and b/content/glossary/images/15138870.png differ diff --git a/content/glossary/images/15138919.png b/content/glossary/images/15138919.png new file mode 100644 index 0000000..f939641 Binary files /dev/null and b/content/glossary/images/15138919.png differ diff --git a/content/glossary/images/15138931.png b/content/glossary/images/15138931.png new file mode 100644 index 0000000..d9e3d3d Binary files /dev/null and b/content/glossary/images/15138931.png differ diff --git a/content/glossary/images/15138956.png b/content/glossary/images/15138956.png new file mode 100644 index 0000000..82cb674 Binary files /dev/null and b/content/glossary/images/15138956.png differ diff --git a/content/glossary/images/15204369.png b/content/glossary/images/15204369.png new file mode 100644 index 0000000..82cb674 Binary files /dev/null and b/content/glossary/images/15204369.png differ diff --git a/content/glossary/images/15237176.png b/content/glossary/images/15237176.png new file mode 100644 index 0000000..0904057 Binary files /dev/null and b/content/glossary/images/15237176.png differ diff --git a/content/glossary/images/15237186.png b/content/glossary/images/15237186.png new file mode 100644 index 0000000..b2c08ab Binary files /dev/null and b/content/glossary/images/15237186.png differ diff --git a/content/glossary/images/15237222.png b/content/glossary/images/15237222.png new file mode 100644 index 0000000..a596ae1 Binary files /dev/null and b/content/glossary/images/15237222.png differ diff --git a/content/glossary/images/15302712.png b/content/glossary/images/15302712.png new file mode 100644 index 0000000..3d8f632 Binary files /dev/null and b/content/glossary/images/15302712.png differ diff --git a/content/glossary/images/15302756.png b/content/glossary/images/15302756.png new file mode 100644 index 0000000..82cb674 Binary files /dev/null and b/content/glossary/images/15302756.png differ diff --git a/content/glossary/images/15302771.png b/content/glossary/images/15302771.png new file mode 100644 index 0000000..d9e3d3d Binary files /dev/null and b/content/glossary/images/15302771.png differ diff --git a/content/glossary/images/56033307.png b/content/glossary/images/56033307.png new file mode 100644 index 0000000..87888d0 Binary files /dev/null and b/content/glossary/images/56033307.png differ diff --git a/content/glossary/index.md b/content/glossary/index.md index b168c47..2dc3b65 100644 --- a/content/glossary/index.md +++ b/content/glossary/index.md @@ -9,15 +9,43 @@ Terms used across telecoms, networking, embedded and IoT engineering, software a Each entry starts with a short definition. Acronyms give their expansion first, in bold. A sentence or two of detail follows where it helps, then links to related terms. The entries are deliberately short: they say what a term means and how it fits with the others, and leave the full story to the standards and books. -| Section | Covers, for example | +## Browse + +[A](a.md) · [B](b.md) · [C](c.md) · [D](d.md) · [E](e.md) · [F](f.md) · [G](g.md) · [H](h.md) · [I](i.md) · [J](j.md) · [K](k.md) · [L](l.md) · [M](m.md) · [N](n.md) · [O](o.md) · [P](p.md) · [Q](q.md) · [R](r.md) · [S](s.md) · [T](t.md) · [U](u.md) · [V](v.md) · [W](w.md) · [X](x.md) · [Y](y.md) · [Z](z.md) · [Numbers](numbers.md) · [Symbols](symbols.md) + +There is one page per letter. **Numbers** holds the terms that start with a digit, such as 5G and 802.11, and **Symbols** the ones that start with another character, such as .NET. + +| Page | Terms | | --- | --- | -| [A to C](a-c.md) | ATM, BGP, Bluetooth, C and C++, CDMA, change control, continuous integration | -| [D to F](d-f.md) | DNS and its records, DSL, DTLS, duplex, estimation, Ethernet, FOTA | -| [G to K](g-k.md) | G.711 and other codecs, GSM, HTTP, IMS, IPv4 and IPv6, ISDN, Kanban | -| [L to N](l-n.md) | L2TP, LTE, LwM2M, MPLS VPNs, MQTT, multicast, NAT | -| [O to R](o-r.md) | OSPF, PCM, PPP, QoS, REST, risk management, RTOS, RTP | -| [S to Z and numbers](s-z.md) | Scrum, SIP, SS7, TCP, TLS, UML, VLANs, Wi-Fi, 5G, 802.11 | - -[[CAPTION:Table]] The sections of the glossary. +| [A](a.md) | 53 | +| [B](b.md) | 46 | +| [C](c.md) | 91 | +| [D](d.md) | 78 | +| [E](e.md) | 42 | +| [F](f.md) | 30 | +| [G](g.md) | 33 | +| [H](h.md) | 24 | +| [I](i.md) | 70 | +| [J](j.md) | 7 | +| [K](k.md) | 12 | +| [L](l.md) | 39 | +| [M](m.md) | 69 | +| [N](n.md) | 27 | +| [O](o.md) | 16 | +| [P](p.md) | 79 | +| [Q](q.md) | 11 | +| [R](r.md) | 59 | +| [S](s.md) | 119 | +| [T](t.md) | 43 | +| [U](u.md) | 32 | +| [V](v.md) | 28 | +| [W](w.md) | 35 | +| [X](x.md) | 9 | +| [Y](y.md) | 1 | +| [Z](z.md) | 1 | +| [Numbers](numbers.md) | 5 | +| [Symbols](symbols.md) | 1 | + +[[CAPTION:Table]] The glossary pages. Search finds any term; on the sites, press `/` or `Ctrl+K`. diff --git a/content/glossary/j.md b/content/glossary/j.md new file mode 100644 index 0000000..5fb82a6 --- /dev/null +++ b/content/glossary/j.md @@ -0,0 +1,52 @@ +--- +title: J +--- + +# J + +## Java +A class-based, object-oriented language released by Sun Microsystems in 1995. It compiles to bytecode that runs on the Java virtual machine, so the same code runs on any platform that has one. + +See also: [JavaScript](#javascript), [C#](c.md#c_2). + +## JavaScript +The scripting language of the web, run by every browser and, through Node.js, on servers. It is dynamic and supports object-oriented, functional and event-driven styles. It is standardised as ECMAScript. + +See also: [JSON](#json), [Java](#java). + +## Jitter +Variation in the timing of a signal or in the delay of packets. In packet networks it is also called packet delay variation, and it affects real-time traffic such as voice and video. + +![Jitter](images/15008430.png) + +[[CAPTION:Figure]] Jitter. + +![Jitter](images/15008438.png) + +[[CAPTION:Figure]] Jitter. + +See also: [Jitter buffer](#jitter-buffer), [Latency](l.md#latency). + +## Jitter buffer +A buffer at a receiver that holds arriving voice or video packets for a short time, so they can be played out at an even rate despite jitter. Adaptive buffers change their size to match conditions, trading delay against loss. + +![Jitter buffer](images/15008430.png) + +[[CAPTION:Figure]] Jitter buffer. + +See also: [Jitter](#jitter), [RTP](r.md#rtp). + +## Joint Application Development +**JAD.** A facilitated workshop in which customer representatives and developers draft requirements together. + +See also: [RAD](r.md#rad). + +## JSON +**JavaScript Object Notation.** A lightweight text format for data made of objects (name and value pairs) and arrays, defined in RFC 8259. It is the usual format for web APIs. + +See also: [JSON-RPC](#json-rpc), [YAML](y.md#yaml), [XML](x.md#xml), [CBOR](c.md#cbor). + +## JSON-RPC +A simple remote procedure call protocol that encodes calls and results in JSON. + +See also: [XML-RPC](x.md#xml-rpc), [RPC](r.md#rpc). diff --git a/content/glossary/k.md b/content/glossary/k.md new file mode 100644 index 0000000..1dd31f9 --- /dev/null +++ b/content/glossary/k.md @@ -0,0 +1,73 @@ +--- +title: K +--- + +# K + +## K & R {#k-r} +**Kernighan and Ritchie.** Brian Kernighan and Dennis Ritchie of Bell Labs, authors of *The C Programming Language* (1978). Ritchie created C and, with Ken Thompson, Unix. + +See also: [K & R C](#k-r-c), [C Language](c.md#c-language). + +## K & R C {#k-r-c} +The version of C described in the first edition of Kernighan and Ritchie's book, which served as its informal specification before the ANSI standard of 1989. + +See also: [K & R](#k-r), [C Language](c.md#c-language). + +## K & R indent style {#k-r-indent-style} +The brace style used in Kernighan and Ritchie's book. A function's opening brace goes on its own line, while blocks inside the function open on the same line as their control statement. + +See also: [Construction Standard](c.md#construction-standard). + +## Kanban +A method for managing knowledge work by making the workflow visible, limiting work in progress and improving the flow step by step. + +See also: [Kanban Board](#kanban-board), [Lean](l.md#lean). + +## Kanban Board +A board with a column for each stage of a workflow, on which cards representing work items move from left to right. + +![Kanban Board](images/15008447.png) + +[[CAPTION:Figure]] Kanban Board. + +See also: [Kanban](#kanban), [Task Board](t.md#task-board). + +## KDL +A document language with XML-like semantics and a syntax that looks like a series of commands. It is used for configuration and data, like JSON, YAML or XML. + +See also: [YAML](y.md#yaml), [TOML](t.md#toml). + +## Kerberos +A network authentication protocol, developed at MIT, in which a trusted key distribution centre issues time-limited tickets. Clients and services then prove their identities to each other without sending passwords. Windows Active Directory uses it. + +See also: [AD](a.md#ad), [LDAP](l.md#ldap). + +## Kernel +The core of an operating system, which runs with full control of the hardware. It manages processes, memory, devices and system calls, and keeps applications apart from each other. + +![Kernel](images/15008455.png) + +[[CAPTION:Figure]] Kernel. + +See also: [Linux](l.md#linux), [Driver](d.md#driver). + +## KISS Principle +**Keep it simple, stupid.** The principle that most systems work best when kept simple, and that unnecessary complexity should be avoided. + +See also: [Rule of Simplicity](r.md#rule-of-simplicity), [Simple Design](s.md#simple-design). + +## Knowledge Management +**KM.** Capturing, sharing and reusing an organisation's knowledge so that people can find what they need. + +See also: [Wiki](w.md#wiki). + +## KPI +**Key Performance Indicator.** A measure of how well an organisation or activity is meeting an important goal. + +See also: [Metric](m.md#metric), [SLA](s.md#sla). + +## Ksh +The KornShell, a Unix shell written by David Korn at Bell Labs in the early 1980s. It is compatible with the Bourne shell and adds features from the C shell. + +See also: [Sh](s.md#sh), [Bash](b.md#bash), [Csh](c.md#csh). diff --git a/content/glossary/l-n.md b/content/glossary/l-n.md deleted file mode 100644 index bbc1c6d..0000000 --- a/content/glossary/l-n.md +++ /dev/null @@ -1,685 +0,0 @@ ---- -title: Glossary, L to N ---- - -# L to N - -## L - -### L1 -**Layer 1, the physical layer.** The lowest layer of the OSI model, which sends raw bits over a medium such as copper, fibre or radio. It defines the connectors, signals, timing and encoding. - -See also: [L2](#l2), [Hub](g-k.md#hub). - -### L2 -**Layer 2, the data link layer.** The OSI layer that moves frames between devices on the same link or LAN segment, with local (MAC) addressing and error detection. Ethernet and Wi-Fi work here; switches forward traffic at this layer. - -See also: [L1](#l1), [MAC](#mac), [Switch](s-z.md#switch), [DLP](d-f.md#dlp). - -### L2F -**Layer 2 Forwarding.** A Cisco tunnelling protocol for carrying PPP sessions across an IP network. It has no encryption of its own, and L2TP replaced it. - -See also: [L2TP](#l2tp), [PPTP](o-r.md#pptp). - -### L2TP -**Layer 2 Tunnelling Protocol.** A protocol that tunnels PPP sessions over an IP network, used by ISPs to hand broadband sessions between networks and to build VPNs. It does not encrypt, so it is often run inside IPsec. - -See also: [LAC](#lac), [LNS](#lns), [LTS](#lts), [IPSec](g-k.md#ipsec). - -### L2VPN -**Layer 2 VPN.** A provider service that carries a customer's Ethernet or other layer-2 traffic between sites over the provider's MPLS network. The sites then appear to share one LAN or wire. Point-to-point and multipoint (VPLS) forms exist. - -See also: [VPLS](s-z.md#vpls), [VPWS](s-z.md#vpws), [L3VPN](#l3vpn), [PW](o-r.md#pw). - -### L3VPN -**Layer 3 VPN.** A provider service, usually MPLS-based as in RFC 4364, that routes a customer's IP traffic between sites. The provider edge routers keep each customer's routes in a separate VRF; customer edge routers connect to them, and provider core routers just switch labels. - -See also: [VRF](s-z.md#vrf), [CE](a-c.md#ce), [L2VPN](#l2vpn), [LSP](#lsp). - -### LAC -**L2TP Access Concentrator.** The device at one end of an L2TP tunnel that receives a user's PPP session and forwards it through the tunnel to the LNS. - -See also: [L2TP](#l2tp), [LNS](#lns). - -### LACP -**Link Aggregation Control Protocol.** The IEEE 802.1AX (formerly 802.3ad) protocol with which two devices agree to bundle several physical links into one logical link and check that each member is working. - -See also: [LAG](#lag). - -### LAG -**Link Aggregation Group.** Several physical links bundled into one logical link, to add capacity and survive the failure of a member. LACP usually manages it. - -See also: [LACP](#lacp). - -### LAN -**Local Area Network.** A network that connects devices within a limited area, such as a home, office or campus, usually with Ethernet and Wi-Fi. - -See also: [WAN](s-z.md#wan), [MAN](#man). - -### LANE -**LAN Emulation.** An ATM Forum standard that made an ATM network look like an Ethernet or Token Ring LAN to the devices attached to it. - -See also: [ATM](a-c.md#atm). - -### Latency -The delay between something being sent and it arriving. For telephone calls it is the mouth-to-ear delay, made up of codec, network and playout delays. ITU-T G.114 advises keeping one-way delay under about 150 ms for good conversation. - -See also: [Jitter](g-k.md#jitter), [Jitter buffer](g-k.md#jitter-buffer), [Bandwidth](a-c.md#bandwidth). - -### Lawful intercept -**LI.** Facilities in telecoms networks that let authorised agencies, with a warrant, intercept a named target's communications. Operators must provide them, through standardised interfaces. It differs from bulk surveillance in being targeted at individuals. - -### LCR -**Least Cost Routing.** Choosing the route for each outgoing call by cost, across several carriers. Carriers revise their tables often, and PBXs can do the same for a business. - -See also: [CPS](a-c.md#cps). - -### LDAP -**Lightweight Directory Access Protocol.** An open protocol, defined in RFC 4511, for reading and updating directory services, such as a company's directory of users and groups. Active Directory supports it. - -See also: [AD](a-c.md#ad), [Kerberos](g-k.md#kerberos). - -### Lead Time -The time from a customer's request to its delivery. In software, often the time from a requirement being raised to it being met in production. - -See also: [Kanban](g-k.md#kanban). - -### Lean -A way of working, from Toyota's production system, that aims to deliver value to the customer by removing waste: anything that does not add value. - -See also: [Lean programming](#lean-programming), [Kanban](g-k.md#kanban). - -### Lean programming -Lean ideas applied to software development: removing waste, delivering fast, deciding as late as possible and building quality in. - -See also: [Lean](#lean). - -### Leased Line -A dedicated, symmetrical circuit between two points, rented for a fixed fee and always on. In the UK it is also called a private circuit. Businesses use leased lines to link sites and for internet access. - -See also: [EAD](d-f.md#ead), [Dark Fibre](d-f.md#dark-fibre), [CSI](a-c.md#csi). - -### LeSS -**Large-Scale Scrum.** A framework for scaling Scrum to many teams working on one product, with one product owner and one product backlog. - -See also: [Scrum](s-z.md#scrum), [SAFe](s-z.md#safe), [DAD](d-f.md#dad). - -### Lifecycle -A model of the activities from the first idea for a system to its retirement, usually divided into phases. Projects choose and tailor a lifecycle to suit their risks. - -See also: [Waterfall](s-z.md#waterfall), [Spiral Lifecycle](s-z.md#spiral-lifecycle), [SDLC](s-z.md#sdlc), [Phase](o-r.md#phase). - -### Lines of Code -**LOC.** A measure of software size as the number of source lines, usually counting neither comments nor blank lines. KLOC means thousands of lines. - -See also: [Function Point](d-f.md#function-point). - -### Linux -A family of free, open-source operating systems built on the Linux kernel, first released by Linus Torvalds in 1991. It is packaged in distributions, such as Debian, Ubuntu, Fedora and Red Hat Enterprise Linux. It runs most servers and embedded devices, and underlies Android. - -See also: [GNU](g-k.md#gnu), [Unix](s-z.md#unix), [Kernel](g-k.md#kernel), [Android](a-c.md#android). - -### Lisp -A family of languages, first specified in 1958, with a fully bracketed prefix syntax in which code and data share the same form. Common Lisp, Scheme and Clojure are its main dialects. Lisp introduced many ideas now common, such as garbage collection and the read-eval-print loop. - -### LL -See [Local Loop](#local-loop). - -### LNS -**L2TP Network Server.** The device at the far end of an L2TP tunnel from the LAC, which terminates the users' PPP sessions, typically at the ISP. - -See also: [L2TP](#l2tp), [LAC](#lac), [LTS](#lts). - -### Local Build -A build done in a developer's local development environment. - -See also: [Project Build](o-r.md#project-build), [Build](a-c.md#build). - -### Local Development Environment -**LDE.** A developer's own development environment, usually on their own machine. - -See also: [Development Environment](d-f.md#development-environment), [Build Environment](a-c.md#build-environment). - -### Local Loop -The line that connects a customer's premises to the operator's network, traditionally a copper pair to the local exchange. Also called the last mile or subscriber line. - -See also: [MPF](#mpf), [MDF](#mdf), [DP](d-f.md#dp). - -### Local Test Environment -The part of the local development environment used for construction testing. - -See also: [Local Development Environment](#local-development-environment). - -### LoFi Prototype -See [Low Fidelity Prototype](#low-fidelity-prototype). - -### Low Fidelity Prototype -A rough mock-up of a system made with paper, sticky notes or simple drawings, used to explore ideas cheaply before building anything. - -See also: [High Fidelity Prototype](g-k.md#high-fidelity-prototype), [Storyboard](s-z.md#storyboard). - -### Low Level Design -**LLD.** The design level between high-level design and construction, detailed enough to build from directly. - -See also: [High Level Design](g-k.md#high-level-design), [Detailed Design](d-f.md#detailed-design). - -### LSP -**Label Switched Path.** A path through an MPLS network, set up by a protocol such as LDP or RSVP-TE. The first router adds a label, each router along the way swaps it, and the last removes it. - -See also: [LSR](#lsr), [RSVP](o-r.md#rsvp), [L3VPN](#l3vpn). - -### LSR -**Label Switch Router.** A router in the middle of an MPLS network that forwards packets by their labels alone. - -See also: [LSP](#lsp). - -### LTE (4G) -**Long-Term Evolution.** The 3GPP mobile standard (Release 8 onwards) that followed UMTS, with an OFDMA radio interface and an all-IP core, the Evolved Packet Core. LTE-Advanced meets the ITU's 4G requirements. Voice runs over IP as VoLTE. - -See also: [E-UTRAN](d-f.md#e-utran), [E NodeB](d-f.md#e-nodeb), [MME](#mme), [S-GW](s-z.md#s-gw), [PGW](o-r.md#pgw), [VoLTE](s-z.md#volte). - -### LTS -**L2TP Tunnel Switch.** A router between a LAC and an LNS that acts as both. It ends incoming L2TP tunnels and forwards the sessions through new tunnels to other LNSs, for example to different ISPs. - -See also: [L2TP](#l2tp), [LAC](#lac), [LNS](#lns). - -### Lua -A small, fast scripting language from Brazil, designed in 1993 to be embedded in applications through a simple C API. It is used widely in games and network equipment. - -### LwM2M -**Lightweight Machine to Machine.** The OMA SpecWorks protocol for managing IoT devices and their data over CoAP, usually secured with DTLS. Devices expose standard objects, such as device information, connectivity and firmware update, so servers can manage different makers' devices in the same way. - -See also: [M2M](#m2m), [DTLS](d-f.md#dtls), [CBOR](a-c.md#cbor), [FOTA](d-f.md#fota). - -## M - -### M2M -**Machine to Machine.** Direct communication between devices, such as meters, sensors and vehicles, and the systems that use their data, without people in the loop. It was the forerunner of the Internet of Things. - -See also: [LwM2M](#lwm2m), [MQTT](#mqtt). - -### MAC -**Medium Access Control.** The lower sublayer of the data link layer in IEEE 802 networks. It controls access to the shared medium and handles framing and hardware addressing; the logical link control (LLC) sublayer sits above it. - -See also: [MAC Address](#mac-address), [L2](#l2), [CSMA](a-c.md#csma). - -### MAC Address -**Media Access Control address.** The 48-bit hardware address of a network interface on Ethernet, Wi-Fi and other IEEE 802 networks. It is written as six hex pairs, such as `00:1A:2B:3C:4D:5E`. The first half normally identifies the manufacturer. - -See also: [MAC](#mac), [MAC Spoofing](#mac-spoofing), [ARP spoofing](a-c.md#arp-spoofing). - -### MAC Flooding -An attack that fills a switch's address table with fake MAC addresses. The switch then floods frames out of every port, letting the attacker see traffic meant for others. - -See also: [Switch](s-z.md#switch), [MAC Spoofing](#mac-spoofing). - -### MAC Spoofing -Changing the MAC address a network interface presents, usually through its driver, to pose as another device or hide its identity. - -See also: [MAC Address](#mac-address), [ARP spoofing](a-c.md#arp-spoofing). - -### Machine code -Instructions in the binary form a processor runs directly. People rarely write it by hand: compilers and assemblers produce it. - -See also: [Compiler](a-c.md#compiler), [Compiled Language](a-c.md#compiled-language). - -### Macro Estimation -Estimating something as a whole. See [Top-Down Estimation](s-z.md#top-down-estimation). - -### Maintenance -Software maintenance: correcting, updating, adapting and operating software after its first release. - -See also: [Maintenance Plan](#maintenance-plan). - -### Maintenance Plan -A plan for maintaining a system after its first release. - -### MAN -**Metropolitan Area Network.** A network that covers a city or region: larger than a LAN, smaller than a WAN, and often linking several LANs. - -See also: [LAN](#lan), [WAN](s-z.md#wan). - -### Management by Walking Around -**MBWA.** Managing through frequent, informal contact with the team, alongside formal tracking, to spot risks and problems early. - -### Management Plane -The part of a network device or network that configures, monitors and manages it, through interfaces such as the CLI, SNMP or NETCONF. - -See also: [Control Plane](a-c.md#control-plane), [Data Plane](d-f.md#data-plane), [SNMP](s-z.md#snmp). - -### MANET -**Mobile Ad hoc Network.** A self-configuring wireless network of mobile devices with no fixed infrastructure, in which every device also routes traffic for the others. - -See also: [WANET](s-z.md#wanet), [Wireless Mesh Network](s-z.md#wireless-mesh-network), [Geocast](g-k.md#geocast). - -### Manual Testing -Testing in which a person runs the test cases and records the results. - -See also: [Automated Testing](a-c.md#automated-testing). - -### MAP -**Mobile Application Part.** The SS7 protocol with which GSM and UMTS core network nodes, such as the HLR, VLR, MSC and SGSN, exchange information. It covers location updates, authentication and SMS delivery. - -See also: [SS7](s-z.md#ss7), [HLR](g-k.md#hlr), [VLR](s-z.md#vlr), [TCAP](s-z.md#tcap). - -### MBMS -**Multimedia Broadcast Multicast Services.** 3GPP standards for broadcasting and multicasting content to many mobile devices at once. In LTE it is called eMBMS or LTE Broadcast. - -See also: [Multicast](#multicast), [LTE (4G)](#lte-4g). - -### MBORC -**Matters Beyond Our Reasonable Control.** A clause in Openreach contracts that relieves Openreach of its service-level obligations when an outside event, such as severe weather or third-party damage, prevents it from meeting them. - -### MCU -**Microcontroller Unit.** A single chip with a processor core, flash and RAM, and peripherals, used in small embedded devices. It usually runs bare-metal code or an RTOS. - -See also: [MPU](#mpu), [RTOS](o-r.md#rtos), [Embedded systems programming](d-f.md#embedded-systems-programming). - -### MD5 -**Message Digest 5.** A 128-bit hash function from 1991. It is broken for security uses because collisions are easy to make, so it should be used only as a checksum against accidental corruption. - -See also: [SHA](s-z.md#sha), [Hash Function](g-k.md#hash-function). - -### MDF -**Main Distribution Frame.** The frame in a telephone exchange where the external cables to customers are connected to the exchange's equipment by jumpers. - -See also: [HDF](g-k.md#hdf), [Local Loop](#local-loop), [MPF](#mpf). - -### Mediation -In telecoms, the systems that collect usage records, such as CDRs, from network elements, then clean, combine and convert them into the formats that billing, fraud and other systems need. - -See also: [CDR](a-c.md#cdr), [OSS](o-r.md#oss). - -### MELP -**Mixed-Excitation Linear Prediction.** A US military speech codec standard. Its enhanced version, MELPe, works at 2,400, 1,200 or 600 bit/s for secure and narrow-band radio links. - -See also: [CVSD](a-c.md#cvsd), [Audio Codecs](a-c.md#audio-codecs). - -### Metric -A measurement taken from a project, process or product to judge whether it has the qualities wanted. - -See also: [KPI](g-k.md#kpi). - -### MGCF -**Media Gateway Control Function.** In IMS, the function that converts call signalling between SIP and the circuit-switched network's ISUP or BICC, and controls the media gateway through H.248. - -See also: [MGW](#mgw), [IMS](g-k.md#ims), [BGCF](a-c.md#bgcf). - -### MGCP -**Media Gateway Control Protocol.** A VoIP protocol in which a central call agent controls simple media gateways and endpoints. It is an IETF informational RFC; H.248 is the formal standard for the same role. - -See also: [H.248](g-k.md#h248), [Softswitch](s-z.md#softswitch). - -### MGW -**Media Gateway.** A device that converts media between networks, typically between PCM on telephone circuits and RTP streams on IP. It also transcodes when the two sides use different codecs. - -See also: [MGCF](#mgcf), [H.248](g-k.md#h248), [RTP](o-r.md#rtp). - -### MIB -**Management Information Base.** The tree-structured set of objects that a device exposes for management, mostly through SNMP. Each object is identified by an object identifier (OID). - -See also: [SNMP](s-z.md#snmp). - -### Micro Estimation -Estimating the details of something. See [Bottom-Up Estimation](a-c.md#bottom-up-estimation). - -### Microservice -An architecture style in which an application is a set of small services, each built around a business capability. Each can be deployed independently and communicates through lightweight APIs. It suits large systems with many teams, at the cost of operational complexity. - -See also: [SOA](s-z.md#soa), [BFF](a-c.md#bff), [Docker](d-f.md#docker). - -### Microstone -See [Miniature Milestone](#miniature-milestone). - -### Mil-COTS -COTS products for military use. See [COTS](a-c.md#cots). - -### Milestone -A significant point in a schedule, such as an achievement or a date. - -See also: [Miniature Milestone](#miniature-milestone), [Business Schedule](a-c.md#business-schedule). - -### Milestone Release -A release made for a milestone in the project plan. - -### Milestone Retrospective -A detailed review by the team of a project's significant events, after a major milestone or at the end of the project. - -See also: [Postmortem](o-r.md#postmortem), [Sprint retrospective](s-z.md#sprint-retrospective). - -### Milestone Schedule -A business schedule that sets out the project's formal milestones. - -See also: [Business Schedule](a-c.md#business-schedule). - -### MIMO -**Multiple-Input, Multiple-Output.** Using several antennas at both ends of a radio link to send several data streams at once over the same channel, exploiting multipath propagation. Wi-Fi, LTE and 5G all rely on it. - -See also: [SISO](s-z.md#siso), [SDMA](s-z.md#sdma), [Wi-Fi](s-z.md#wi-fi). - -### Miniature Milestone -A small milestone, a day or two of work, that counts as either done or not done, never partly done. It makes progress easy to track. - -See also: [Milestone](#milestone), [Inch-Pebble](g-k.md#inch-pebble). - -### MME -**Mobility Management Entity.** The main control node of the LTE core. It authenticates users with the HSS, tracks and pages idle devices, chooses gateways and manages bearers and handovers. - -See also: [HSS](g-k.md#hss), [S-GW](s-z.md#s-gw), [E NodeB](d-f.md#e-nodeb), [LTE (4G)](#lte-4g). - -### MMF -**Minimum Marketable Feature.** The smallest feature that is worth releasing on its own because it gives users real value. - -See also: [MVP](#mvp). - -### MNC -**Mobile Network Code.** The two- or three-digit code that, with the mobile country code (MCC), identifies a mobile network. Both form the start of an IMSI. - -See also: [IMSI](g-k.md#imsi). - -### Mob Programming -The whole team working on the same thing, at the same time, in the same place, at one computer. - -See also: [Pair Programming](o-r.md#pair-programming). - -### Mobile Cell -The area served by one base station in a cellular network. Splitting coverage into cells lets the same frequencies be reused in cells further apart. Cell sizes range from a few metres to tens of kilometres. - -See also: [BTS](a-c.md#bts), [FemtoCell](d-f.md#femtocell). - -### Mock Objects -Stand-ins for real components, created for a test, that imitate the real behaviour and check how the code under test uses them. - -See also: [Unit Testing](s-z.md#unit-testing), [TDD](s-z.md#tdd). - -### Model -A simplified representation of a system or process, often as diagrams, that describes its important characteristics. - -See also: [Modelling](#modelling), [UML](s-z.md#uml). - -### Modelling -Using models in requirements and design work. - -See also: [Model](#model). - -### Moderator -The person who leads an inspection and makes sure it follows the process. - -See also: [Inspection](g-k.md#inspection). - -### MOS -**Mean Opinion Score.** A rating of perceived quality, especially of voice calls, on a scale from 1 (bad) to 5 (excellent). It is either the average of listeners' scores or estimated by an algorithm. ITU-T P.800 defines it. - -| MOS | Quality | Impairment | -| --- | --- | --- | -| 5 | Excellent | Imperceptible | -| 4 | Good | Perceptible but not annoying | -| 3 | Fair | Slightly annoying | -| 2 | Poor | Annoying | -| 1 | Bad | Very annoying | - -[[CAPTION:Table]] The MOS scale. - -See also: [Latency](#latency), [Jitter](g-k.md#jitter). - -### MP3 -**MPEG-1 or MPEG-2 Audio Layer III.** A lossy audio coding format that removes sounds people are unlikely to hear. It made music files small enough to share and became the most common format for digital music. - -See also: [AAC](a-c.md#aac), [MPEG](#mpeg). - -### MPEG -**Moving Picture Experts Group.** The ISO and IEC working group that sets standards for audio and video coding, such as MPEG-2 and MPEG-4. - -See also: [MPEG4](#mpeg4), [MP3](#mp3). - -### MPEG4 -**MPEG-4.** A family of ISO/IEC standards (14496) for coding audio and video. It includes AAC audio and the H.264 (AVC) video codec, widely used for streaming and broadcast. - -See also: [MPEG](#mpeg), [AAC](a-c.md#aac). - -### MPF -**Metallic Path Facility.** An Openreach product that gives another operator a whole copper line from the exchange to the customer, so the operator provides both voice and broadband. - -See also: [SMPF](s-z.md#smpf), [Local Loop](#local-loop), [MDF](#mdf). - -### MPU -**Microprocessor Unit.** A processor that relies on external memory and usually has a memory management unit, so it can run a full operating system such as Linux. - -See also: [MCU](#mcu). - -### MQTT -A lightweight publish-subscribe messaging protocol, over TCP, in which clients publish and subscribe to topics through a broker. It was designed in 1999 for low-bandwidth links, is an OASIS standard, and is widely used in IoT. - -See also: [M2M](#m2m), [LwM2M](#lwm2m). - -### MRF -**Media Resource Function.** In IMS, the function that plays announcements and tones, mixes conference audio and does other media processing. - -See also: [IMS](g-k.md#ims). - -### MRS -**Multiple Residence data.** Royal Mail address data that lists the separate homes, such as flats, behind a single delivery point. - -### MRTIE -**Maximum Relative Time Interval Error.** A measure of clock stability in synchronised networks: the largest change in a clock's time error against a reference over a given observation interval. - -See also: [PTP](o-r.md#ptp), [Jitter](g-k.md#jitter). - -### MSAN -**Multi-Service Access Node.** Equipment in an exchange or cabinet that connects customers' lines to the core network and provides voice, ISDN and broadband from one platform. It replaces separate exchange line cards and DSLAMs. - -See also: [DSLAM](d-f.md#dslam), [BT 21CN](a-c.md#bt-21cn), [NGN](#ngn). - -### MSC -**Mobile Switching Centre.** The switch in a GSM or UMTS core network that sets up and routes voice calls and SMS, handles handovers between areas and produces charging records. - -See also: [GMSC](g-k.md#gmsc), [HLR](g-k.md#hlr), [VLR](s-z.md#vlr), [BSC](a-c.md#bsc). - -### MTP -**Message Transfer Part.** The lower levels (1 to 3) of SS7, which carry signalling messages reliably and in order between signalling points. - -See also: [SS7](s-z.md#ss7), [SCCP](s-z.md#sccp). - -### MTR -**Mean Time to Repair (MTTR).** The average time taken to restore failed equipment or a service to working order. - -### MTU -**Maximum Transmission Unit.** The largest packet a network link can carry without fragmentation, 1,500 bytes on standard Ethernet. Tunnels add headers, so they reduce the usable MTU. - -See also: [PDU](o-r.md#pdu). - -### Mu-law -**µ-law.** A companding algorithm, one of the two variants of ITU-T G.711, used for 8-bit PCM speech in North America and Japan. It is also written u-law. - -See also: [A-law](a-c.md#a-law), [G.711](g-k.md#g711). - -### Multicast -Sending data to a group of receivers at once, with the network copying packets only where the paths divide. IPTV uses it, with receivers joining groups through IGMP. - -```mermaid -flowchart LR - S((Sender)) --> R1[Router] - R1 --> R2[Router] - R1 --> R3[Router] - R2 --> A((Member)) - R2 --> B((Member)) - R3 --> C((Member)) -``` - -[[CAPTION:Figure]] Multicast: one stream, copied only where the paths divide. - -See also: [Unicast](s-z.md#unicast), [Broadcast](a-c.md#broadcast), [Anycast](a-c.md#anycast), [IGMP](g-k.md#igmp). - -### Multiplex -Combining several signals or data streams onto one shared medium, by time, frequency, wavelength or code, and separating them again at the far end. - -See also: [TDM](s-z.md#tdm), [OFDMA](o-r.md#ofdma), [CDMA](a-c.md#cdma). - -### Mutex -A lock that lets only one thread at a time hold a shared resource, so threads cannot corrupt it by using it at once. - -See also: [Mutual Exclusion](#mutual-exclusion), [Semaphore](s-z.md#semaphore). - -### Mutual Exclusion -The requirement that only one thread at a time runs a critical section that uses a shared resource, to prevent race conditions. Edsger Dijkstra first described and solved the problem in 1965. - -See also: [Mutex](#mutex), [Semaphore](s-z.md#semaphore). - -### MVNE -**Mobile Virtual Network Enabler.** A company that provides MVNOs with the systems they need, such as billing, provisioning and network integration, without dealing with consumers itself. - -See also: [MVNO](#mvno). - -### MVNO -**Mobile Virtual Network Operator.** A mobile operator that sells services under its own brand but does not own the radio network. It buys capacity wholesale from a network operator. - -See also: [MVNE](#mvne). - -### MVP -**Minimum Viable Product.** The first version of a product built to learn as much as possible about customers with the least effort, as described by Eric Ries. - -See also: [MMF](#mmf), [Customer Development](a-c.md#customer-development). - -## N - -### NaaS -**Network as a Service.** Network connectivity, such as VPNs and bandwidth on demand, offered as a cloud-style service, often built on network virtualisation. - -See also: [IaaS](g-k.md#iaas), [SDN](s-z.md#sdn). - -### NAPT -**Network Address and Port Translation.** NAT that also rewrites port numbers, so that many private hosts can share one public IP address. It is also called PAT or IP masquerading. - -See also: [NAT](#nat). - -### NAS -**Network Access Server.** A device that gives users access to a network, such as a dial-up or broadband access server. It usually authenticates them through RADIUS. In LTE, NAS also stands for the non-access stratum: the signalling between the device and the core. - -See also: [Radius](o-r.md#radius), [BRAS](a-c.md#bras). - -### NAT -**Network Address Translation.** Rewriting IP addresses in packets as they pass through a router, usually so that a private network can share public addresses. It has kept IPv4 going despite address exhaustion, but it complicates peer-to-peer protocols. - -See also: [NAPT](#napt), [STUN](s-z.md#stun), [ICE](g-k.md#ice), [IPv4](g-k.md#ipv4). - -### Native Risk Management -See [Intrinsic Risk Management](g-k.md#intrinsic-risk-management). - -### NB-IoT -**Narrowband IoT.** A 3GPP low-power, wide-area radio technology for simple IoT devices, such as meters, that send little data and must run for years on a battery. - -See also: [LTE (4G)](#lte-4g), [LwM2M](#lwm2m). - -### NBAP -**Node B Application Part.** The UMTS signalling protocol on the Iub interface with which the RNC controls a Node B, setting up cells and radio links. - -See also: [Node B](#node-b), [RNC](o-r.md#rnc), [UTRAN](s-z.md#utran). - -### NCSC -**National Cyber Security Centre.** The UK's technical authority on cyber security, part of GCHQ, which advises government, businesses and the public. It opened in 2016. - -See also: [GCHQ](g-k.md#gchq), [CESG](a-c.md#cesg). - -### Network Topologies -The arrangement of a network's nodes and links, such as bus, star, ring, mesh or tree. The physical topology is how it is cabled; the logical topology is how data flows. - -```mermaid -flowchart LR - subgraph Star - H((Hub)) --- s1((A)) - H --- s2((B)) - H --- s3((C)) - end - subgraph Ring - r1((A)) --- r2((B)) --- r3((C)) --- r1 - end - subgraph Mesh - m1((A)) --- m2((B)) - m2 --- m3((C)) - m3 --- m1 - end -``` - -[[CAPTION:Figure]] Star, ring and mesh topologies. - -### NFS -**Network File System.** A protocol, first developed by Sun in 1984, that lets a computer use files on a remote server as if they were local. - -### NGN -**Next-Generation Network.** A packet-based network, usually all-IP, that carries every service, voice included, with service control separate from transport. ITU-T and ETSI TISPAN defined NGN architectures in the 2000s, and IMS became their service layer. - -See also: [IMS](g-k.md#ims), [BT 21CN](a-c.md#bt-21cn), [Softswitch](s-z.md#softswitch). - -### NGWFQ -A form of weighted fair queuing. Weighted fair queuing shares a link's capacity between flows or classes in proportion to weights set for each. - -See also: [PWFQ](o-r.md#pwfq), [QoS](o-r.md#qos). - -### Niko-niko Calendar -A calendar on which each team member records their mood each day, often with a smiley face, to show patterns in how the team feels. - -See also: [Information Radiators](g-k.md#information-radiators). - -### NIST -**National Institute of Standards and Technology.** The US agency that maintains measurement standards and publishes widely used computer security standards and guidance, such as FIPS and the SP 800 series. - -See also: [FIPS](d-f.md#fips), [AES](a-c.md#aes). - -### NNI -**Network-to-Network Interface.** The interface between two networks, or two parts of one network, with its signalling and management. It differs from the user-network interface (UNI), between a network and its customer. - -See also: [UNI](s-z.md#uni). - -### NNTP -**Network News Transfer Protocol.** The protocol, first defined in RFC 977 in 1986, for distributing, reading and posting Usenet news articles. - -### NOC -**Network Operations Centre.** The place from which an operator monitors and manages its network. - -### Node B -The UMTS base station, the equivalent of the GSM BTS. It handles the radio link with mobile devices, under the control of an RNC. - -See also: [RNC](o-r.md#rnc), [BTS](a-c.md#bts), [E NodeB](d-f.md#e-nodeb), [NBAP](#nbap). - -### Nominal Path -The path through a system's behaviour where everything goes as expected, with no errors or exceptions. Also called the happy path. - -See also: [Use Case](s-z.md#use-case), [Happy Path](g-k.md#happy-path). - -### Non-functional Requirement -A requirement about how well a system works rather than what it does, such as performance, reliability, security, usability or compatibility. Also called a quality attribute. - -See also: [Functional Requirement](d-f.md#functional-requirement), [How Requirement](g-k.md#how-requirement). - -### NoSQL -Databases that store data in forms other than relational tables, such as key-value, document, wide-column or graph stores. They often trade strict consistency for easier horizontal scaling. - -See also: [RDBMS](o-r.md#rdbms), [SQL](s-z.md#sql). - -### NPE -**Network Provider Edge.** A provider edge (PE) router, which connects customer networks or other providers' networks to a provider's core. - -See also: [CE](a-c.md#ce), [L3VPN](#l3vpn). - -### NSS -**Network Switching Subsystem.** The GSM core network: the MSCs, HLR, VLR and related nodes that switch calls and manage mobility. GPRS added a packet core alongside it. - -See also: [MSC](#msc), [HLR](g-k.md#hlr), [GSM](g-k.md#gsm). - -### NTP -**Network Time Protocol.** The protocol that synchronises computer clocks over packet networks. It gets within tens of milliseconds of UTC over the internet, and usually within a millisecond on a LAN. - -See also: [SNTP](s-z.md#sntp), [PTP](o-r.md#ptp), [UTC](s-z.md#utc). - -### NTS -**Number Translation Service.** A non-geographic number, such as a UK 08 number, that the network translates to an ordinary number, so the destination can change without changing the published number. - -See also: [IN](g-k.md#in). - -### NTU -**Network Terminating Unit.** The device that marks the boundary between the operator's line and the customer's equipment, and gives a point for testing the line. - -See also: [CPE](a-c.md#cpe). - -### NYB -**Not Yet Built.** Royal Mail address data for properties that are planned or under construction. diff --git a/content/glossary/l.md b/content/glossary/l.md new file mode 100644 index 0000000..e30128a --- /dev/null +++ b/content/glossary/l.md @@ -0,0 +1,242 @@ +--- +title: L +--- + +# L + +## L1 +**Layer 1, the physical layer.** The lowest layer of the OSI model, which sends raw bits over a medium such as copper, fibre or radio. It defines the connectors, signals, timing and encoding. + +![L1](images/15008464.png) + +[[CAPTION:Figure]] L1. + +See also: [L2](#l2), [Hub](h.md#hub). + +## L2 +**Layer 2, the data link layer.** The OSI layer that moves frames between devices on the same link or LAN segment, with local (MAC) addressing and error detection. Ethernet and Wi-Fi work here; switches forward traffic at this layer. + +![L2](images/15008464.png) + +[[CAPTION:Figure]] L2. + +See also: [L1](#l1), [MAC](m.md#mac), [Switch](s.md#switch), [DLP](d.md#dlp). + +## L2F +**Layer 2 Forwarding.** A Cisco tunnelling protocol for carrying PPP sessions across an IP network. It has no encryption of its own, and L2TP replaced it. + +See also: [L2TP](#l2tp), [PPTP](p.md#pptp). + +## L2TP +**Layer 2 Tunnelling Protocol.** A protocol that tunnels PPP sessions over an IP network, used by ISPs to hand broadband sessions between networks and to build VPNs. It does not encrypt, so it is often run inside IPsec. + +See also: [LAC](#lac), [LNS](#lns), [LTS](#lts), [IPSec](i.md#ipsec). + +## L2VPN +**Layer 2 VPN.** A provider service that carries a customer's Ethernet or other layer-2 traffic between sites over the provider's MPLS network. The sites then appear to share one LAN or wire. Point-to-point and multipoint (VPLS) forms exist. + +See also: [VPLS](v.md#vpls), [VPWS](v.md#vpws), [L3VPN](#l3vpn), [PW](p.md#pw). + +## L3VPN +**Layer 3 VPN.** A provider service, usually MPLS-based as in RFC 4364, that routes a customer's IP traffic between sites. The provider edge routers keep each customer's routes in a separate VRF; customer edge routers connect to them, and provider core routers just switch labels. + +See also: [VRF](v.md#vrf), [CE](c.md#ce), [L2VPN](#l2vpn), [LSP](#lsp). + +## LAC +**L2TP Access Concentrator.** The device at one end of an L2TP tunnel that receives a user's PPP session and forwards it through the tunnel to the LNS. + +![LAC](images/15008478.png) + +[[CAPTION:Figure]] LAC. + +See also: [L2TP](#l2tp), [LNS](#lns). + +## LACP +**Link Aggregation Control Protocol.** The IEEE 802.1AX (formerly 802.3ad) protocol with which two devices agree to bundle several physical links into one logical link and check that each member is working. + +See also: [LAG](#lag). + +## LAG +**Link Aggregation Group.** Several physical links bundled into one logical link, to add capacity and survive the failure of a member. LACP usually manages it. + +![LAG](images/15008484.png) + +[[CAPTION:Figure]] LAG. + +See also: [LACP](#lacp). + +## LAN +**Local Area Network.** A network that connects devices within a limited area, such as a home, office or campus, usually with Ethernet and Wi-Fi. + +![LAN](images/15008490.png) + +[[CAPTION:Figure]] LAN. + +See also: [WAN](w.md#wan), [MAN](m.md#man). + +## LANE +**LAN Emulation.** An ATM Forum standard that made an ATM network look like an Ethernet or Token Ring LAN to the devices attached to it. + +See also: [ATM](a.md#atm). + +## Latency +The delay between something being sent and it arriving. For telephone calls it is the mouth-to-ear delay, made up of codec, network and playout delays. ITU-T G.114 advises keeping one-way delay under about 150 ms for good conversation. + +![Latency](images/15008496.png) + +[[CAPTION:Figure]] Latency. + +See also: [Jitter](j.md#jitter), [Jitter buffer](j.md#jitter-buffer), [Bandwidth](b.md#bandwidth). + +## Lawful intercept +**LI.** Facilities in telecoms networks that let authorised agencies, with a warrant, intercept a named target's communications. Operators must provide them, through standardised interfaces. It differs from bulk surveillance in being targeted at individuals. + +## LCR +**Least Cost Routing.** Choosing the route for each outgoing call by cost, across several carriers. Carriers revise their tables often, and PBXs can do the same for a business. + +See also: [CPS](c.md#cps). + +## LDAP +**Lightweight Directory Access Protocol.** An open protocol, defined in RFC 4511, for reading and updating directory services, such as a company's directory of users and groups. Active Directory supports it. + +See also: [AD](a.md#ad), [Kerberos](k.md#kerberos). + +## Lead Time +The time from a customer's request to its delivery. In software, often the time from a requirement being raised to it being met in production. + +See also: [Kanban](k.md#kanban). + +## Lean +A way of working, from Toyota's production system, that aims to deliver value to the customer by removing waste: anything that does not add value. + +See also: [Lean programming](#lean-programming), [Kanban](k.md#kanban). + +## Lean programming +Lean ideas applied to software development: removing waste, delivering fast, deciding as late as possible and building quality in. + +See also: [Lean](#lean). + +## Leased Line +A dedicated, symmetrical circuit between two points, rented for a fixed fee and always on. In the UK it is also called a private circuit. Businesses use leased lines to link sites and for internet access. + +See also: [EAD](e.md#ead), [Dark Fibre](d.md#dark-fibre), [CSI](c.md#csi). + +## LeSS +**Large-Scale Scrum.** A framework for scaling Scrum to many teams working on one product, with one product owner and one product backlog. + +See also: [Scrum](s.md#scrum), [SAFe](s.md#safe), [DAD](d.md#dad). + +## Lifecycle +A model of the activities from the first idea for a system to its retirement, usually divided into phases. Projects choose and tailor a lifecycle to suit their risks. + +See also: [Waterfall](w.md#waterfall), [Spiral Lifecycle](s.md#spiral-lifecycle), [SDLC](s.md#sdlc), [Phase](p.md#phase). + +## Lines of Code +**LOC.** A measure of software size as the number of source lines, usually counting neither comments nor blank lines. KLOC means thousands of lines. + +See also: [Function Point](f.md#function-point). + +## Linux +A family of free, open-source operating systems built on the Linux kernel, first released by Linus Torvalds in 1991. It is packaged in distributions, such as Debian, Ubuntu, Fedora and Red Hat Enterprise Linux. It runs most servers and embedded devices, and underlies Android. + +![Linux](images/15008502.png) + +[[CAPTION:Figure]] Linux. + +![Linux](images/14647331.svg) + +[[CAPTION:Figure]] Linux. + +![Linux](images/15008508.png) + +[[CAPTION:Figure]] Linux. + +See also: [GNU](g.md#gnu), [Unix](u.md#unix), [Kernel](k.md#kernel), [Android](a.md#android). + +## Lisp +A family of languages, first specified in 1958, with a fully bracketed prefix syntax in which code and data share the same form. Common Lisp, Scheme and Clojure are its main dialects. Lisp introduced many ideas now common, such as garbage collection and the read-eval-print loop. + +## LL +See [Local Loop](#local-loop). + +![LL](images/15008514.png) + +[[CAPTION:Figure]] LL. + +## LNS +**L2TP Network Server.** The device at the far end of an L2TP tunnel from the LAC, which terminates the users' PPP sessions, typically at the ISP. + +![LNS](images/15008478.png) + +[[CAPTION:Figure]] LNS. + +See also: [L2TP](#l2tp), [LAC](#lac), [LTS](#lts). + +## Local Build +A build done in a developer's local development environment. + +See also: [Project Build](p.md#project-build), [Build](b.md#build). + +## Local Development Environment +**LDE.** A developer's own development environment, usually on their own machine. + +See also: [Development Environment](d.md#development-environment), [Build Environment](b.md#build-environment). + +## Local Loop +The line that connects a customer's premises to the operator's network, traditionally a copper pair to the local exchange. Also called the last mile or subscriber line. + +See also: [MPF](m.md#mpf), [MDF](m.md#mdf), [DP](d.md#dp). + +## Local Test Environment +The part of the local development environment used for construction testing. + +See also: [Local Development Environment](#local-development-environment). + +## LoFi Prototype +See [Low Fidelity Prototype](#low-fidelity-prototype). + +## Low Fidelity Prototype +A rough mock-up of a system made with paper, sticky notes or simple drawings, used to explore ideas cheaply before building anything. + +See also: [High Fidelity Prototype](h.md#high-fidelity-prototype), [Storyboard](s.md#storyboard). + +## Low Level Design +**LLD.** The design level between high-level design and construction, detailed enough to build from directly. + +See also: [High Level Design](h.md#high-level-design), [Detailed Design](d.md#detailed-design). + +## LSP +**Label Switched Path.** A path through an MPLS network, set up by a protocol such as LDP or RSVP-TE. The first router adds a label, each router along the way swaps it, and the last removes it. + +See also: [LSR](#lsr), [RSVP](r.md#rsvp), [L3VPN](#l3vpn). + +## LSR +**Label Switch Router.** A router in the middle of an MPLS network that forwards packets by their labels alone. + +See also: [LSP](#lsp). + +## LTE (4G) +**Long-Term Evolution.** The 3GPP mobile standard (Release 8 onwards) that followed UMTS, with an OFDMA radio interface and an all-IP core, the Evolved Packet Core. LTE-Advanced meets the ITU's 4G requirements. Voice runs over IP as VoLTE. + +![LTE (4G)](images/15008520.png) + +[[CAPTION:Figure]] LTE (4G). + +See also: [E-UTRAN](e.md#e-utran), [E NodeB](e.md#e-nodeb), [MME](m.md#mme), [S-GW](s.md#s-gw), [PGW](p.md#pgw), [VoLTE](v.md#volte). + +## LTS +**L2TP Tunnel Switch.** A router between a LAC and an LNS that acts as both. It ends incoming L2TP tunnels and forwards the sessions through new tunnels to other LNSs, for example to different ISPs. + +![LTS](images/15008526.png) + +[[CAPTION:Figure]] LTS. + +See also: [L2TP](#l2tp), [LAC](#lac), [LNS](#lns). + +## Lua +A small, fast scripting language from Brazil, designed in 1993 to be embedded in applications through a simple C API. It is used widely in games and network equipment. + +## LwM2M +**Lightweight Machine to Machine.** The OMA SpecWorks protocol for managing IoT devices and their data over CoAP, usually secured with DTLS. Devices expose standard objects, such as device information, connectivity and firmware update, so servers can manage different makers' devices in the same way. + +See also: [M2M](m.md#m2m), [DTLS](d.md#dtls), [CBOR](c.md#cbor), [FOTA](f.md#fota). diff --git a/content/glossary/m.md b/content/glossary/m.md new file mode 100644 index 0000000..b6b9fa0 --- /dev/null +++ b/content/glossary/m.md @@ -0,0 +1,452 @@ +--- +title: M +--- + +# M + +## M2M +**Machine to Machine.** Direct communication between devices, such as meters, sensors and vehicles, and the systems that use their data, without people in the loop. It was the forerunner of the Internet of Things. + +See also: [LwM2M](l.md#lwm2m), [MQTT](#mqtt). + +## MAC +**Medium Access Control.** The lower sublayer of the data link layer in IEEE 802 networks. It controls access to the shared medium and handles framing and hardware addressing; the logical link control (LLC) sublayer sits above it. + +See also: [MAC Address](#mac-address), [L2](l.md#l2), [CSMA](c.md#csma). + +## MAC Address +**Media Access Control address.** The 48-bit hardware address of a network interface on Ethernet, Wi-Fi and other IEEE 802 networks. It is written as six hex pairs, such as `00:1A:2B:3C:4D:5E`. The first half normally identifies the manufacturer. + +![MAC Address](images/15008537.png) + +[[CAPTION:Figure]] MAC Address. + +See also: [MAC](#mac), [MAC Spoofing](#mac-spoofing), [ARP spoofing](a.md#arp-spoofing). + +## MAC Flooding +An attack that fills a switch's address table with fake MAC addresses. The switch then floods frames out of every port, letting the attacker see traffic meant for others. + +See also: [Switch](s.md#switch), [MAC Spoofing](#mac-spoofing). + +## MAC Spoofing +Changing the MAC address a network interface presents, usually through its driver, to pose as another device or hide its identity. + +See also: [MAC Address](#mac-address), [ARP spoofing](a.md#arp-spoofing). + +## Machine code +Instructions in the binary form a processor runs directly. People rarely write it by hand: compilers and assemblers produce it. + +See also: [Compiler](c.md#compiler), [Compiled Language](c.md#compiled-language). + +## Macro Estimation +Estimating something as a whole. See [Top-Down Estimation](t.md#top-down-estimation). + +## Maintenance +Software maintenance: correcting, updating, adapting and operating software after its first release. + +See also: [Maintenance Plan](#maintenance-plan). + +## Maintenance Plan +A plan for maintaining a system after its first release. + +## MAN +**Metropolitan Area Network.** A network that covers a city or region: larger than a LAN, smaller than a WAN, and often linking several LANs. + +![MAN](images/15008545.png) + +[[CAPTION:Figure]] MAN. + +See also: [LAN](l.md#lan), [WAN](w.md#wan). + +## Management by Walking Around +**MBWA.** Managing through frequent, informal contact with the team, alongside formal tracking, to spot risks and problems early. + +## Management Plane +The part of a network device or network that configures, monitors and manages it, through interfaces such as the CLI, SNMP or NETCONF. + +![Management Plane](images/15138919.png) + +[[CAPTION:Figure]] Management Plane. + +See also: [Control Plane](c.md#control-plane), [Data Plane](d.md#data-plane), [SNMP](s.md#snmp). + +## MANET +**Mobile Ad hoc Network.** A self-configuring wireless network of mobile devices with no fixed infrastructure, in which every device also routes traffic for the others. + +![MANET](images/15008551.png) + +[[CAPTION:Figure]] MANET. + +![MANET](images/15008557.png) + +[[CAPTION:Figure]] MANET. + +See also: [WANET](w.md#wanet), [Wireless Mesh Network](w.md#wireless-mesh-network), [Geocast](g.md#geocast). + +## Manual Testing +Testing in which a person runs the test cases and records the results. + +See also: [Automated Testing](a.md#automated-testing). + +## MAP +**Mobile Application Part.** The SS7 protocol with which GSM and UMTS core network nodes, such as the HLR, VLR, MSC and SGSN, exchange information. It covers location updates, authentication and SMS delivery. + +See also: [SS7](s.md#ss7), [HLR](h.md#hlr), [VLR](v.md#vlr), [TCAP](t.md#tcap). + +## MBMS +**Multimedia Broadcast Multicast Services.** 3GPP standards for broadcasting and multicasting content to many mobile devices at once. In LTE it is called eMBMS or LTE Broadcast. + +See also: [Multicast](#multicast), [LTE (4G)](l.md#lte-4g). + +## MBORC +**Matters Beyond Our Reasonable Control.** A clause in Openreach contracts that relieves Openreach of its service-level obligations when an outside event, such as severe weather or third-party damage, prevents it from meeting them. + +## MCU +**Microcontroller Unit.** A single chip with a processor core, flash and RAM, and peripherals, used in small embedded devices. It usually runs bare-metal code or an RTOS. + +See also: [MPU](#mpu), [RTOS](r.md#rtos), [Embedded systems programming](e.md#embedded-systems-programming). + +## MD5 +**Message Digest 5.** A 128-bit hash function from 1991. It is broken for security uses because collisions are easy to make, so it should be used only as a checksum against accidental corruption. + +See also: [SHA](s.md#sha), [Hash Function](h.md#hash-function). + +## MDF +**Main Distribution Frame.** The frame in a telephone exchange where the external cables to customers are connected to the exchange's equipment by jumpers. + +![MDF](images/15237176.png) + +[[CAPTION:Figure]] MDF. + +See also: [HDF](h.md#hdf), [Local Loop](l.md#local-loop), [MPF](#mpf). + +## Mediation +In telecoms, the systems that collect usage records, such as CDRs, from network elements, then clean, combine and convert them into the formats that billing, fraud and other systems need. + +![Mediation](images/15008563.png) + +[[CAPTION:Figure]] Mediation. + +![Mediation](images/15008569.png) + +[[CAPTION:Figure]] Mediation. + +See also: [CDR](c.md#cdr), [OSS](o.md#oss). + +## MELP +**Mixed-Excitation Linear Prediction.** A US military speech codec standard. Its enhanced version, MELPe, works at 2,400, 1,200 or 600 bit/s for secure and narrow-band radio links. + +![MELP](images/14909510.png) + +[[CAPTION:Figure]] MELP. + +See also: [CVSD](c.md#cvsd), [Audio Codecs](a.md#audio-codecs). + +## Metric +A measurement taken from a project, process or product to judge whether it has the qualities wanted. + +See also: [KPI](k.md#kpi). + +## MGCF +**Media Gateway Control Function.** In IMS, the function that converts call signalling between SIP and the circuit-switched network's ISUP or BICC, and controls the media gateway through H.248. + +![MGCF](images/14123073.png) + +[[CAPTION:Figure]] MGCF. + +See also: [MGW](#mgw), [IMS](i.md#ims), [BGCF](b.md#bgcf). + +## MGCP +**Media Gateway Control Protocol.** A VoIP protocol in which a central call agent controls simple media gateways and endpoints. It is an IETF informational RFC; H.248 is the formal standard for the same role. + +![MGCP](images/15008575.png) + +[[CAPTION:Figure]] MGCP. + +See also: [H.248](h.md#h248), [Softswitch](s.md#softswitch). + +## MGW +**Media Gateway.** A device that converts media between networks, typically between PCM on telephone circuits and RTP streams on IP. It also transcodes when the two sides use different codecs. + +![MGW](images/15302756.png) + +[[CAPTION:Figure]] MGW. + +See also: [MGCF](#mgcf), [H.248](h.md#h248), [RTP](r.md#rtp). + +## MIB +**Management Information Base.** The tree-structured set of objects that a device exposes for management, mostly through SNMP. Each object is identified by an object identifier (OID). + +![MIB](images/15008581.png) + +[[CAPTION:Figure]] MIB. + +See also: [SNMP](s.md#snmp). + +## Micro Estimation +Estimating the details of something. See [Bottom-Up Estimation](b.md#bottom-up-estimation). + +## Microservice +An architecture style in which an application is a set of small services, each built around a business capability. Each can be deployed independently and communicates through lightweight APIs. It suits large systems with many teams, at the cost of operational complexity. + +![Microservice](images/15008587.png) + +[[CAPTION:Figure]] Microservice. + +![Microservice](images/15008593.png) + +[[CAPTION:Figure]] Microservice. + +See also: [SOA](s.md#soa), [BFF](b.md#bff), [Docker](d.md#docker). + +## Microstone +See [Miniature Milestone](#miniature-milestone). + +## Mil-COTS +COTS products for military use. See [COTS](c.md#cots). + +## Milestone +A significant point in a schedule, such as an achievement or a date. + +See also: [Miniature Milestone](#miniature-milestone), [Business Schedule](b.md#business-schedule). + +## Milestone Release +A release made for a milestone in the project plan. + +## Milestone Retrospective +A detailed review by the team of a project's significant events, after a major milestone or at the end of the project. + +See also: [Postmortem](p.md#postmortem), [Sprint retrospective](s.md#sprint-retrospective). + +## Milestone Schedule +A business schedule that sets out the project's formal milestones. + +See also: [Business Schedule](b.md#business-schedule). + +## MIMO +**Multiple-Input, Multiple-Output.** Using several antennas at both ends of a radio link to send several data streams at once over the same channel, exploiting multipath propagation. Wi-Fi, LTE and 5G all rely on it. + +![MIMO](images/15008599.png) + +[[CAPTION:Figure]] MIMO. + +See also: [SISO](s.md#siso), [SDMA](s.md#sdma), [Wi-Fi](w.md#wi-fi). + +## Miniature Milestone +A small milestone, a day or two of work, that counts as either done or not done, never partly done. It makes progress easy to track. + +See also: [Milestone](#milestone), [Inch-Pebble](i.md#inch-pebble). + +## MME +**Mobility Management Entity.** The main control node of the LTE core. It authenticates users with the HSS, tracks and pages idle devices, chooses gateways and manages bearers and handovers. + +See also: [HSS](h.md#hss), [S-GW](s.md#s-gw), [E NodeB](e.md#e-nodeb), [LTE (4G)](l.md#lte-4g). + +## MMF +**Minimum Marketable Feature.** The smallest feature that is worth releasing on its own because it gives users real value. + +See also: [MVP](#mvp). + +## MNC +**Mobile Network Code.** The two- or three-digit code that, with the mobile country code (MCC), identifies a mobile network. Both form the start of an IMSI. + +See also: [IMSI](i.md#imsi). + +## Mob Programming +The whole team working on the same thing, at the same time, in the same place, at one computer. + +See also: [Pair Programming](p.md#pair-programming). + +## Mobile Cell +The area served by one base station in a cellular network. Splitting coverage into cells lets the same frequencies be reused in cells further apart. Cell sizes range from a few metres to tens of kilometres. + +![Mobile Cell](images/15008605.png) + +[[CAPTION:Figure]] Mobile Cell. + +See also: [BTS](b.md#bts), [FemtoCell](f.md#femtocell). + +## Mock Objects +Stand-ins for real components, created for a test, that imitate the real behaviour and check how the code under test uses them. + +See also: [Unit Testing](u.md#unit-testing), [TDD](t.md#tdd). + +## Model +A simplified representation of a system or process, often as diagrams, that describes its important characteristics. + +See also: [Modelling](#modelling), [UML](u.md#uml). + +## Modelling +Using models in requirements and design work. + +See also: [Model](#model). + +## Moderator +The person who leads an inspection and makes sure it follows the process. + +See also: [Inspection](i.md#inspection). + +## MOS +**Mean Opinion Score.** A rating of perceived quality, especially of voice calls, on a scale from 1 (bad) to 5 (excellent). It is either the average of listeners' scores or estimated by an algorithm. ITU-T P.800 defines it. + +![MOS](images/15008617.png) + +[[CAPTION:Figure]] MOS. + +![MOS](images/15237186.png) + +[[CAPTION:Figure]] MOS. + +| MOS | Quality | Impairment | +| --- | --- | --- | +| 5 | Excellent | Imperceptible | +| 4 | Good | Perceptible but not annoying | +| 3 | Fair | Slightly annoying | +| 2 | Poor | Annoying | +| 1 | Bad | Very annoying | + +[[CAPTION:Table]] The MOS scale. + +See also: [Latency](l.md#latency), [Jitter](j.md#jitter). + +## MP3 +**MPEG-1 or MPEG-2 Audio Layer III.** A lossy audio coding format that removes sounds people are unlikely to hear. It made music files small enough to share and became the most common format for digital music. + +See also: [AAC](a.md#aac), [MPEG](#mpeg). + +## MPEG +**Moving Picture Experts Group.** The ISO and IEC working group that sets standards for audio and video coding, such as MPEG-2 and MPEG-4. + +See also: [MPEG4](#mpeg4), [MP3](#mp3). + +## MPEG4 +**MPEG-4.** A family of ISO/IEC standards (14496) for coding audio and video. It includes AAC audio and the H.264 (AVC) video codec, widely used for streaming and broadcast. + +See also: [MPEG](#mpeg), [AAC](a.md#aac). + +## MPF +**Metallic Path Facility.** An Openreach product that gives another operator a whole copper line from the exchange to the customer, so the operator provides both voice and broadband. + +![MPF](images/15008623.png) + +[[CAPTION:Figure]] MPF. + +See also: [SMPF](s.md#smpf), [Local Loop](l.md#local-loop), [MDF](#mdf). + +## MPU +**Microprocessor Unit.** A processor that relies on external memory and usually has a memory management unit, so it can run a full operating system such as Linux. + +See also: [MCU](#mcu). + +## MQTT +A lightweight publish-subscribe messaging protocol, over TCP, in which clients publish and subscribe to topics through a broker. It was designed in 1999 for low-bandwidth links, is an OASIS standard, and is widely used in IoT. + +See also: [M2M](#m2m), [LwM2M](l.md#lwm2m). + +## MRF +**Media Resource Function.** In IMS, the function that plays announcements and tones, mixes conference audio and does other media processing. + +![MRF](images/14123067.png) + +[[CAPTION:Figure]] MRF. + +See also: [IMS](i.md#ims). + +## MRS +**Multiple Residence data.** Royal Mail address data that lists the separate homes, such as flats, behind a single delivery point. + +## MRTIE +**Maximum Relative Time Interval Error.** A measure of clock stability in synchronised networks: the largest change in a clock's time error against a reference over a given observation interval. + +See also: [PTP](p.md#ptp), [Jitter](j.md#jitter). + +## MSAN +**Multi-Service Access Node.** Equipment in an exchange or cabinet that connects customers' lines to the core network and provides voice, ISDN and broadband from one platform. It replaces separate exchange line cards and DSLAMs. + +![MSAN](images/15008635.png) + +[[CAPTION:Figure]] MSAN. + +See also: [DSLAM](d.md#dslam), [BT 21CN](b.md#bt-21cn), [NGN](n.md#ngn). + +## MSC +**Mobile Switching Centre.** The switch in a GSM or UMTS core network that sets up and routes voice calls and SMS, handles handovers between areas and produces charging records. + +![MSC](images/15138931.png) + +[[CAPTION:Figure]] MSC. + +See also: [GMSC](g.md#gmsc), [HLR](h.md#hlr), [VLR](v.md#vlr), [BSC](b.md#bsc). + +## MTP +**Message Transfer Part.** The lower levels (1 to 3) of SS7, which carry signalling messages reliably and in order between signalling points. + +![MTP](images/15008641.png) + +[[CAPTION:Figure]] MTP. + +See also: [SS7](s.md#ss7), [SCCP](s.md#sccp). + +## MTR +**Mean Time to Repair (MTTR).** The average time taken to restore failed equipment or a service to working order. + +## MTU +**Maximum Transmission Unit.** The largest packet a network link can carry without fragmentation, 1,500 bytes on standard Ethernet. Tunnels add headers, so they reduce the usable MTU. + +See also: [PDU](p.md#pdu). + +## Mu-law +**µ-law.** A companding algorithm, one of the two variants of ITU-T G.711, used for 8-bit PCM speech in North America and Japan. It is also written u-law. + +See also: [A-law](a.md#a-law), [G.711](g.md#g711). + +## Multicast +Sending data to a group of receivers at once, with the network copying packets only where the paths divide. IPTV uses it, with receivers joining groups through IGMP. + +![Multicast](images/15008647.png) + +[[CAPTION:Figure]] Multicast. + +```mermaid +flowchart LR + S((Sender)) --> R1[Router] + R1 --> R2[Router] + R1 --> R3[Router] + R2 --> A((Member)) + R2 --> B((Member)) + R3 --> C((Member)) +``` + +[[CAPTION:Figure]] Multicast: one stream, copied only where the paths divide. + +See also: [Unicast](u.md#unicast), [Broadcast](b.md#broadcast), [Anycast](a.md#anycast), [IGMP](i.md#igmp). + +## Multiplex +Combining several signals or data streams onto one shared medium, by time, frequency, wavelength or code, and separating them again at the far end. + +See also: [TDM](t.md#tdm), [OFDMA](o.md#ofdma), [CDMA](c.md#cdma). + +## Mutex +A lock that lets only one thread at a time hold a shared resource, so threads cannot corrupt it by using it at once. + +See also: [Mutual Exclusion](#mutual-exclusion), [Semaphore](s.md#semaphore). + +## Mutual Exclusion +The requirement that only one thread at a time runs a critical section that uses a shared resource, to prevent race conditions. Edsger Dijkstra first described and solved the problem in 1965. + +See also: [Mutex](#mutex), [Semaphore](s.md#semaphore). + +## MVNE +**Mobile Virtual Network Enabler.** A company that provides MVNOs with the systems they need, such as billing, provisioning and network integration, without dealing with consumers itself. + +See also: [MVNO](#mvno). + +## MVNO +**Mobile Virtual Network Operator.** A mobile operator that sells services under its own brand but does not own the radio network. It buys capacity wholesale from a network operator. + +See also: [MVNE](#mvne). + +## MVP +**Minimum Viable Product.** The first version of a product built to learn as much as possible about customers with the least effort, as described by Eric Ries. + +See also: [MMF](#mmf), [Customer Development](c.md#customer-development). diff --git a/content/glossary/n.md b/content/glossary/n.md new file mode 100644 index 0000000..0cd2d8c --- /dev/null +++ b/content/glossary/n.md @@ -0,0 +1,183 @@ +--- +title: N +--- + +# N + +## NaaS +**Network as a Service.** Network connectivity, such as VPNs and bandwidth on demand, offered as a cloud-style service, often built on network virtualisation. + +See also: [IaaS](i.md#iaas), [SDN](s.md#sdn). + +## NAPT +**Network Address and Port Translation.** NAT that also rewrites port numbers, so that many private hosts can share one public IP address. It is also called PAT or IP masquerading. + +See also: [NAT](#nat). + +## NAS +**Network Access Server.** A device that gives users access to a network, such as a dial-up or broadband access server. It usually authenticates them through RADIUS. In LTE, NAS also stands for the non-access stratum: the signalling between the device and the core. + +![NAS](images/15008661.png) + +[[CAPTION:Figure]] NAS. + +See also: [Radius](r.md#radius), [BRAS](b.md#bras). + +## NAT +**Network Address Translation.** Rewriting IP addresses in packets as they pass through a router, usually so that a private network can share public addresses. It has kept IPv4 going despite address exhaustion, but it complicates peer-to-peer protocols. + +![NAT](images/15008669.png) + +[[CAPTION:Figure]] NAT. + +See also: [NAPT](#napt), [STUN](s.md#stun), [ICE](i.md#ice), [IPv4](i.md#ipv4). + +## Native Risk Management +See [Intrinsic Risk Management](i.md#intrinsic-risk-management). + +## NB-IoT +**Narrowband IoT.** A 3GPP low-power, wide-area radio technology for simple IoT devices, such as meters, that send little data and must run for years on a battery. + +See also: [LTE (4G)](l.md#lte-4g), [LwM2M](l.md#lwm2m). + +## NBAP +**Node B Application Part.** The UMTS signalling protocol on the Iub interface with which the RNC controls a Node B, setting up cells and radio links. + +See also: [Node B](#node-b), [RNC](r.md#rnc), [UTRAN](u.md#utran). + +## NCSC +**National Cyber Security Centre.** The UK's technical authority on cyber security, part of GCHQ, which advises government, businesses and the public. It opened in 2016. + +See also: [GCHQ](g.md#gchq), [CESG](c.md#cesg). + +## Network Topologies +The arrangement of a network's nodes and links, such as bus, star, ring, mesh or tree. The physical topology is how it is cabled; the logical topology is how data flows. + +![Network Topologies](images/15008675.png) + +[[CAPTION:Figure]] Network Topologies. + +```mermaid +flowchart LR + subgraph Star + H((Hub)) --- s1((A)) + H --- s2((B)) + H --- s3((C)) + end + subgraph Ring + r1((A)) --- r2((B)) --- r3((C)) --- r1 + end + subgraph Mesh + m1((A)) --- m2((B)) + m2 --- m3((C)) + m3 --- m1 + end +``` + +[[CAPTION:Figure]] Star, ring and mesh topologies. + +## NFS +**Network File System.** A protocol, first developed by Sun in 1984, that lets a computer use files on a remote server as if they were local. + +## NGN +**Next-Generation Network.** A packet-based network, usually all-IP, that carries every service, voice included, with service control separate from transport. ITU-T and ETSI TISPAN defined NGN architectures in the 2000s, and IMS became their service layer. + +See also: [IMS](i.md#ims), [BT 21CN](b.md#bt-21cn), [Softswitch](s.md#softswitch). + +## NGWFQ +A form of weighted fair queuing. Weighted fair queuing shares a link's capacity between flows or classes in proportion to weights set for each. + +![NGWFQ](images/15008681.png) + +[[CAPTION:Figure]] NGWFQ. + +See also: [PWFQ](p.md#pwfq), [QoS](q.md#qos). + +## Niko-niko Calendar +A calendar on which each team member records their mood each day, often with a smiley face, to show patterns in how the team feels. + +![Niko-niko Calendar](images/15008687.png) + +[[CAPTION:Figure]] Niko-niko Calendar. + +See also: [Information Radiators](i.md#information-radiators). + +## NIST +**National Institute of Standards and Technology.** The US agency that maintains measurement standards and publishes widely used computer security standards and guidance, such as FIPS and the SP 800 series. + +See also: [FIPS](f.md#fips), [AES](a.md#aes). + +## NNI +**Network-to-Network Interface.** The interface between two networks, or two parts of one network, with its signalling and management. It differs from the user-network interface (UNI), between a network and its customer. + +![NNI](images/15008693.png) + +[[CAPTION:Figure]] NNI. + +See also: [UNI](u.md#uni). + +## NNTP +**Network News Transfer Protocol.** The protocol, first defined in RFC 977 in 1986, for distributing, reading and posting Usenet news articles. + +## NOC +**Network Operations Centre.** The place from which an operator monitors and manages its network. + +## Node B +The UMTS base station, the equivalent of the GSM BTS. It handles the radio link with mobile devices, under the control of an RNC. + +See also: [RNC](r.md#rnc), [BTS](b.md#bts), [E NodeB](e.md#e-nodeb), [NBAP](#nbap). + +## Nominal Path +The path through a system's behaviour where everything goes as expected, with no errors or exceptions. Also called the happy path. + +See also: [Use Case](u.md#use-case), [Happy Path](h.md#happy-path). + +## Non-functional Requirement +A requirement about how well a system works rather than what it does, such as performance, reliability, security, usability or compatibility. Also called a quality attribute. + +See also: [Functional Requirement](f.md#functional-requirement), [How Requirement](h.md#how-requirement). + +## NoSQL +Databases that store data in forms other than relational tables, such as key-value, document, wide-column or graph stores. They often trade strict consistency for easier horizontal scaling. + +See also: [RDBMS](r.md#rdbms), [SQL](s.md#sql). + +## NPE +**Network Provider Edge.** A provider edge (PE) router, which connects customer networks or other providers' networks to a provider's core. + +![NPE](images/15008699.png) + +[[CAPTION:Figure]] NPE. + +See also: [CE](c.md#ce), [L3VPN](l.md#l3vpn). + +## NSS +**Network Switching Subsystem.** The GSM core network: the MSCs, HLR, VLR and related nodes that switch calls and manage mobility. GPRS added a packet core alongside it. + +![NSS](images/15008705.png) + +[[CAPTION:Figure]] NSS. + +See also: [MSC](m.md#msc), [HLR](h.md#hlr), [GSM](g.md#gsm). + +## NTP +**Network Time Protocol.** The protocol that synchronises computer clocks over packet networks. It gets within tens of milliseconds of UTC over the internet, and usually within a millisecond on a LAN. + +See also: [SNTP](s.md#sntp), [PTP](p.md#ptp), [UTC](u.md#utc). + +## NTS +**Number Translation Service.** A non-geographic number, such as a UK 08 number, that the network translates to an ordinary number, so the destination can change without changing the published number. + +See also: [IN](i.md#in). + +## NTU +**Network Terminating Unit.** The device that marks the boundary between the operator's line and the customer's equipment, and gives a point for testing the line. + +![NTU](images/15008711.png) + +[[CAPTION:Figure]] NTU. + +See also: [CPE](c.md#cpe). + +## NYB +**Not Yet Built.** Royal Mail address data for properties that are planned or under construction. diff --git a/content/glossary/numbers.md b/content/glossary/numbers.md new file mode 100644 index 0000000..c611d4b --- /dev/null +++ b/content/glossary/numbers.md @@ -0,0 +1,30 @@ +--- +title: Numbers +--- + +# Numbers + +## 100 Mbps (Fast Ethernet) {#100-mbps-fast-ethernet} +**Fast Ethernet.** Ethernet at 100 Mbit/s, introduced in 1995 as IEEE 802.3u. 100BASE-TX over Cat 5 cable is by far the most common form. + +See also: [Gigabit Ethernet](g.md#gigabit-ethernet), [Ethernet over twisted pair](e.md#ethernet-over-twisted-pair). + +## 5ESS {#5ess} +A digital telephone switch developed by Western Electric for AT&T, first in service in 1982. It became one of the most widely used local exchanges in the United States. + +See also: [SystemX](s.md#systemx), [Class 5 Softswitch](c.md#class-5-softswitch). + +## 5G {#5g} +The fifth generation of mobile networks, standardised by 3GPP from Release 15 (2018) to meet the ITU's IMT-2020 targets. It uses the New Radio (NR) air interface and a service-based core. It offers faster broadband, low-latency services and very large numbers of IoT devices. Speeds depend heavily on the frequency band used. + +See also: [LTE (4G)](l.md#lte-4g), [OFDMA](o.md#ofdma), [MIMO](m.md#mimo). + +## 802.11 {#80211} +The IEEE family of standards for wireless LANs, first published in 1997, which defines the physical and MAC layers that Wi-Fi products use. Amendments add speed and features, such as 802.11n, ac, ax and be. + +See also: [Wi-Fi](w.md#wi-fi), [IEEE](i.md#ieee). + +## 802.3 Physical Layer {#8023-physical-layer} +**IEEE 802.3.** The IEEE working group and family of standards that define wired Ethernet: the physical layers and the MAC. They cover speeds from 10 Mbit/s to hundreds of gigabits per second, over copper and fibre. + +See also: [Ethernet over twisted pair](e.md#ethernet-over-twisted-pair), [MAC](m.md#mac), [CSMA](c.md#csma). diff --git a/content/glossary/o-r.md b/content/glossary/o-r.md deleted file mode 100644 index 9bcd7e5..0000000 --- a/content/glossary/o-r.md +++ /dev/null @@ -1,809 +0,0 @@ ---- -title: Glossary, O to R ---- - -# O to R - -## O - -### Objective C -An object-oriented language that adds Smalltalk-style messaging to C. It was Apple's main language for macOS and iOS apps until Swift arrived. - -See also: [Swift](s-z.md#swift), [C Language](a-c.md#c-language). - -### Objectives -See [Goals](g-k.md#goals). - -### ODF -**Optical Distribution Frame.** A frame that terminates fibre cables and lets connections between them be made and changed with patch leads. - -See also: [MDF](l-n.md#mdf), [OFC](#ofc). - -### OFC -**Optical Fibre Connector.** A connector that joins optical fibres, aligning their cores so light passes with little loss. It is quicker to connect and disconnect than a splice. Common types include LC, SC and ST. - -See also: [ODF](#odf). - -### Ofcom -**Office of Communications.** The UK regulator for broadcasting, telecoms, post and the radio spectrum, set up under the Communications Act 2003. - -### OFDMA -**Orthogonal Frequency-Division Multiple Access.** The multi-user form of OFDM, in which different users get different sets of closely spaced subcarriers at the same time. LTE downlinks, 5G and Wi-Fi 6 use it. - -See also: [SC-FDMA](s-z.md#sc-fdma), [COFDM](a-c.md#cofdm), [LTE (4G)](l-n.md#lte-4g). - -### OO -**Object-Oriented.** A paradigm that organises software around objects, which bundle data with the code that acts on it. Most OO languages are class-based, with inheritance and polymorphism. - -See also: [Programming paradigm](#programming-paradigm), [Polymorphism](#polymorphism), [UML](s-z.md#uml). - -### Operations Plan -A plan for running a system after deployment: roles, responsibilities, procedures and maintenance schedules. - -See also: [Deployment Plan](d-f.md#deployment-plan), [Maintenance Plan](l-n.md#maintenance-plan). - -### OPUS -**Opus.** An open, royalty-free audio codec, standardised by the IETF in RFC 6716. It combines the SILK speech coder with the CELT low-delay coder and adapts from narrowband speech to full-band music. It is the standard codec for WebRTC. - -See also: [SPEEX](s-z.md#speex), [Vorbis](s-z.md#vorbis), [Audio Codecs](a-c.md#audio-codecs). - -### OS X -**macOS.** Apple's operating system for Mac computers, released in 2001 as Mac OS X and renamed OS X and then macOS. It is a Unix system built on the Darwin kernel. - -See also: [Unix](s-z.md#unix). - -### OSA-FC -**Optical Spectrum Access Filter Connect.** An Openreach service that gives a provider a managed fibre connection on which it runs its own optical equipment. It works like dark fibre, but Openreach monitors and repairs the link. - -See also: [Dark Fibre](d-f.md#dark-fibre), [EAD](d-f.md#ead). - -### OSPF -**Open Shortest Path First.** A link-state interior gateway protocol: each router learns the whole topology of its area and calculates the shortest paths. Version 2 (RFC 2328) is for IPv4 and version 3 for IPv6. - -See also: [IGP](g-k.md#igp), [RIP](#rip), [BGP](a-c.md#bgp). - -### OSS -**Operations Support Systems.** The systems a telecoms operator uses to run its network, such as inventory, provisioning, configuration, fault and performance management. They work with the business support systems (BSS). - -See also: [BSS](a-c.md#bss), [Mediation](l-n.md#mediation). - -### OTN -**Optical Transport Network.** The ITU-T G.709 standard for carrying client signals, such as Ethernet or SDH, over wavelengths. It adds framing, forward error correction and management to optical networks. - -See also: [SDH](s-z.md#sdh). - -### Out-of-order delivery -Packets arriving in a different order from the one in which they were sent, for example because they took different paths. TCP puts the data back in order; real-time protocols must cope with it. - -See also: [TCP](s-z.md#tcp), [Jitter buffer](g-k.md#jitter-buffer). - -### Overlap Dialling -Sending a dialled number in pieces as the caller enters it, rather than all at once, so that routing can begin before the number is complete. The far end waits for more digits until a timer runs out. - -See also: [ISUP](g-k.md#isup), [Q.931](#q931). - -## P - -### PaaS -**Platform as a Service.** Cloud computing that provides a managed platform for building and running applications, so the customer does not manage servers or operating systems. - -See also: [IaaS](g-k.md#iaas), [SaaS](s-z.md#saas). - -### Packet sniffing -Capturing and decoding the packets crossing a network, usually with a packet analyser such as Wireshark, to troubleshoot problems or, in an attack, to steal data. - -See also: [Wireshark](s-z.md#wireshark), [DPI](d-f.md#dpi). - -### Packet Switched -A network that splits data into packets, each with a header that routers use to forward it independently, sharing links with other traffic. The internet works this way. - -See also: [Circuit Switched](a-c.md#circuit-switched). - -### PAF -**Postcode Address File.** Royal Mail's database of every UK delivery address and its postcode. - -### Pair Programming -Two developers working together at one computer: one writes the code while the other reviews it and thinks ahead, and they swap often. - -See also: [Mob Programming](l-n.md#mob-programming), [XP](s-z.md#xp). - -### PAP -**Password Authentication Protocol.** A simple PPP authentication method that sends the username and password in clear text. It should be used only when nothing stronger, such as CHAP or EAP, is available. - -See also: [CHAP](a-c.md#chap), [PPP](#ppp). - -### Parametric Estimation -Estimating with a model that calculates effort, cost or schedule from input factors, using formulas based on historical data. COCOMO is an example. - -See also: [Estimation](d-f.md#estimation), [Statistical Estimation](s-z.md#statistical-estimation). - -### Pascal -A procedural language designed by Niklaus Wirth around 1970 to teach structured programming. Object Pascal and Delphi descend from it. - -See also: [Delphi](d-f.md#delphi), [Ada](a-c.md#ada). - -### Pattern -A reusable template or model for creating an artefact or solving a recurring problem. - -See also: [Design Pattern](d-f.md#design-pattern), [Antipattern](a-c.md#antipattern). - -### PCM -**Pulse-Code Modulation.** The standard way of representing an analogue signal digitally: sample it at regular intervals and quantise each sample to a binary value. Telephone speech uses 8,000 samples a second at 8 bits (G.711), and CDs use 44,100 at 16 bits. - -See also: [G.711](g-k.md#g711), [A-law](a-c.md#a-law), [Mu-law](l-n.md#mu-law). - -### PCP -**Primary Connection Point.** The green roadside cabinet in the UK access network where the main cables from the exchange connect to the cables to local distribution points. Fibre-to-the-cabinet equipment sits next to it. - -In Ethernet, PCP also means the **Priority Code Point**: the 3-bit priority field in an 802.1Q tag. - -See also: [DP](d-f.md#dp), [VDSL](s-z.md#vdsl), [Dot1Q](d-f.md#dot1q). - -### PCRF -**Policy and Charging Rules Function.** The node in a mobile core that decides, in real time, the policy and charging rules for each subscriber's data sessions, such as quality of service and usage limits. The gateways then enforce them. - -See also: [PGW](#pgw), [QoS](#qos). - -### PCS -**Physical Coding Sublayer.** The top sublayer of the Ethernet physical layer, which encodes and decodes data, for example with 8b/10b or 64b/66b coding, and scrambles it. - -See also: [L1](l-n.md#l1). - -### PDCP -**Packet Data Convergence Protocol.** A 3GPP radio protocol layer, in UMTS, LTE and 5G, that compresses headers, ciphers data and delivers packets in order between the device and the base station. - -See also: [RRC](#rrc), [E-UTRAN](d-f.md#e-utran). - -### PDH -**Plesiochronous Digital Hierarchy.** The older digital transmission hierarchy. Links such as E1 (2 Mbit/s) and E3 run at nearly, but not exactly, the same rates and are multiplexed by bit stuffing. SDH replaced it in backbones. - -See also: [SDH](s-z.md#sdh), [TDM](s-z.md#tdm). - -### PDU -**Protocol Data Unit.** The unit of data a protocol layer exchanges with its peer. It is a frame at layer 2, a packet at layer 3, and a segment (TCP) or datagram (UDP) at layer 4. - -See also: [MTU](l-n.md#mtu). - -### PEAP -**Protected EAP.** An EAP method that sets up a TLS tunnel first and then runs a simpler authentication, often a password, inside it. It is common for Wi-Fi in organisations. - -See also: [EAP](d-f.md#eap), [TLS](s-z.md#tls). - -### Peer review -Checking work by people with similar skills and roles, to see that it meets its criteria. - -See also: [Review](#review). - -### Pen testing -**Penetration testing.** An authorised simulated attack on a system to find vulnerabilities that a real attacker could exploit. Testers may get full information about the system (white box) or none (black box). - -See also: [Ethical Hacking](d-f.md#ethical-hacking). - -### PER -**Packed Encoding Rules.** Compact ASN.1 encoding rules that use the limits in the schema to encode values in as few bits as possible. The decoder needs the schema. Mobile radio protocols such as RRC use PER. - -See also: [ASN.1](a-c.md#asn1), [BER](a-c.md#ber). - -### Performance -How well a system does what it does, such as speed, resource use, reliability and availability. The term is sometimes also used for the mix of features and quality a project delivers. - -See also: [Non-functional Requirement](l-n.md#non-functional-requirement). - -### Perl -A general-purpose scripting language created by Larry Wall in 1987, strong at text processing. Raku began as Perl 6 and became a separate language. - -See also: [Awk](a-c.md#awk), [Sed](s-z.md#sed). - -### Personas -Short, fictional profiles of typical users, used to keep their goals and needs in mind during design. - -See also: [User Experience](s-z.md#user-experience). - -### Perspective -In an inspection, a point of view given to an inspector, such as the tester's or the maintainer's, so that different inspectors find different issues. - -See also: [Inspection](g-k.md#inspection). - -### PERT chart - -**Program Evaluation and Review Technique.** - -A project planning technique, developed by the US Navy in the 1950s, that shows tasks as a network of dependencies. It estimates each task's duration from optimistic, likely and pessimistic values. - -See also: [CPM](a-c.md#cpm), [Gantt chart](g-k.md#gantt-chart). - -### PGP -**Pretty Good Privacy.** Encryption software, created by Phil Zimmermann in 1991, for signing and encrypting email and files. The OpenPGP standard (RFC 4880) describes its format. - -### PGW -**Packet Data Network Gateway.** The LTE core gateway that connects devices to external packet networks, such as the internet. It allocates IP addresses, enforces policy and charging, and anchors mobility. - -See also: [S-GW](s-z.md#s-gw), [PCRF](#pcrf), [GGSN](g-k.md#ggsn), [APN](a-c.md#apn). - -### Phase -A group of activities in a lifecycle, or a period in a schedule, often marking a major transition in a project. Sometimes used to mean a stage. - -See also: [Stage](s-z.md#stage), [Lifecycle](l-n.md#lifecycle). - -### Phased Estimation -Re-estimating throughout a project's lifecycle, using the techniques that best suit each phase. - -See also: [Estimation](d-f.md#estimation). - -### PHP -A server-side scripting language for the web, created by Rasmus Lerdorf in 1994. Its code can be embedded in HTML pages, and it powers many content management systems. - -### Planning and Tracking Lead -**PTL.** The person responsible for planning a project's technical work and tracking it: work breakdown, estimation, scheduling and progress. - -### Planning Poker -An agile estimation technique in which each team member chooses a card for their estimate and all reveal at once. Differences are then discussed and the round repeated until the estimates converge. - -See also: [Points](#points), [Relative Estimation](#relative-estimation). - -### PLL -**Phase-Locked Loop.** A control circuit that adjusts an oscillator so that its output stays in phase with a reference signal. PLLs are used for clock recovery, frequency synthesis and synchronisation. - -See also: [Jitter](g-k.md#jitter). - -### PNT -**Pointer.** Ordnance Survey of Northern Ireland's address database for Northern Ireland. - -See also: [PAF](#paf). - -### Points -**Story points.** Units for estimating the relative size or effort of user stories, rather than hours. - -See also: [Planning Poker](#planning-poker), [Velocity](s-z.md#velocity). - -### Polymorphism -The ability of one interface to work with different types, so that the same call does different things depending on the object it is made on. - -See also: [OO](#oo). - -### POP -**Point of Presence.** A location, such as a building or data centre, where a network provider has equipment and where others can connect to its network. - -See also: [ISP](g-k.md#isp). - -### Portability -How easily software can be moved to a different platform or operating system without major rework. - -See also: [POSIX](#posix). - -### POSIX -**Portable Operating System Interface.** IEEE standards that define the API, shell and utilities of Unix-like operating systems, so software can move between them. - -See also: [Unix](s-z.md#unix), [Sh](s-z.md#sh). - -### Postmortem -A review at the end of a project, and its report, in which the team looks at what went well and badly and draws lessons for next time. - -See also: [Milestone Retrospective](l-n.md#milestone-retrospective). - -### POTS -**Plain Old Telephone Service.** Basic analogue voice telephony over a copper line. - -See also: [PSTN](#pstn), [FXS](d-f.md#fxs). - -### PPP -**Point-to-Point Protocol.** A data link protocol for a direct link between two nodes, with authentication, link negotiation and network-layer configuration. Its variants, PPPoE and PPPoA, carry broadband sessions. - -See also: [PPPoE](#pppoe), [CHAP](a-c.md#chap), [PAP](#pap), [L2TP](l-n.md#l2tp). - -### PPPoE -**PPP over Ethernet.** PPP frames carried inside Ethernet frames, so broadband providers can authenticate users and manage sessions over Ethernet-based access networks. - -See also: [PPP](#ppp), [BNG](a-c.md#bng). - -### PPTP -**Point-to-Point Tunnelling Protocol.** An old VPN protocol that tunnels PPP over GRE with a TCP control channel. Its security is broken, so it should not be used. - -See also: [L2TP](l-n.md#l2tp), [IPSec](g-k.md#ipsec). - -### Preemption -Interrupting a running task, without its cooperation, to run another, intending to resume it later. A preemptive scheduler does this, typically to run a higher-priority task. - -See also: [Scheduler](s-z.md#scheduler), [RTOS](#rtos). - -### Preview -A meeting, with material sent out beforehand, to probe a proposed solution and suggest alternatives. Unlike a review, it looks for solutions rather than defects. - -See also: [Engineering Discussion](d-f.md#engineering-discussion), [Review](#review). - -### PRI -**Primary Rate Interface.** ISDN access for larger sites. On an E1 (2.048 Mbit/s) it has 30 bearer channels and one signalling channel; on a T1 (1.544 Mbit/s), 23 bearers and one signalling channel. BT sold it as ISDN30. - -See also: [ISDN](g-k.md#isdn), [BRI](a-c.md#bri), [ISDN30](g-k.md#isdn30), [Q.931](#q931). - -### PRINCE2 -**Projects in Controlled Environments.** A process-based project management method that originated in UK government and is widely used in the UK and Europe. - -See also: [CCTA](a-c.md#ccta). - -### Process -A standard way of doing an activity or task, whether written down or not. A defined process is documented. The word also refers to running the process, as in "the build process". - -See also: [Process Flow](#process-flow), [Engineering Process](d-f.md#engineering-process). - -### Process Flow -A model of a process that defines its steps, events, inputs and outputs. - -See also: [Process](#process). - -### Product Owner -The Scrum role responsible for maximising the value of the product: managing the product backlog and deciding what the team works on next. - -See also: [Scrum](s-z.md#scrum), [Backlog](a-c.md#backlog). - -### Product Release -A release made for distribution or deployment to end users. - -See also: [Release](#release). - -### Programming language -A formal language for writing instructions that a computer can carry out, after compilation or interpretation. - -See also: [Programming paradigm](#programming-paradigm), [Compiled Language](a-c.md#compiled-language), [Interpreted Language](g-k.md#interpreted-language). - -### Programming paradigm -A style of programming, and a way of classifying languages by the features that support it. Examples are imperative, procedural, object-oriented, functional, declarative and logic programming. Most modern languages support several. - -See also: [OO](#oo), [AOP](a-c.md#aop). - -### Progressive Elaboration -Defining a project's requirements and plans in more detail as the project goes on and more becomes known. The term comes from the PMBOK. - -See also: [Rolling Wave Planning](#rolling-wave-planning), [Project Headlights](#project-headlights). - -### Project -A temporary piece of work to create a unique product, service or result. - -See also: [Project Charter](#project-charter), [Project Plan](#project-plan). - -### Project Build -A build made through the project's shared build process, usually in a dedicated build environment. - -See also: [Local Build](l-n.md#local-build), [Build Environment](a-c.md#build-environment). - -### Project Business Manager -**PBM.** The person responsible for a project's business outcome: staffing, resources, people issues and dealing with the client. They make the top-level decisions and usually defer technical ones to the technical leads. - -See also: [Project Manager](#project-manager). - -### Project Charter -The document that starts a project, setting out its objectives, business case, risks, resources and constraints. - -See also: [Project Sponsor](#project-sponsor), [Fuzzy Front End](d-f.md#fuzzy-front-end). - -### Project Chartering -Summarising a project's key success factors on one sheet, often a flip chart on the team room wall. - -See also: [Project Charter](#project-charter). - -### Project Estimate -An estimate for a whole project, or a large part of one, usually using several techniques. - -See also: [Task Estimate](s-z.md#task-estimate), [Estimate](d-f.md#estimate). - -### Project Headlights -The idea that a project can only plan with confidence a limited distance ahead, like driving within the range of a car's headlights. Beyond that, plans are rougher. - -See also: [Rolling Wave Planning](#rolling-wave-planning), [Progressive Elaboration](#progressive-elaboration). - -### Project History -A summary of the significant information and statistics about a whole project. - -See also: [Project Log](#project-log), [Postmortem](#postmortem). - -### Project Log -A record of the significant information and statistics about each phase of a project. - -See also: [Project History](#project-history). - -### Project Management -**PM.** Planning, tracking, controlling and reporting on a project. - -See also: [PRINCE2](#prince2), [Project Management Body of Knowledge](#project-management-body-of-knowledge). - -### Project Management Body of Knowledge -**PMBOK.** The Project Management Institute's guide to the generally recognised knowledge and practices of project management. - -See also: [Project Management Institute](#project-management-institute). - -### Project Management Institute -**PMI.** A professional association for project managers, which publishes the PMBOK and runs certifications. - -See also: [Project Management Body of Knowledge](#project-management-body-of-knowledge). - -### Project Manager -**PM.** The person who manages a project. The role often combines those of the project business manager and the planning and tracking lead. - -See also: [Project Business Manager](#project-business-manager), [Planning and Tracking Lead](#planning-and-tracking-lead). - -### Project Plan -**PP.** The controlling document that defines how a project will be carried out. - -See also: [Project Charter](#project-charter), [Work Plan](s-z.md#work-plan). - -### Project Reviewer -Someone outside a project who reviews or audits it, to check that processes are being followed and risks managed. - -See also: [Audit](a-c.md#audit). - -### Project Sponsor -The person or organisation that sponsors a project: starts it, provides its resources, oversees its manager and makes sure it meets stakeholders' needs. - -See also: [Authority](a-c.md#authority), [Project Charter](#project-charter). - -### Pseudocode -An informal, readable description of an algorithm, written in structured natural language rather than a real programming language. - -### PSTN -**Public Switched Telephone Network.** The world's interconnected circuit-switched telephone networks, fixed and mobile. Its core is now digital, and many countries are moving its services to IP. - -See also: [POTS](#pots), [Circuit Switched](a-c.md#circuit-switched), [SS7](s-z.md#ss7). - -### PTP -**Precision Time Protocol.** The IEEE 1588 protocol for synchronising clocks across a network to sub-microsecond accuracy, with hardware timestamping. Mobile networks, finance and industrial control use it. - -See also: [NTP](l-n.md#ntp), [MRTIE](l-n.md#mrtie). - -### PTT -**Push-to-Talk.** Talking over a half-duplex channel by holding a button to transmit and releasing it to listen, as on a two-way radio. - -See also: [Half Duplex](g-k.md#half-duplex). - -### PVC -**Permanent Virtual Circuit.** A virtual circuit set up permanently between two endpoints, so no call set-up is needed. Frame Relay and ATM services were usually built from PVCs. - -See also: [SVC](s-z.md#svc), [Frame Relay](d-f.md#frame-relay), [ATM](a-c.md#atm). - -### PW -**Pseudowire.** The emulation of a point-to-point circuit, such as Ethernet, TDM or ATM, over a packet network, typically MPLS. - -See also: [VPWS](s-z.md#vpws), [L2VPN](l-n.md#l2vpn). - -### PWFQ -**Priority Weighted Fair Queuing.** A queuing scheme that serves strict-priority queues first and shares the remaining capacity between the other queues by weight. - -See also: [NGWFQ](l-n.md#ngwfq), [QoS](#qos). - -### Python -A high-level, general-purpose language created by Guido van Rossum and first released in 1991. It is known for readable code with significant indentation and a large standard library. Python 2 reached end of life in 2020, and Python 3 is the current version. - -See also: [Interpreted Language](g-k.md#interpreted-language). - -## Q - -### Q.931 {#q931} -The ITU-T ISDN signalling protocol on the D channel, which sets up, maintains and clears calls between the user and the network. H.323 and QSIG reuse its messages. - -See also: [ISDN](g-k.md#isdn), [PRI](#pri), [QSIG](#qsig). - -### QA Lead -**QAL.** The person responsible for the quality of a project's process and product. - -See also: [Quality Assurance](#quality-assurance). - -### QAM -**Quadrature Amplitude Modulation.** Modulation that sends data by varying the amplitude of two carriers 90° apart, so that each symbol carries several bits. Higher orders, such as 256-QAM, carry more bits but need a cleaner signal. DSL, cable, Wi-Fi and mobile networks use it. - -See also: [FSK](d-f.md#fsk), [OFDMA](#ofdma). - -### QoS -**Quality of Service.** The mechanisms that give different traffic different treatment in a network, through classification, marking, queuing and policing. They protect delay-sensitive traffic, such as voice, from loss, delay and jitter. The term also means the measured performance of a service. - -See also: [DSCP](d-f.md#dscp), [CoS](a-c.md#cos), [RSVP](#rsvp), [PWFQ](#pwfq). - -### QSIG -An ISDN-based signalling protocol between PBXs in a private network, standardised by Ecma and ISO. It supports basic calls and supplementary services across different vendors' equipment. - -See also: [DPNSS](d-f.md#dpnss), [Q.931](#q931). - -### Quality -How well a system and its artefacts meet their requirements and standards, and the activities that make sure they do. - -See also: [Quality Assurance](#quality-assurance), [Quality Control](#quality-control). - -### Quality Assurance -**QA.** Activities that prevent defects, by making sure the right processes are in place and followed. - -See also: [Quality Control](#quality-control), [Quality Plan](#quality-plan). - -### Quality Assurance Plan -See [Quality Plan](#quality-plan). - -### Quality Control -**QC.** Activities that find defects, mainly reviews and testing. - -See also: [Quality Assurance](#quality-assurance), [Testing](s-z.md#testing). - -### Quality Plan -**QP.** A plan for the quality methods and practices a project will use, and which artefacts they cover. It often leaves the testing details to the test plan. - -See also: [Test Plan](s-z.md#test-plan). - -### Quick Design Session -A short meeting, often at a whiteboard, where two or more developers settle a design choice that has wide consequences. - -See also: [Simple Design](s-z.md#simple-design). - -## R - -### R Language -A language and free environment for statistical computing and graphics, widely used by statisticians and data scientists. - -See also: [Python](#python). - -### RACI -**Responsible, Accountable, Consulted, Informed.** A responsibility matrix that sets out, for each task or deliverable, who does the work, who is accountable for it, who is consulted and who is kept informed. - -### RAD -**Rapid Application Development.** An approach that develops software quickly through prototypes, workshops, early user testing and component reuse, with less formal documentation. - -See also: [Rapid Application Development](#rapid-application-development), [DSDM](d-f.md#dsdm), [Joint Application Development](g-k.md#joint-application-development). - -### RAD Environment -A development tool that combines a language and an IDE with a toolbox of ready-made user interface components, for building applications quickly. - -See also: [IDE](g-k.md#ide), [RAD](#rad). - -### Radius -**RADIUS: Remote Authentication Dial-In User Service.** The protocol (RFC 2865) that network access servers use to authenticate users, authorise their access and record their usage against a central server. It is widely used for broadband, Wi-Fi and VPN access. - -See also: [NAS](l-n.md#nas), [EAP](d-f.md#eap), [FNA](d-f.md#fna). - -### RAN -**Radio Access Network.** The part of a mobile network between devices and the core: the base stations and their controllers. GERAN, UTRAN and E-UTRAN are the 2G, 3G and 4G versions. - -See also: [GERAN](g-k.md#geran), [UTRAN](s-z.md#utran), [E-UTRAN](d-f.md#e-utran), [Backhaul](a-c.md#backhaul). - -### RANAP -**Radio Access Network Application Part.** The UMTS signalling protocol on the Iu interface between the radio access network (the RNC) and the core (MSC or SGSN). - -See also: [RNC](#rnc), [UTRAN](s-z.md#utran). - -### RANCID -**Really Awesome New Cisco confIg Differ.** An open-source tool that logs in to network devices, collects their configurations, stores them in version control and reports changes. - -### Rapid Application Development -See [RAD](#rad). - -### RARP -**Reverse Address Resolution Protocol.** An obsolete protocol (RFC 903) with which a device that knows only its MAC address asks a server for its IP address. BOOTP and DHCP replaced it. - -See also: [BOOTP](a-c.md#bootp), [DHCP](d-f.md#dhcp). - -### RAS -**Registration, Admission and Status.** The part of H.323 (in H.225) with which endpoints register with a gatekeeper and ask permission to make calls. RAS also stands for remote access server. - -See also: [H.323](g-k.md#h323). - -### RDBMS -**Relational Database Management System.** A database system based on Edgar Codd's relational model, which stores data in tables with rows and columns and is queried with SQL. - -See also: [SQL](s-z.md#sql), [NoSQL](l-n.md#nosql). - -### RDP -**Remote Desktop Protocol.** Microsoft's protocol for using another computer's desktop over a network, on TCP and UDP port 3389. - -### Reading Inspection -An inspection that follows every step of the full process except the inspection meeting. - -See also: [Inspection](g-k.md#inspection). - -### Refactoring -Improving the internal structure of code without changing what it does, in small steps backed by tests. - -See also: [Technical Debt](s-z.md#technical-debt), [TDD](s-z.md#tdd). - -### Regression testing -Re-running tests after a change to check that what worked before still works. - -See also: [Automated Testing](a-c.md#automated-testing). - -### Regular expression -A pattern that describes a set of strings, used to search, match, validate and replace text. It is also called a regex. - -See also: [Sed](s-z.md#sed), [Awk](a-c.md#awk). - -### Relative Estimation -Estimating work by comparing items with each other, or grouping items of similar difficulty, rather than estimating each in absolute units. - -See also: [Points](#points), [T-Shirt Sizing](s-z.md#t-shirt-sizing). - -### Release -Delivering a deployable version of a system to a customer, such as a test group or end users. The word also means the version delivered. - -See also: [Release Management](#release-management), [Product Release](#product-release). - -### Release Management -Identifying, packaging and delivering the parts of a product to its customers, internal or external. - -See also: [Release](#release), [Semantic Versioning](s-z.md#semantic-versioning). - -### Release Test -A test that checks software is ready for release. - -### Requirement -A statement of something a system must do, or a quality it must have. - -See also: [Functional Requirement](d-f.md#functional-requirement), [Non-functional Requirement](l-n.md#non-functional-requirement). - -### Requirements -The work of finding out, recording and managing what a software system should do. - -See also: [Requirement](#requirement), [Software Requirements Specification](s-z.md#software-requirements-specification). - -### Requirements Lead -The person responsible for defining, maintaining and tracing a product's requirements, and for the user documentation that follows from them. - -### REST -**Representational State Transfer.** An architecture style for web services in which resources are identified by URLs and handled through a uniform set of stateless operations, usually the HTTP methods. `GET` reads, `POST` creates, `PUT` replaces and `DELETE` removes. - -See also: [HTTP](g-k.md#http), [SOAP](s-z.md#soap), [API](a-c.md#api). - -### Review -A defined process in which people other than an artefact's author examine it to find issues. Reviews range from informal desk checks to formal inspections. - -See also: [Inspection](g-k.md#inspection), [Desk Check](d-f.md#desk-check), [Walkthrough](s-z.md#walkthrough). - -### Reviewer -A person who takes part in a review. - -See also: [Review](#review). - -### Revision -One recorded version of an artefact, usually as numbered by a version control system. - -See also: [Version Control](s-z.md#version-control). - -### Revision Control -See [Version Control](s-z.md#version-control). - -### Rework -Unplanned work needed to bring an artefact into line with the project's needs, such as fixing defects. - -See also: [Update](s-z.md#update). - -### RFC -**Request for Comments.** The IETF's numbered document series, which includes the internet's standards and many informational and experimental documents. - -See also: [IETF](g-k.md#ietf). - -### RIP -**Routing Information Protocol.** One of the oldest distance-vector routing protocols. It uses hop count as its metric, with a limit of 15 hops, so it suits only small networks. - -See also: [OSPF](#ospf), [IGRP](g-k.md#igrp), [IGP](g-k.md#igp). - -### Risk -A possible future event that would harm a project's features, budget, schedule or quality if it happened. - -See also: [Risk Management](#risk-management), [Risk List](#risk-list). - -### Risk Analysis -Assessing each risk for its likelihood and impact. - -See also: [Risk Management](#risk-management). - -### Risk Identification -Finding and recording a project's current risks. - -See also: [Risk Management](#risk-management). - -### Risk List -A list of a project's current and past risks, with their analysis and mitigation plans. - -See also: [Risk Management](#risk-management), [Extrinsic Risk Management](d-f.md#extrinsic-risk-management). - -### Risk Management -Identifying, analysing and controlling a project's risks, either explicitly or as part of how the project is run. - -See also: [Risk](#risk), [Extrinsic Risk Management](d-f.md#extrinsic-risk-management), [Intrinsic Risk Management](g-k.md#intrinsic-risk-management), [Risk Mitigation](#risk-mitigation). - -### Risk Mitigation -Planning and taking action to reduce a risk's likelihood or impact. - -See also: [Risk Management](#risk-management). - -### Risk Reward Matrix -A two-by-two grid for comparing options, with risk on one axis and reward on the other. High-reward, low-risk options are the best candidates; high-risk, low-reward ones are to be avoided. The size of each marker can show the resources each option needs. - -### RNC -**Radio Network Controller.** The UMTS node that controls a group of Node Bs. It manages radio resources and handovers, and ciphers user data. It connects to the circuit-switched core through the Iu-CS interface and to the packet core through Iu-PS. - -See also: [Node B](l-n.md#node-b), [UTRAN](s-z.md#utran), [RANAP](#ranap), [BSC](a-c.md#bsc). - -### RNSAP -**Radio Network Subsystem Application Part.** The UMTS signalling protocol between RNCs, on the Iur interface, used for soft handover and moving a connection from one RNC to another. - -See also: [RNC](#rnc), [UTRAN](s-z.md#utran). - -### Role-feature-reason -The usual template for writing user stories: "As a *role*, I want *feature*, so that *reason*." - -See also: [User Stories](s-z.md#user-stories). - -### Rolling Wave Planning -Planning the near future in detail and later work only roughly, then adding detail as each period comes closer. - -See also: [Project Headlights](#project-headlights), [Progressive Elaboration](#progressive-elaboration). - -### Root of Trust -**RoT.** The hardware, firmware and keys that a device trusts without checking, because everything else is checked against them. It anchors secure boot, device identity and key storage, and is usually built into the chip or a secure element. - -See also: [RoT](#rot), [Secure Element](s-z.md#secure-element), [TrustZone](s-z.md#trustzone). - -### RoT -See [Root of Trust](#root-of-trust). - -### Round Robin -A scheduling method that gives each task or queue a turn in a fixed circular order, with equal time slices and no priorities. It is simple and no task is starved. - -See also: [Scheduler](s-z.md#scheduler), [Preemption](#preemption). - -### Router -A device that forwards packets between networks, choosing the next hop for each from its routing table, which it builds from configuration and routing protocols. - -See also: [Switch](s-z.md#switch), [Control Plane](a-c.md#control-plane), [BGP](a-c.md#bgp), [OSPF](#ospf). - -### RPC -**Remote Procedure Call.** Calling a procedure that runs in another process or on another machine as if it were a local call, with the messaging hidden by the framework. - -See also: [JSON-RPC](g-k.md#json-rpc), [XML-RPC](s-z.md#xml-rpc), [SOAP](s-z.md#soap). - -### RRC -**Radio Resource Control.** The 3GPP control protocol between a device and the radio network in UMTS, LTE and 5G. It sets up and releases connections and radio bearers, broadcasts system information and manages mobility. A state machine, with idle and connected states, governs the device. - -See also: [PDCP](#pdcp), [E-UTRAN](d-f.md#e-utran). - -### RSTP -**Rapid Spanning Tree Protocol.** IEEE 802.1w, a faster version of the spanning tree protocol that recovers from a topology change in seconds rather than tens of seconds. It is now part of 802.1D. - -See also: [STP](s-z.md#stp). - -### RSVP -**Resource Reservation Protocol.** A protocol (RFC 2205) with which receivers reserve network resources for a flow along its path. Its extension, RSVP-TE, sets up traffic-engineered MPLS paths. - -See also: [QoS](#qos), [LSP](l-n.md#lsp). - -### RTCP -**RTP Control Protocol.** The companion to RTP, defined in RFC 3550. It reports statistics on an RTP session, such as packet loss, jitter and round-trip time, and helps synchronise streams. - -See also: [RTP](#rtp). - -### RTOS -**Real-Time Operating System.** An operating system that guarantees tasks respond within defined time limits, usually by priority-based preemptive scheduling. FreeRTOS, Zephyr and VxWorks are examples. - -See also: [Scheduler](s-z.md#scheduler), [Preemption](#preemption), [MCU](l-n.md#mcu). - -### RTP -**Real-time Transport Protocol.** The protocol (RFC 3550) that carries audio and video over IP, usually on UDP, with sequence numbers and timestamps so receivers can play media out smoothly. SIP or another protocol sets up the session. - -See also: [RTCP](#rtcp), [SIP](s-z.md#sip), [VoIP](s-z.md#voip). - -### RTSP -**Real Time Streaming Protocol.** A protocol for controlling streaming media servers with commands such as play, pause and record. The media itself usually travels over RTP. - -See also: [RTP](#rtp). - -### Ruby -A dynamic, object-oriented scripting language created by Yukihiro Matsumoto in the mid-1990s, designed to make programmers productive and happy. - -See also: [Ruby on Rails](#ruby-on-rails). - -### Ruby on Rails -A web application framework written in Ruby, built on the model-view-controller pattern and known for "convention over configuration". - -See also: [Ruby](#ruby). - -### Rule of Simplicity -Kent Beck's rules, in priority order, for judging whether code is simple enough: it passes the tests, reveals its intent, has no duplication and has the fewest elements. - -See also: [Simple Design](s-z.md#simple-design), [KISS Principle](g-k.md#kiss-principle). - -### RUP -**Rational Unified Process.** An iterative software development process framework from Rational Software (later IBM), with four phases: inception, elaboration, construction and transition. - -See also: [UML](s-z.md#uml), [Iterative Development](g-k.md#iterative-development). diff --git a/content/glossary/o.md b/content/glossary/o.md new file mode 100644 index 0000000..3458802 --- /dev/null +++ b/content/glossary/o.md @@ -0,0 +1,97 @@ +--- +title: O +--- + +# O + +## Objective C +An object-oriented language that adds Smalltalk-style messaging to C. It was Apple's main language for macOS and iOS apps until Swift arrived. + +See also: [Swift](s.md#swift), [C Language](c.md#c-language). + +## Objectives +See [Goals](g.md#goals). + +## ODF +**Optical Distribution Frame.** A frame that terminates fibre cables and lets connections between them be made and changed with patch leads. + +See also: [MDF](m.md#mdf), [OFC](#ofc). + +## OFC +**Optical Fibre Connector.** A connector that joins optical fibres, aligning their cores so light passes with little loss. It is quicker to connect and disconnect than a splice. Common types include LC, SC and ST. + +![OFC](images/15008720.png) + +[[CAPTION:Figure]] OFC. + +See also: [ODF](#odf). + +## Ofcom +**Office of Communications.** The UK regulator for broadcasting, telecoms, post and the radio spectrum, set up under the Communications Act 2003. + +## OFDMA +**Orthogonal Frequency-Division Multiple Access.** The multi-user form of OFDM, in which different users get different sets of closely spaced subcarriers at the same time. LTE downlinks, 5G and Wi-Fi 6 use it. + +See also: [SC-FDMA](s.md#sc-fdma), [COFDM](c.md#cofdm), [LTE (4G)](l.md#lte-4g). + +## OO +**Object-Oriented.** A paradigm that organises software around objects, which bundle data with the code that acts on it. Most OO languages are class-based, with inheritance and polymorphism. + +See also: [Programming paradigm](p.md#programming-paradigm), [Polymorphism](p.md#polymorphism), [UML](u.md#uml). + +## Operations Plan +A plan for running a system after deployment: roles, responsibilities, procedures and maintenance schedules. + +See also: [Deployment Plan](d.md#deployment-plan), [Maintenance Plan](m.md#maintenance-plan). + +## OPUS +**Opus.** An open, royalty-free audio codec, standardised by the IETF in RFC 6716. It combines the SILK speech coder with the CELT low-delay coder and adapts from narrowband speech to full-band music. It is the standard codec for WebRTC. + +See also: [SPEEX](s.md#speex), [Vorbis](v.md#vorbis), [Audio Codecs](a.md#audio-codecs). + +## OS X +**macOS.** Apple's operating system for Mac computers, released in 2001 as Mac OS X and renamed OS X and then macOS. It is a Unix system built on the Darwin kernel. + +See also: [Unix](u.md#unix). + +## OSA-FC +**Optical Spectrum Access Filter Connect.** An Openreach service that gives a provider a managed fibre connection on which it runs its own optical equipment. It works like dark fibre, but Openreach monitors and repairs the link. + +![OSA-FC](images/15008728.png) + +[[CAPTION:Figure]] OSA-FC. + +See also: [Dark Fibre](d.md#dark-fibre), [EAD](e.md#ead). + +## OSPF +**Open Shortest Path First.** A link-state interior gateway protocol: each router learns the whole topology of its area and calculates the shortest paths. Version 2 (RFC 2328) is for IPv4 and version 3 for IPv6. + +See also: [IGP](i.md#igp), [RIP](r.md#rip), [BGP](b.md#bgp). + +## OSS +**Operations Support Systems.** The systems a telecoms operator uses to run its network, such as inventory, provisioning, configuration, fault and performance management. They work with the business support systems (BSS). + +![OSS](images/15008734.png) + +[[CAPTION:Figure]] OSS. + +See also: [BSS](b.md#bss), [Mediation](m.md#mediation). + +## OTN +**Optical Transport Network.** The ITU-T G.709 standard for carrying client signals, such as Ethernet or SDH, over wavelengths. It adds framing, forward error correction and management to optical networks. + +See also: [SDH](s.md#sdh). + +## Out-of-order delivery +Packets arriving in a different order from the one in which they were sent, for example because they took different paths. TCP puts the data back in order; real-time protocols must cope with it. + +![Out-of-order delivery](images/15008740.png) + +[[CAPTION:Figure]] Out-of-order delivery. + +See also: [TCP](t.md#tcp), [Jitter buffer](j.md#jitter-buffer). + +## Overlap Dialling +Sending a dialled number in pieces as the caller enters it, rather than all at once, so that routing can begin before the number is complete. The far end waits for more digits until a timer runs out. + +See also: [ISUP](i.md#isup), [Q.931](q.md#q931). diff --git a/content/glossary/p.md b/content/glossary/p.md new file mode 100644 index 0000000..1f56b44 --- /dev/null +++ b/content/glossary/p.md @@ -0,0 +1,463 @@ +--- +title: P +--- + +# P + +## PaaS +**Platform as a Service.** Cloud computing that provides a managed platform for building and running applications, so the customer does not manage servers or operating systems. + +![PaaS](images/14909529.png) + +[[CAPTION:Figure]] PaaS. + +See also: [IaaS](i.md#iaas), [SaaS](s.md#saas). + +## Packet sniffing +Capturing and decoding the packets crossing a network, usually with a packet analyser such as Wireshark, to troubleshoot problems or, in an attack, to steal data. + +See also: [Wireshark](w.md#wireshark), [DPI](d.md#dpi). + +## Packet Switched +A network that splits data into packets, each with a header that routers use to forward it independently, sharing links with other traffic. The internet works this way. + +![Packet Switched](images/15008750.png) + +[[CAPTION:Figure]] Packet Switched. + +See also: [Circuit Switched](c.md#circuit-switched). + +## PAF +**Postcode Address File.** Royal Mail's database of every UK delivery address and its postcode. + +## Pair Programming +Two developers working together at one computer: one writes the code while the other reviews it and thinks ahead, and they swap often. + +See also: [Mob Programming](m.md#mob-programming), [XP](x.md#xp). + +## PAP +**Password Authentication Protocol.** A simple PPP authentication method that sends the username and password in clear text. It should be used only when nothing stronger, such as CHAP or EAP, is available. + +![PAP](images/15008758.png) + +[[CAPTION:Figure]] PAP. + +See also: [CHAP](c.md#chap), [PPP](#ppp). + +## Parametric Estimation +Estimating with a model that calculates effort, cost or schedule from input factors, using formulas based on historical data. COCOMO is an example. + +See also: [Estimation](e.md#estimation), [Statistical Estimation](s.md#statistical-estimation). + +## Pascal +A procedural language designed by Niklaus Wirth around 1970 to teach structured programming. Object Pascal and Delphi descend from it. + +See also: [Delphi](d.md#delphi), [Ada](a.md#ada). + +## Pattern +A reusable template or model for creating an artefact or solving a recurring problem. + +See also: [Design Pattern](d.md#design-pattern), [Antipattern](a.md#antipattern). + +## PCM +**Pulse-Code Modulation.** The standard way of representing an analogue signal digitally: sample it at regular intervals and quantise each sample to a binary value. Telephone speech uses 8,000 samples a second at 8 bits (G.711), and CDs use 44,100 at 16 bits. + +See also: [G.711](g.md#g711), [A-law](a.md#a-law), [Mu-law](m.md#mu-law). + +## PCP +**Primary Connection Point.** The green roadside cabinet in the UK access network where the main cables from the exchange connect to the cables to local distribution points. Fibre-to-the-cabinet equipment sits next to it. + +![PCP](images/15008764.png) + +[[CAPTION:Figure]] PCP. + +In Ethernet, PCP also means the **Priority Code Point**: the 3-bit priority field in an 802.1Q tag. + +See also: [DP](d.md#dp), [VDSL](v.md#vdsl), [Dot1Q](d.md#dot1q). + +## PCRF +**Policy and Charging Rules Function.** The node in a mobile core that decides, in real time, the policy and charging rules for each subscriber's data sessions, such as quality of service and usage limits. The gateways then enforce them. + +See also: [PGW](#pgw), [QoS](q.md#qos). + +## PCS +**Physical Coding Sublayer.** The top sublayer of the Ethernet physical layer, which encodes and decodes data, for example with 8b/10b or 64b/66b coding, and scrambles it. + +See also: [L1](l.md#l1). + +## PDCP +**Packet Data Convergence Protocol.** A 3GPP radio protocol layer, in UMTS, LTE and 5G, that compresses headers, ciphers data and delivers packets in order between the device and the base station. + +See also: [RRC](r.md#rrc), [E-UTRAN](e.md#e-utran). + +## PDH +**Plesiochronous Digital Hierarchy.** The older digital transmission hierarchy. Links such as E1 (2 Mbit/s) and E3 run at nearly, but not exactly, the same rates and are multiplexed by bit stuffing. SDH replaced it in backbones. + +See also: [SDH](s.md#sdh), [TDM](t.md#tdm). + +## PDU +**Protocol Data Unit.** The unit of data a protocol layer exchanges with its peer. It is a frame at layer 2, a packet at layer 3, and a segment (TCP) or datagram (UDP) at layer 4. + +![PDU](images/15008770.png) + +[[CAPTION:Figure]] PDU. + +See also: [MTU](m.md#mtu). + +## PEAP +**Protected EAP.** An EAP method that sets up a TLS tunnel first and then runs a simpler authentication, often a password, inside it. It is common for Wi-Fi in organisations. + +![PEAP](images/15008776.png) + +[[CAPTION:Figure]] PEAP. + +See also: [EAP](e.md#eap), [TLS](t.md#tls). + +## Peer review +Checking work by people with similar skills and roles, to see that it meets its criteria. + +See also: [Review](r.md#review). + +## Pen testing +**Penetration testing.** An authorised simulated attack on a system to find vulnerabilities that a real attacker could exploit. Testers may get full information about the system (white box) or none (black box). + +See also: [Ethical Hacking](e.md#ethical-hacking). + +## PER +**Packed Encoding Rules.** Compact ASN.1 encoding rules that use the limits in the schema to encode values in as few bits as possible. The decoder needs the schema. Mobile radio protocols such as RRC use PER. + +See also: [ASN.1](a.md#asn1), [BER](b.md#ber). + +## Performance +How well a system does what it does, such as speed, resource use, reliability and availability. The term is sometimes also used for the mix of features and quality a project delivers. + +See also: [Non-functional Requirement](n.md#non-functional-requirement). + +## Perl +A general-purpose scripting language created by Larry Wall in 1987, strong at text processing. Raku began as Perl 6 and became a separate language. + +See also: [Awk](a.md#awk), [Sed](s.md#sed). + +## Personas +Short, fictional profiles of typical users, used to keep their goals and needs in mind during design. + +See also: [User Experience](u.md#user-experience). + +## Perspective +In an inspection, a point of view given to an inspector, such as the tester's or the maintainer's, so that different inspectors find different issues. + +See also: [Inspection](i.md#inspection). + +## PERT chart + +**Program Evaluation and Review Technique.** + +A project planning technique, developed by the US Navy in the 1950s, that shows tasks as a network of dependencies. It estimates each task's duration from optimistic, likely and pessimistic values. + +![PERT chart](images/15008782.png) + +[[CAPTION:Figure]] PERT chart. + +See also: [CPM](c.md#cpm), [Gantt chart](g.md#gantt-chart). + +## PGP +**Pretty Good Privacy.** Encryption software, created by Phil Zimmermann in 1991, for signing and encrypting email and files. The OpenPGP standard (RFC 4880) describes its format. + +![PGP](images/15008788.png) + +[[CAPTION:Figure]] PGP. + +## PGW +**Packet Data Network Gateway.** The LTE core gateway that connects devices to external packet networks, such as the internet. It allocates IP addresses, enforces policy and charging, and anchors mobility. + +![PGW](images/14876769.png) + +[[CAPTION:Figure]] PGW. + +See also: [S-GW](s.md#s-gw), [PCRF](#pcrf), [GGSN](g.md#ggsn), [APN](a.md#apn). + +## Phase +A group of activities in a lifecycle, or a period in a schedule, often marking a major transition in a project. Sometimes used to mean a stage. + +See also: [Stage](s.md#stage), [Lifecycle](l.md#lifecycle). + +## Phased Estimation +Re-estimating throughout a project's lifecycle, using the techniques that best suit each phase. + +See also: [Estimation](e.md#estimation). + +## PHP +A server-side scripting language for the web, created by Rasmus Lerdorf in 1994. Its code can be embedded in HTML pages, and it powers many content management systems. + +## Planning and Tracking Lead +**PTL.** The person responsible for planning a project's technical work and tracking it: work breakdown, estimation, scheduling and progress. + +## Planning Poker +An agile estimation technique in which each team member chooses a card for their estimate and all reveal at once. Differences are then discussed and the round repeated until the estimates converge. + +See also: [Points](#points), [Relative Estimation](r.md#relative-estimation). + +## PLL +**Phase-Locked Loop.** A control circuit that adjusts an oscillator so that its output stays in phase with a reference signal. PLLs are used for clock recovery, frequency synthesis and synchronisation. + +See also: [Jitter](j.md#jitter). + +## PNT +**Pointer.** Ordnance Survey of Northern Ireland's address database for Northern Ireland. + +See also: [PAF](#paf). + +## Points +**Story points.** Units for estimating the relative size or effort of user stories, rather than hours. + +See also: [Planning Poker](#planning-poker), [Velocity](v.md#velocity). + +## Polymorphism +The ability of one interface to work with different types, so that the same call does different things depending on the object it is made on. + +See also: [OO](o.md#oo). + +## POP +**Point of Presence.** A location, such as a building or data centre, where a network provider has equipment and where others can connect to its network. + +![POP](images/15008794.png) + +[[CAPTION:Figure]] POP. + +See also: [ISP](i.md#isp). + +## Portability +How easily software can be moved to a different platform or operating system without major rework. + +See also: [POSIX](#posix). + +## POSIX +**Portable Operating System Interface.** IEEE standards that define the API, shell and utilities of Unix-like operating systems, so software can move between them. + +See also: [Unix](u.md#unix), [Sh](s.md#sh). + +## Postmortem +A review at the end of a project, and its report, in which the team looks at what went well and badly and draws lessons for next time. + +See also: [Milestone Retrospective](m.md#milestone-retrospective). + +## POTS +**Plain Old Telephone Service.** Basic analogue voice telephony over a copper line. + +![POTS](images/15008800.png) + +[[CAPTION:Figure]] POTS. + +See also: [PSTN](#pstn), [FXS](f.md#fxs). + +## PPP +**Point-to-Point Protocol.** A data link protocol for a direct link between two nodes, with authentication, link negotiation and network-layer configuration. Its variants, PPPoE and PPPoA, carry broadband sessions. + +See also: [PPPoE](#pppoe), [CHAP](c.md#chap), [PAP](#pap), [L2TP](l.md#l2tp). + +## PPPoE +**PPP over Ethernet.** PPP frames carried inside Ethernet frames, so broadband providers can authenticate users and manage sessions over Ethernet-based access networks. + +See also: [PPP](#ppp), [BNG](b.md#bng). + +## PPTP +**Point-to-Point Tunnelling Protocol.** An old VPN protocol that tunnels PPP over GRE with a TCP control channel. Its security is broken, so it should not be used. + +![PPTP](images/15008806.png) + +[[CAPTION:Figure]] PPTP. + +See also: [L2TP](l.md#l2tp), [IPSec](i.md#ipsec). + +## Preemption +Interrupting a running task, without its cooperation, to run another, intending to resume it later. A preemptive scheduler does this, typically to run a higher-priority task. + +See also: [Scheduler](s.md#scheduler), [RTOS](r.md#rtos). + +## Preview +A meeting, with material sent out beforehand, to probe a proposed solution and suggest alternatives. Unlike a review, it looks for solutions rather than defects. + +See also: [Engineering Discussion](e.md#engineering-discussion), [Review](r.md#review). + +## PRI +**Primary Rate Interface.** ISDN access for larger sites. On an E1 (2.048 Mbit/s) it has 30 bearer channels and one signalling channel; on a T1 (1.544 Mbit/s), 23 bearers and one signalling channel. BT sold it as ISDN30. + +See also: [ISDN](i.md#isdn), [BRI](b.md#bri), [ISDN30](i.md#isdn30), [Q.931](q.md#q931). + +## PRINCE2 +**Projects in Controlled Environments.** A process-based project management method that originated in UK government and is widely used in the UK and Europe. + +See also: [CCTA](c.md#ccta). + +## Process +A standard way of doing an activity or task, whether written down or not. A defined process is documented. The word also refers to running the process, as in "the build process". + +See also: [Process Flow](#process-flow), [Engineering Process](e.md#engineering-process). + +## Process Flow +A model of a process that defines its steps, events, inputs and outputs. + +See also: [Process](#process). + +## Product Owner +The Scrum role responsible for maximising the value of the product: managing the product backlog and deciding what the team works on next. + +See also: [Scrum](s.md#scrum), [Backlog](b.md#backlog). + +## Product Release +A release made for distribution or deployment to end users. + +See also: [Release](r.md#release). + +## Programming language +A formal language for writing instructions that a computer can carry out, after compilation or interpretation. + +![Programming language](images/15008812.png) + +[[CAPTION:Figure]] Programming language. + +![Programming language](images/15008818.png) + +[[CAPTION:Figure]] Programming language. + +See also: [Programming paradigm](#programming-paradigm), [Compiled Language](c.md#compiled-language), [Interpreted Language](i.md#interpreted-language). + +## Programming paradigm +A style of programming, and a way of classifying languages by the features that support it. Examples are imperative, procedural, object-oriented, functional, declarative and logic programming. Most modern languages support several. + +![Programming paradigm](images/15008824.png) + +[[CAPTION:Figure]] Programming paradigm. + +See also: [OO](o.md#oo), [AOP](a.md#aop). + +## Progressive Elaboration +Defining a project's requirements and plans in more detail as the project goes on and more becomes known. The term comes from the PMBOK. + +See also: [Rolling Wave Planning](r.md#rolling-wave-planning), [Project Headlights](#project-headlights). + +## Project +A temporary piece of work to create a unique product, service or result. + +See also: [Project Charter](#project-charter), [Project Plan](#project-plan). + +## Project Build +A build made through the project's shared build process, usually in a dedicated build environment. + +See also: [Local Build](l.md#local-build), [Build Environment](b.md#build-environment). + +## Project Business Manager +**PBM.** The person responsible for a project's business outcome: staffing, resources, people issues and dealing with the client. They make the top-level decisions and usually defer technical ones to the technical leads. + +See also: [Project Manager](#project-manager). + +## Project Charter +The document that starts a project, setting out its objectives, business case, risks, resources and constraints. + +See also: [Project Sponsor](#project-sponsor), [Fuzzy Front End](f.md#fuzzy-front-end). + +## Project Chartering +Summarising a project's key success factors on one sheet, often a flip chart on the team room wall. + +See also: [Project Charter](#project-charter). + +## Project Estimate +An estimate for a whole project, or a large part of one, usually using several techniques. + +See also: [Task Estimate](t.md#task-estimate), [Estimate](e.md#estimate). + +## Project Headlights +The idea that a project can only plan with confidence a limited distance ahead, like driving within the range of a car's headlights. Beyond that, plans are rougher. + +See also: [Rolling Wave Planning](r.md#rolling-wave-planning), [Progressive Elaboration](#progressive-elaboration). + +## Project History +A summary of the significant information and statistics about a whole project. + +See also: [Project Log](#project-log), [Postmortem](#postmortem). + +## Project Log +A record of the significant information and statistics about each phase of a project. + +See also: [Project History](#project-history). + +## Project Management +**PM.** Planning, tracking, controlling and reporting on a project. + +See also: [PRINCE2](#prince2), [Project Management Body of Knowledge](#project-management-body-of-knowledge). + +## Project Management Body of Knowledge +**PMBOK.** The Project Management Institute's guide to the generally recognised knowledge and practices of project management. + +See also: [Project Management Institute](#project-management-institute). + +## Project Management Institute +**PMI.** A professional association for project managers, which publishes the PMBOK and runs certifications. + +See also: [Project Management Body of Knowledge](#project-management-body-of-knowledge). + +## Project Manager +**PM.** The person who manages a project. The role often combines those of the project business manager and the planning and tracking lead. + +See also: [Project Business Manager](#project-business-manager), [Planning and Tracking Lead](#planning-and-tracking-lead). + +## Project Plan +**PP.** The controlling document that defines how a project will be carried out. + +See also: [Project Charter](#project-charter), [Work Plan](w.md#work-plan). + +## Project Reviewer +Someone outside a project who reviews or audits it, to check that processes are being followed and risks managed. + +See also: [Audit](a.md#audit). + +## Project Sponsor +The person or organisation that sponsors a project: starts it, provides its resources, oversees its manager and makes sure it meets stakeholders' needs. + +See also: [Authority](a.md#authority), [Project Charter](#project-charter). + +## Pseudocode +An informal, readable description of an algorithm, written in structured natural language rather than a real programming language. + +## PSTN +**Public Switched Telephone Network.** The world's interconnected circuit-switched telephone networks, fixed and mobile. Its core is now digital, and many countries are moving its services to IP. + +![PSTN](images/15008830.png) + +[[CAPTION:Figure]] PSTN. + +See also: [POTS](#pots), [Circuit Switched](c.md#circuit-switched), [SS7](s.md#ss7). + +## PTP +**Precision Time Protocol.** The IEEE 1588 protocol for synchronising clocks across a network to sub-microsecond accuracy, with hardware timestamping. Mobile networks, finance and industrial control use it. + +See also: [NTP](n.md#ntp), [MRTIE](m.md#mrtie). + +## PTT +**Push-to-Talk.** Talking over a half-duplex channel by holding a button to transmit and releasing it to listen, as on a two-way radio. + +See also: [Half Duplex](h.md#half-duplex). + +## PVC +**Permanent Virtual Circuit.** A virtual circuit set up permanently between two endpoints, so no call set-up is needed. Frame Relay and ATM services were usually built from PVCs. + +See also: [SVC](s.md#svc), [Frame Relay](f.md#frame-relay), [ATM](a.md#atm). + +## PW +**Pseudowire.** The emulation of a point-to-point circuit, such as Ethernet, TDM or ATM, over a packet network, typically MPLS. + +![PW](images/15008836.png) + +[[CAPTION:Figure]] PW. + +See also: [VPWS](v.md#vpws), [L2VPN](l.md#l2vpn). + +## PWFQ +**Priority Weighted Fair Queuing.** A queuing scheme that serves strict-priority queues first and shares the remaining capacity between the other queues by weight. + +See also: [NGWFQ](n.md#ngwfq), [QoS](q.md#qos). + +## Python +A high-level, general-purpose language created by Guido van Rossum and first released in 1991. It is known for readable code with significant indentation and a large standard library. Python 2 reached end of life in 2020, and Python 3 is the current version. + +See also: [Interpreted Language](i.md#interpreted-language). diff --git a/content/glossary/q.md b/content/glossary/q.md new file mode 100644 index 0000000..ab78926 --- /dev/null +++ b/content/glossary/q.md @@ -0,0 +1,58 @@ +--- +title: Q +--- + +# Q + +## Q.931 {#q931} +The ITU-T ISDN signalling protocol on the D channel, which sets up, maintains and clears calls between the user and the network. H.323 and QSIG reuse its messages. + +See also: [ISDN](i.md#isdn), [PRI](p.md#pri), [QSIG](#qsig). + +## QA Lead +**QAL.** The person responsible for the quality of a project's process and product. + +See also: [Quality Assurance](#quality-assurance). + +## QAM +**Quadrature Amplitude Modulation.** Modulation that sends data by varying the amplitude of two carriers 90° apart, so that each symbol carries several bits. Higher orders, such as 256-QAM, carry more bits but need a cleaner signal. DSL, cable, Wi-Fi and mobile networks use it. + +See also: [FSK](f.md#fsk), [OFDMA](o.md#ofdma). + +## QoS +**Quality of Service.** The mechanisms that give different traffic different treatment in a network, through classification, marking, queuing and policing. They protect delay-sensitive traffic, such as voice, from loss, delay and jitter. The term also means the measured performance of a service. + +See also: [DSCP](d.md#dscp), [CoS](c.md#cos), [RSVP](r.md#rsvp), [PWFQ](p.md#pwfq). + +## QSIG +An ISDN-based signalling protocol between PBXs in a private network, standardised by Ecma and ISO. It supports basic calls and supplementary services across different vendors' equipment. + +See also: [DPNSS](d.md#dpnss), [Q.931](#q931). + +## Quality +How well a system and its artefacts meet their requirements and standards, and the activities that make sure they do. + +See also: [Quality Assurance](#quality-assurance), [Quality Control](#quality-control). + +## Quality Assurance +**QA.** Activities that prevent defects, by making sure the right processes are in place and followed. + +See also: [Quality Control](#quality-control), [Quality Plan](#quality-plan). + +## Quality Assurance Plan +See [Quality Plan](#quality-plan). + +## Quality Control +**QC.** Activities that find defects, mainly reviews and testing. + +See also: [Quality Assurance](#quality-assurance), [Testing](t.md#testing). + +## Quality Plan +**QP.** A plan for the quality methods and practices a project will use, and which artefacts they cover. It often leaves the testing details to the test plan. + +See also: [Test Plan](t.md#test-plan). + +## Quick Design Session +A short meeting, often at a whiteboard, where two or more developers settle a design choice that has wide consequences. + +See also: [Simple Design](s.md#simple-design). diff --git a/content/glossary/r.md b/content/glossary/r.md new file mode 100644 index 0000000..fdef46b --- /dev/null +++ b/content/glossary/r.md @@ -0,0 +1,330 @@ +--- +title: R +--- + +# R + +## R Language +A language and free environment for statistical computing and graphics, widely used by statisticians and data scientists. + +See also: [Python](p.md#python). + +## RACI +**Responsible, Accountable, Consulted, Informed.** A responsibility matrix that sets out, for each task or deliverable, who does the work, who is accountable for it, who is consulted and who is kept informed. + +![RACI](images/15008851.png) + +[[CAPTION:Figure]] RACI. + +## RAD +**Rapid Application Development.** An approach that develops software quickly through prototypes, workshops, early user testing and component reuse, with less formal documentation. + +See also: [Rapid Application Development](#rapid-application-development), [DSDM](d.md#dsdm), [Joint Application Development](j.md#joint-application-development). + +## RAD Environment +A development tool that combines a language and an IDE with a toolbox of ready-made user interface components, for building applications quickly. + +See also: [IDE](i.md#ide), [RAD](#rad). + +## Radius +**RADIUS: Remote Authentication Dial-In User Service.** The protocol (RFC 2865) that network access servers use to authenticate users, authorise their access and record their usage against a central server. It is widely used for broadband, Wi-Fi and VPN access. + +![Radius](images/15008859.png) + +[[CAPTION:Figure]] Radius. + +See also: [NAS](n.md#nas), [EAP](e.md#eap), [FNA](f.md#fna). + +## RAN +**Radio Access Network.** The part of a mobile network between devices and the core: the base stations and their controllers. GERAN, UTRAN and E-UTRAN are the 2G, 3G and 4G versions. + +![RAN](images/15008865.png) + +[[CAPTION:Figure]] RAN. + +See also: [GERAN](g.md#geran), [UTRAN](u.md#utran), [E-UTRAN](e.md#e-utran), [Backhaul](b.md#backhaul). + +## RANAP +**Radio Access Network Application Part.** The UMTS signalling protocol on the Iu interface between the radio access network (the RNC) and the core (MSC or SGSN). + +See also: [RNC](#rnc), [UTRAN](u.md#utran). + +## RANCID +**Really Awesome New Cisco confIg Differ.** An open-source tool that logs in to network devices, collects their configurations, stores them in version control and reports changes. + +## Rapid Application Development +See [RAD](#rad). + +## RARP +**Reverse Address Resolution Protocol.** An obsolete protocol (RFC 903) with which a device that knows only its MAC address asks a server for its IP address. BOOTP and DHCP replaced it. + +See also: [BOOTP](b.md#bootp), [DHCP](d.md#dhcp). + +## RAS +**Registration, Admission and Status.** The part of H.323 (in H.225) with which endpoints register with a gatekeeper and ask permission to make calls. RAS also stands for remote access server. + +See also: [H.323](h.md#h323). + +## RDBMS +**Relational Database Management System.** A database system based on Edgar Codd's relational model, which stores data in tables with rows and columns and is queried with SQL. + +![RDBMS](images/15008871.png) + +[[CAPTION:Figure]] RDBMS. + +See also: [SQL](s.md#sql), [NoSQL](n.md#nosql). + +## RDP +**Remote Desktop Protocol.** Microsoft's protocol for using another computer's desktop over a network, on TCP and UDP port 3389. + +## Reading Inspection +An inspection that follows every step of the full process except the inspection meeting. + +See also: [Inspection](i.md#inspection). + +## Refactoring +Improving the internal structure of code without changing what it does, in small steps backed by tests. + +See also: [Technical Debt](t.md#technical-debt), [TDD](t.md#tdd). + +## Regression testing +Re-running tests after a change to check that what worked before still works. + +See also: [Automated Testing](a.md#automated-testing). + +## Regular expression +A pattern that describes a set of strings, used to search, match, validate and replace text. It is also called a regex. + +See also: [Sed](s.md#sed), [Awk](a.md#awk). + +## Relative Estimation +Estimating work by comparing items with each other, or grouping items of similar difficulty, rather than estimating each in absolute units. + +See also: [Points](p.md#points), [T-Shirt Sizing](t.md#t-shirt-sizing). + +## Release +Delivering a deployable version of a system to a customer, such as a test group or end users. The word also means the version delivered. + +See also: [Release Management](#release-management), [Product Release](p.md#product-release). + +## Release Management +Identifying, packaging and delivering the parts of a product to its customers, internal or external. + +See also: [Release](#release), [Semantic Versioning](s.md#semantic-versioning). + +## Release Test +A test that checks software is ready for release. + +## Requirement +A statement of something a system must do, or a quality it must have. + +See also: [Functional Requirement](f.md#functional-requirement), [Non-functional Requirement](n.md#non-functional-requirement). + +## Requirements +The work of finding out, recording and managing what a software system should do. + +See also: [Requirement](#requirement), [Software Requirements Specification](s.md#software-requirements-specification). + +## Requirements Lead +The person responsible for defining, maintaining and tracing a product's requirements, and for the user documentation that follows from them. + +## REST +**Representational State Transfer.** An architecture style for web services in which resources are identified by URLs and handled through a uniform set of stateless operations, usually the HTTP methods. `GET` reads, `POST` creates, `PUT` replaces and `DELETE` removes. + +![REST](images/15008877.png) + +[[CAPTION:Figure]] REST. + +See also: [HTTP](h.md#http), [SOAP](s.md#soap), [API](a.md#api). + +## Review +A defined process in which people other than an artefact's author examine it to find issues. Reviews range from informal desk checks to formal inspections. + +See also: [Inspection](i.md#inspection), [Desk Check](d.md#desk-check), [Walkthrough](w.md#walkthrough). + +## Reviewer +A person who takes part in a review. + +See also: [Review](#review). + +## Revision +One recorded version of an artefact, usually as numbered by a version control system. + +See also: [Version Control](v.md#version-control). + +## Revision Control +See [Version Control](v.md#version-control). + +## Rework +Unplanned work needed to bring an artefact into line with the project's needs, such as fixing defects. + +See also: [Update](u.md#update). + +## RFC +**Request for Comments.** The IETF's numbered document series, which includes the internet's standards and many informational and experimental documents. + +See also: [IETF](i.md#ietf). + +## RIP +**Routing Information Protocol.** One of the oldest distance-vector routing protocols. It uses hop count as its metric, with a limit of 15 hops, so it suits only small networks. + +See also: [OSPF](o.md#ospf), [IGRP](i.md#igrp), [IGP](i.md#igp). + +## Risk +A possible future event that would harm a project's features, budget, schedule or quality if it happened. + +See also: [Risk Management](#risk-management), [Risk List](#risk-list). + +## Risk Analysis +Assessing each risk for its likelihood and impact. + +See also: [Risk Management](#risk-management). + +## Risk Identification +Finding and recording a project's current risks. + +See also: [Risk Management](#risk-management). + +## Risk List +A list of a project's current and past risks, with their analysis and mitigation plans. + +See also: [Risk Management](#risk-management), [Extrinsic Risk Management](e.md#extrinsic-risk-management). + +## Risk Management +Identifying, analysing and controlling a project's risks, either explicitly or as part of how the project is run. + +See also: [Risk](#risk), [Extrinsic Risk Management](e.md#extrinsic-risk-management), [Intrinsic Risk Management](i.md#intrinsic-risk-management), [Risk Mitigation](#risk-mitigation). + +## Risk Mitigation +Planning and taking action to reduce a risk's likelihood or impact. + +See also: [Risk Management](#risk-management). + +## Risk Reward Matrix +A two-by-two grid for comparing options, with risk on one axis and reward on the other. High-reward, low-risk options are the best candidates; high-risk, low-reward ones are to be avoided. The size of each marker can show the resources each option needs. + +![Risk Reward Matrix](images/15008883.png) + +[[CAPTION:Figure]] Risk Reward Matrix. + +![Risk Reward Matrix](images/15008889.png) + +[[CAPTION:Figure]] Risk Reward Matrix. + +![Risk Reward Matrix](images/15008895.png) + +[[CAPTION:Figure]] Risk Reward Matrix. + +## RNC +**Radio Network Controller.** The UMTS node that controls a group of Node Bs. It manages radio resources and handovers, and ciphers user data. It connects to the circuit-switched core through the Iu-CS interface and to the packet core through Iu-PS. + +![RNC](images/15008901.png) + +[[CAPTION:Figure]] RNC. + +See also: [Node B](n.md#node-b), [UTRAN](u.md#utran), [RANAP](#ranap), [BSC](b.md#bsc). + +## RNSAP +**Radio Network Subsystem Application Part.** The UMTS signalling protocol between RNCs, on the Iur interface, used for soft handover and moving a connection from one RNC to another. + +See also: [RNC](#rnc), [UTRAN](u.md#utran). + +## Role-feature-reason +The usual template for writing user stories: "As a *role*, I want *feature*, so that *reason*." + +See also: [User Stories](u.md#user-stories). + +## Rolling Wave Planning +Planning the near future in detail and later work only roughly, then adding detail as each period comes closer. + +See also: [Project Headlights](p.md#project-headlights), [Progressive Elaboration](p.md#progressive-elaboration). + +## Root of Trust +**RoT.** The hardware, firmware and keys that a device trusts without checking, because everything else is checked against them. It anchors secure boot, device identity and key storage, and is usually built into the chip or a secure element. + +See also: [RoT](#rot), [Secure Element](s.md#secure-element), [TrustZone](t.md#trustzone). + +## RoT +See [Root of Trust](#root-of-trust). + +## Round Robin +A scheduling method that gives each task or queue a turn in a fixed circular order, with equal time slices and no priorities. It is simple and no task is starved. + +![Round Robin](images/15008907.png) + +[[CAPTION:Figure]] Round Robin. + +See also: [Scheduler](s.md#scheduler), [Preemption](p.md#preemption). + +## Router +A device that forwards packets between networks, choosing the next hop for each from its routing table, which it builds from configuration and routing protocols. + +See also: [Switch](s.md#switch), [Control Plane](c.md#control-plane), [BGP](b.md#bgp), [OSPF](o.md#ospf). + +## RPC +**Remote Procedure Call.** Calling a procedure that runs in another process or on another machine as if it were a local call, with the messaging hidden by the framework. + +See also: [JSON-RPC](j.md#json-rpc), [XML-RPC](x.md#xml-rpc), [SOAP](s.md#soap). + +## RRC +**Radio Resource Control.** The 3GPP control protocol between a device and the radio network in UMTS, LTE and 5G. It sets up and releases connections and radio bearers, broadcasts system information and manages mobility. A state machine, with idle and connected states, governs the device. + +![RRC](images/15008913.png) + +[[CAPTION:Figure]] RRC. + +![RRC](images/15008919.png) + +[[CAPTION:Figure]] RRC. + +See also: [PDCP](p.md#pdcp), [E-UTRAN](e.md#e-utran). + +## RSTP +**Rapid Spanning Tree Protocol.** IEEE 802.1w, a faster version of the spanning tree protocol that recovers from a topology change in seconds rather than tens of seconds. It is now part of 802.1D. + +See also: [STP](s.md#stp). + +## RSVP +**Resource Reservation Protocol.** A protocol (RFC 2205) with which receivers reserve network resources for a flow along its path. Its extension, RSVP-TE, sets up traffic-engineered MPLS paths. + +See also: [QoS](q.md#qos), [LSP](l.md#lsp). + +## RTCP +**RTP Control Protocol.** The companion to RTP, defined in RFC 3550. It reports statistics on an RTP session, such as packet loss, jitter and round-trip time, and helps synchronise streams. + +See also: [RTP](#rtp). + +## RTOS +**Real-Time Operating System.** An operating system that guarantees tasks respond within defined time limits, usually by priority-based preemptive scheduling. FreeRTOS, Zephyr and VxWorks are examples. + +See also: [Scheduler](s.md#scheduler), [Preemption](p.md#preemption), [MCU](m.md#mcu). + +## RTP +**Real-time Transport Protocol.** The protocol (RFC 3550) that carries audio and video over IP, usually on UDP, with sequence numbers and timestamps so receivers can play media out smoothly. SIP or another protocol sets up the session. + +See also: [RTCP](#rtcp), [SIP](s.md#sip), [VoIP](v.md#voip). + +## RTSP +**Real Time Streaming Protocol.** A protocol for controlling streaming media servers with commands such as play, pause and record. The media itself usually travels over RTP. + +See also: [RTP](#rtp). + +## Ruby +A dynamic, object-oriented scripting language created by Yukihiro Matsumoto in the mid-1990s, designed to make programmers productive and happy. + +See also: [Ruby on Rails](#ruby-on-rails). + +## Ruby on Rails +A web application framework written in Ruby, built on the model-view-controller pattern and known for "convention over configuration". + +See also: [Ruby](#ruby). + +## Rule of Simplicity +Kent Beck's rules, in priority order, for judging whether code is simple enough: it passes the tests, reveals its intent, has no duplication and has the fewest elements. + +See also: [Simple Design](s.md#simple-design), [KISS Principle](k.md#kiss-principle). + +## RUP +**Rational Unified Process.** An iterative software development process framework from Rational Software (later IBM), with four phases: inception, elaboration, construction and transition. + +See also: [UML](u.md#uml), [Iterative Development](i.md#iterative-development). diff --git a/content/glossary/redirects.tsv b/content/glossary/redirects.tsv index b3eccca..0499b5c 100644 --- a/content/glossary/redirects.tsv +++ b/content/glossary/redirects.tsv @@ -2,1139 +2,1146 @@ # mkdocs-redirects takes for redirect_maps (pages relative to docs/), for the tarf.co.uk rows. old_url redirect_from redirect_to note https://tarf.co.uk/Glossary/ Glossary/index.md glossary/index.md -https://tarf.co.uk/Glossary/1/1/ Glossary/1/1.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/1/100%20Mbps---Fast%20Ethernet/ Glossary/1/100 Mbps---Fast Ethernet.md glossary/s-z.md#100-mbps-fast-ethernet renamed to "100 Mbps (Fast Ethernet)" -https://tarf.co.uk/Glossary/5/5/ Glossary/5/5.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/5/5ESS/ Glossary/5/5ESS.md glossary/s-z.md#5ess -https://tarf.co.uk/Glossary/5/5G/ Glossary/5/5G.md glossary/s-z.md#5g -https://tarf.co.uk/Glossary/8/8/ Glossary/8/8.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/8/802.11/ Glossary/8/802.11.md glossary/s-z.md#80211 -https://tarf.co.uk/Glossary/8/802.3%20Physical%20Layer/ Glossary/8/802.3 Physical Layer.md glossary/s-z.md#8023-physical-layer -https://tarf.co.uk/Glossary/a/A-law/ Glossary/a/A-law.md glossary/a-c.md#a-law -https://tarf.co.uk/Glossary/a/a/ Glossary/a/a.md glossary/a-c.md letter index -https://tarf.co.uk/Glossary/a/AA/ Glossary/a/AA.md glossary/a-c.md#aa -https://tarf.co.uk/Glossary/a/AAC/ Glossary/a/AAC.md glossary/a-c.md#aac -https://tarf.co.uk/Glossary/a/AAL1/ Glossary/a/AAL1.md glossary/a-c.md#aal1 -https://tarf.co.uk/Glossary/a/AAL2/ Glossary/a/AAL2.md glossary/a-c.md#aal2 -https://tarf.co.uk/Glossary/a/AAL5/ Glossary/a/AAL5.md glossary/a-c.md#aal5 -https://tarf.co.uk/Glossary/a/Acceptance%20Test/ Glossary/a/Acceptance Test.md glossary/a-c.md#acceptance-test -https://tarf.co.uk/Glossary/a/Acceptance%20Testing/ Glossary/a/Acceptance Testing.md glossary/a-c.md#acceptance-testing -https://tarf.co.uk/Glossary/a/Activity%20Model/ Glossary/a/Activity Model.md glossary/a-c.md#activity-model -https://tarf.co.uk/Glossary/a/Activity/ Glossary/a/Activity.md glossary/a-c.md#activity -https://tarf.co.uk/Glossary/a/Ad%20Hoc%20Testing/ Glossary/a/Ad Hoc Testing.md glossary/a-c.md#ad-hoc-testing -https://tarf.co.uk/Glossary/a/AD/ Glossary/a/AD.md glossary/a-c.md#ad -https://tarf.co.uk/Glossary/a/Ada/ Glossary/a/Ada.md glossary/a-c.md#ada -https://tarf.co.uk/Glossary/a/ADSL/ Glossary/a/ADSL.md glossary/a-c.md#adsl -https://tarf.co.uk/Glossary/a/AES/ Glossary/a/AES.md glossary/a-c.md#aes -https://tarf.co.uk/Glossary/a/Agent/ Glossary/a/Agent.md glossary/a-c.md#agent -https://tarf.co.uk/Glossary/a/Agile%20development/ Glossary/a/Agile development.md glossary/a-c.md#agile-development -https://tarf.co.uk/Glossary/a/Agile%20Manifesto/ Glossary/a/Agile Manifesto.md glossary/a-c.md#agile-manifesto -https://tarf.co.uk/Glossary/a/Agile/ Glossary/a/Agile.md glossary/a-c.md#agile -https://tarf.co.uk/Glossary/a/AIS/ Glossary/a/AIS.md glossary/a-c.md#ais -https://tarf.co.uk/Glossary/a/ALCAP/ Glossary/a/ALCAP.md glossary/a-c.md#alcap -https://tarf.co.uk/Glossary/a/Amazon%20EC2/ Glossary/a/Amazon EC2.md glossary/a-c.md#amazon-ec2 -https://tarf.co.uk/Glossary/a/AMP/ Glossary/a/AMP.md glossary/a-c.md#amp -https://tarf.co.uk/Glossary/a/AMR-NB/ Glossary/a/AMR-NB.md glossary/a-c.md#amr-nb -https://tarf.co.uk/Glossary/a/AMR-WB/ Glossary/a/AMR-WB.md glossary/a-c.md#amr-wb -https://tarf.co.uk/Glossary/a/Analogy%20Estimation/ Glossary/a/Analogy Estimation.md glossary/a-c.md#analogy-estimation -https://tarf.co.uk/Glossary/a/Android/ Glossary/a/Android.md glossary/a-c.md#android -https://tarf.co.uk/Glossary/a/ANDSF/ Glossary/a/ANDSF.md glossary/a-c.md#andsf -https://tarf.co.uk/Glossary/a/ANSI/ Glossary/a/ANSI.md glossary/a-c.md#ansi -https://tarf.co.uk/Glossary/a/Antipattern/ Glossary/a/Antipattern.md glossary/a-c.md#antipattern -https://tarf.co.uk/Glossary/a/Anycast/ Glossary/a/Anycast.md glossary/a-c.md#anycast -https://tarf.co.uk/Glossary/a/AOP/ Glossary/a/AOP.md glossary/a-c.md#aop -https://tarf.co.uk/Glossary/a/API/ Glossary/a/API.md glossary/a-c.md#api -https://tarf.co.uk/Glossary/a/APN/ Glossary/a/APN.md glossary/a-c.md#apn -https://tarf.co.uk/Glossary/a/Architecture/ Glossary/a/Architecture.md glossary/a-c.md#architecture -https://tarf.co.uk/Glossary/a/ARP%20spoofing/ Glossary/a/ARP spoofing.md glossary/a-c.md#arp-spoofing -https://tarf.co.uk/Glossary/a/Artifact/ Glossary/a/Artifact.md glossary/a-c.md#artefact renamed to "Artefact" -https://tarf.co.uk/Glossary/a/ASCII/ Glossary/a/ASCII.md glossary/a-c.md#ascii -https://tarf.co.uk/Glossary/a/ASN.1/ Glossary/a/ASN.1.md glossary/a-c.md#asn1 -https://tarf.co.uk/Glossary/a/Assessment/ Glossary/a/Assessment.md glossary/a-c.md#assessment -https://tarf.co.uk/Glossary/a/ATDD/ Glossary/a/ATDD.md glossary/a-c.md#atdd -https://tarf.co.uk/Glossary/a/ATM/ Glossary/a/ATM.md glossary/a-c.md#atm -https://tarf.co.uk/Glossary/a/Audio%20Codecs/ Glossary/a/Audio Codecs.md glossary/a-c.md#audio-codecs -https://tarf.co.uk/Glossary/a/Audit/ Glossary/a/Audit.md glossary/a-c.md#audit -https://tarf.co.uk/Glossary/a/Author/ Glossary/a/Author.md glossary/a-c.md#author -https://tarf.co.uk/Glossary/a/Authority/ Glossary/a/Authority.md glossary/a-c.md#authority -https://tarf.co.uk/Glossary/a/authorriative%20Answer/ Glossary/a/authorriative Answer.md glossary/a-c.md#authoritative-answer renamed to "Authoritative Answer" -https://tarf.co.uk/Glossary/a/Automated%20Build/ Glossary/a/Automated Build.md glossary/a-c.md#automated-build -https://tarf.co.uk/Glossary/a/Automated%20Testing/ Glossary/a/Automated Testing.md glossary/a-c.md#automated-testing -https://tarf.co.uk/Glossary/a/Awk/ Glossary/a/Awk.md glossary/a-c.md#awk -https://tarf.co.uk/Glossary/a/AWS/ Glossary/a/AWS.md glossary/a-c.md#aws -https://tarf.co.uk/Glossary/a/Azure/ Glossary/a/Azure.md glossary/a-c.md#azure -https://tarf.co.uk/Glossary/b/b/ Glossary/b/b.md glossary/a-c.md letter index -https://tarf.co.uk/Glossary/b/B2B/ Glossary/b/B2B.md glossary/a-c.md#b2b -https://tarf.co.uk/Glossary/b/B2C/ Glossary/b/B2C.md glossary/a-c.md#b2c -https://tarf.co.uk/Glossary/b/Backhaul/ Glossary/b/Backhaul.md glossary/a-c.md#backhaul -https://tarf.co.uk/Glossary/b/Backlog%20Grooming/ Glossary/b/Backlog Grooming.md glossary/a-c.md#backlog-grooming -https://tarf.co.uk/Glossary/b/Backlog/ Glossary/b/Backlog.md glossary/a-c.md#backlog -https://tarf.co.uk/Glossary/b/Bandwidth/ Glossary/b/Bandwidth.md glossary/a-c.md#bandwidth -https://tarf.co.uk/Glossary/b/Baseline/ Glossary/b/Baseline.md glossary/a-c.md#baseline -https://tarf.co.uk/Glossary/b/Bash/ Glossary/b/Bash.md glossary/a-c.md#bash -https://tarf.co.uk/Glossary/b/BASIC/ Glossary/b/BASIC.md glossary/a-c.md#basic -https://tarf.co.uk/Glossary/b/BAU/ Glossary/b/BAU.md glossary/a-c.md#bau -https://tarf.co.uk/Glossary/b/BDD/ Glossary/b/BDD.md glossary/a-c.md#bdd -https://tarf.co.uk/Glossary/b/Bench%20Test/ Glossary/b/Bench Test.md glossary/a-c.md#bench-test -https://tarf.co.uk/Glossary/b/BER/ Glossary/b/BER.md glossary/a-c.md#ber -https://tarf.co.uk/Glossary/b/Best%20Practice%20Description/ Glossary/b/Best Practice Description.md glossary/a-c.md#best-practice renamed to "Best Practice" -https://tarf.co.uk/Glossary/b/Best%20Practice/ Glossary/b/Best Practice.md glossary/a-c.md#best-practice -https://tarf.co.uk/Glossary/b/BGCF/ Glossary/b/BGCF.md glossary/a-c.md#bgcf -https://tarf.co.uk/Glossary/b/BGP/ Glossary/b/BGP.md glossary/a-c.md#bgp -https://tarf.co.uk/Glossary/b/Bjarne%20Stroustrup/ Glossary/b/Bjarne Stroustrup.md glossary/a-c.md#bjarne-stroustrup -https://tarf.co.uk/Glossary/b/Black-Box%20Testing/ Glossary/b/Black-Box Testing.md glossary/a-c.md#black-box-testing -https://tarf.co.uk/Glossary/b/Bluetooth/ Glossary/b/Bluetooth.md glossary/a-c.md#bluetooth -https://tarf.co.uk/Glossary/b/BNF/ Glossary/b/BNF.md glossary/a-c.md#bnf -https://tarf.co.uk/Glossary/b/BNG/ Glossary/b/BNG.md glossary/a-c.md#bng -https://tarf.co.uk/Glossary/b/BOOTP/ Glossary/b/BOOTP.md glossary/a-c.md#bootp -https://tarf.co.uk/Glossary/b/Boston%20Matrix/ Glossary/b/Boston Matrix.md glossary/a-c.md#boston-matrix -https://tarf.co.uk/Glossary/b/Bottom-Up%20Estimation/ Glossary/b/Bottom-Up Estimation.md glossary/a-c.md#bottom-up-estimation -https://tarf.co.uk/Glossary/b/BPEL/ Glossary/b/BPEL.md glossary/a-c.md#bpel -https://tarf.co.uk/Glossary/b/BPMN/ Glossary/b/BPMN.md glossary/a-c.md#bpmn -https://tarf.co.uk/Glossary/b/BRAS/ Glossary/b/BRAS.md glossary/a-c.md#bras -https://tarf.co.uk/Glossary/b/Broadcast/ Glossary/b/Broadcast.md glossary/a-c.md#broadcast -https://tarf.co.uk/Glossary/b/BSC/ Glossary/b/BSC.md glossary/a-c.md#bsc -https://tarf.co.uk/Glossary/b/BSS/ Glossary/b/BSS.md glossary/a-c.md#bss -https://tarf.co.uk/Glossary/b/BT%2021CN/ Glossary/b/BT 21CN.md glossary/a-c.md#bt-21cn -https://tarf.co.uk/Glossary/b/BTNUP/ Glossary/b/BTNUP.md glossary/a-c.md#btnup -https://tarf.co.uk/Glossary/b/BTS/ Glossary/b/BTS.md glossary/a-c.md#bts -https://tarf.co.uk/Glossary/b/bug/ Glossary/b/bug.md glossary/a-c.md#bug -https://tarf.co.uk/Glossary/b/Build%20Environment/ Glossary/b/Build Environment.md glossary/a-c.md#build-environment -https://tarf.co.uk/Glossary/b/Build%20Machine/ Glossary/b/Build Machine.md glossary/a-c.md#build-machine -https://tarf.co.uk/Glossary/b/build%20tool/ Glossary/b/build tool.md glossary/a-c.md#build-tool -https://tarf.co.uk/Glossary/b/Build/ Glossary/b/Build.md glossary/a-c.md#build -https://tarf.co.uk/Glossary/b/Burndown%20Chart/ Glossary/b/Burndown Chart.md glossary/a-c.md#burndown-chart -https://tarf.co.uk/Glossary/b/Burst/ Glossary/b/Burst.md glossary/a-c.md#burst -https://tarf.co.uk/Glossary/b/Business%20Agility%20Ownership/ Glossary/b/Business Agility Ownership.md glossary/a-c.md#business-agility-ownership -https://tarf.co.uk/Glossary/b/Business%20Requirement/ Glossary/b/Business Requirement.md glossary/a-c.md#business-requirement -https://tarf.co.uk/Glossary/b/Business%20Schedule/ Glossary/b/Business Schedule.md glossary/a-c.md#business-schedule -https://tarf.co.uk/Glossary/b/BYOD/ Glossary/b/BYOD.md glossary/a-c.md#byod -https://tarf.co.uk/Glossary/c/C%20Language/ Glossary/c/C Language.md glossary/a-c.md#c-language -https://tarf.co.uk/Glossary/c/C%23/ Glossary/c/C#.md glossary/a-c.md#c_2 -https://tarf.co.uk/Glossary/c/C%2B%2B/ Glossary/c/C++.md glossary/a-c.md#c_1 -https://tarf.co.uk/Glossary/c/C-Tag/ Glossary/c/C-Tag.md glossary/a-c.md#c-tag -https://tarf.co.uk/Glossary/c/c/ Glossary/c/c.md glossary/a-c.md letter index -https://tarf.co.uk/Glossary/c/CA/ Glossary/c/CA.md glossary/a-c.md#ca -https://tarf.co.uk/Glossary/c/Cacti/ Glossary/c/Cacti.md glossary/a-c.md#cacti -https://tarf.co.uk/Glossary/c/Cadence/ Glossary/c/Cadence.md glossary/a-c.md#cadence -https://tarf.co.uk/Glossary/c/Calendar%20Days/ Glossary/c/Calendar Days.md glossary/a-c.md#calendar-days -https://tarf.co.uk/Glossary/c/CAM%20Item/ Glossary/c/CAM Item.md glossary/g-k.md#item renamed to "Item" -https://tarf.co.uk/Glossary/c/Camel%20Case/ Glossary/c/Camel Case.md glossary/a-c.md#camel-case -https://tarf.co.uk/Glossary/c/CAMEL/ Glossary/c/CAMEL.md glossary/a-c.md#camel -https://tarf.co.uk/Glossary/c/CAP/ Glossary/c/CAP.md glossary/a-c.md#cap -https://tarf.co.uk/Glossary/c/Capability%20Maturity%20Model/ Glossary/c/Capability Maturity Model.md glossary/a-c.md#capability-maturity-model -https://tarf.co.uk/Glossary/c/Cat%205/ Glossary/c/Cat 5.md glossary/a-c.md#cat-5 -https://tarf.co.uk/Glossary/c/Cat%205e/ Glossary/c/Cat 5e.md glossary/a-c.md#cat-5e -https://tarf.co.uk/Glossary/c/Cat%206/ Glossary/c/Cat 6.md glossary/a-c.md#cat-6 -https://tarf.co.uk/Glossary/c/CCITT/ Glossary/c/CCITT.md glossary/a-c.md#ccitt -https://tarf.co.uk/Glossary/c/CCS/ Glossary/c/CCS.md glossary/a-c.md#ccs -https://tarf.co.uk/Glossary/c/CCTA/ Glossary/c/CCTA.md glossary/a-c.md#ccta -https://tarf.co.uk/Glossary/c/CDMA/ Glossary/c/CDMA.md glossary/a-c.md#cdma -https://tarf.co.uk/Glossary/c/CDMA2000/ Glossary/c/CDMA2000.md glossary/a-c.md#cdma2000 -https://tarf.co.uk/Glossary/c/CDR/ Glossary/c/CDR.md glossary/a-c.md#cdr -https://tarf.co.uk/Glossary/c/CE/ Glossary/c/CE.md glossary/a-c.md#ce -https://tarf.co.uk/Glossary/c/CER/ Glossary/c/CER.md glossary/a-c.md#cer -https://tarf.co.uk/Glossary/c/Ceremonies/ Glossary/c/Ceremonies.md glossary/a-c.md#ceremonies -https://tarf.co.uk/Glossary/c/CESG/ Glossary/c/CESG.md glossary/a-c.md#cesg -https://tarf.co.uk/Glossary/c/CFi/ Glossary/c/CFi.md glossary/a-c.md#cfi -https://tarf.co.uk/Glossary/c/CGI/ Glossary/c/CGI.md glossary/a-c.md#cgi -https://tarf.co.uk/Glossary/c/Change%20Control%20Board/ Glossary/c/Change Control Board.md glossary/a-c.md#change-control-board -https://tarf.co.uk/Glossary/c/Change%20Control%20Plan/ Glossary/c/Change Control Plan.md glossary/a-c.md#change-control-plan -https://tarf.co.uk/Glossary/c/Change%20Control/ Glossary/c/Change Control.md glossary/a-c.md#change-control -https://tarf.co.uk/Glossary/c/Change%20Management/ Glossary/c/Change Management.md glossary/a-c.md#change-management -https://tarf.co.uk/Glossary/c/Change%20Request/ Glossary/c/Change Request.md glossary/a-c.md#change-request -https://tarf.co.uk/Glossary/c/CHAP/ Glossary/c/CHAP.md glossary/a-c.md#chap -https://tarf.co.uk/Glossary/c/Chrome%20OS/ Glossary/c/Chrome OS.md glossary/a-c.md#chrome-os -https://tarf.co.uk/Glossary/c/Circuit%20Switched/ Glossary/c/Circuit Switched.md glossary/a-c.md#circuit-switched -https://tarf.co.uk/Glossary/c/Class%204%20Softswitch/ Glossary/c/Class 4 Softswitch.md glossary/a-c.md#class-4-softswitch -https://tarf.co.uk/Glossary/c/Class%205%20Softswitch/ Glossary/c/Class 5 Softswitch.md glossary/a-c.md#class-5-softswitch -https://tarf.co.uk/Glossary/c/Class%20Model/ Glossary/c/Class Model.md glossary/a-c.md#class-model -https://tarf.co.uk/Glossary/c/CLF/ Glossary/c/CLF.md glossary/a-c.md#clf -https://tarf.co.uk/Glossary/c/CMDB/ Glossary/c/CMDB.md glossary/a-c.md#cmdb -https://tarf.co.uk/Glossary/c/CNG/ Glossary/c/CNG.md glossary/a-c.md#cng -https://tarf.co.uk/Glossary/c/Cobol/ Glossary/c/Cobol.md glossary/a-c.md#cobol -https://tarf.co.uk/Glossary/c/Code%20and%20Fix%20Lifecycle/ Glossary/c/Code and Fix Lifecycle.md glossary/a-c.md#code-and-fix-lifecycle -https://tarf.co.uk/Glossary/c/Codecs/ Glossary/c/Codecs.md glossary/a-c.md#codecs -https://tarf.co.uk/Glossary/c/Coding%20Standard/ Glossary/c/Coding Standard.md glossary/a-c.md#coding-standard -https://tarf.co.uk/Glossary/c/Coding/ Glossary/c/Coding.md glossary/a-c.md#coding -https://tarf.co.uk/Glossary/c/CoE/ Glossary/c/CoE.md glossary/a-c.md#coe -https://tarf.co.uk/Glossary/c/COFDM/ Glossary/c/COFDM.md glossary/a-c.md#cofdm -https://tarf.co.uk/Glossary/c/Collaboration%20Model/ Glossary/c/Collaboration Model.md glossary/a-c.md#collaboration-model -https://tarf.co.uk/Glossary/c/Collaborative%20Construction/ Glossary/c/Collaborative Construction.md glossary/a-c.md#collaborative-construction -https://tarf.co.uk/Glossary/c/Collective%20code%20ownership/ Glossary/c/Collective code ownership.md glossary/a-c.md#collective-code-ownership -https://tarf.co.uk/Glossary/c/COLO/ Glossary/c/COLO.md glossary/a-c.md#colo -https://tarf.co.uk/Glossary/c/Compatibility%20Test/ Glossary/c/Compatibility Test.md glossary/a-c.md#compatibility-test -https://tarf.co.uk/Glossary/c/Compiled%20Language/ Glossary/c/Compiled Language.md glossary/a-c.md#compiled-language -https://tarf.co.uk/Glossary/c/Compiler%20Back%20end/ Glossary/c/Compiler Back end.md glossary/a-c.md#compiler-back-end -https://tarf.co.uk/Glossary/c/Compiler%20Front%20end/ Glossary/c/Compiler Front end.md glossary/a-c.md#compiler-front-end -https://tarf.co.uk/Glossary/c/Compiler%20Middle%20end/ Glossary/c/Compiler Middle end.md glossary/a-c.md#compiler-middle-end -https://tarf.co.uk/Glossary/c/Compiler%20preprocessor/ Glossary/c/Compiler preprocessor.md glossary/a-c.md#compiler-preprocessor -https://tarf.co.uk/Glossary/c/Complier/ Glossary/c/Complier.md glossary/a-c.md#compiler renamed to "Compiler" -https://tarf.co.uk/Glossary/c/Component%20Test/ Glossary/c/Component Test.md glossary/a-c.md#component-test -https://tarf.co.uk/Glossary/c/Component/ Glossary/c/Component.md glossary/a-c.md#component -https://tarf.co.uk/Glossary/c/Cone%20of%20Uncertainty/ Glossary/c/Cone of Uncertainty.md glossary/a-c.md#cone-of-uncertainty -https://tarf.co.uk/Glossary/c/Configuration%20Item/ Glossary/c/Configuration Item.md glossary/a-c.md#configuration-item -https://tarf.co.uk/Glossary/c/Configuration%20Management/ Glossary/c/Configuration Management.md glossary/a-c.md#configuration-management -https://tarf.co.uk/Glossary/c/Configuration%20Test/ Glossary/c/Configuration Test.md glossary/a-c.md#configuration-test -https://tarf.co.uk/Glossary/c/Construction%20Environment/ Glossary/c/Construction Environment.md glossary/a-c.md#construction-environment -https://tarf.co.uk/Glossary/c/Construction%20Lead/ Glossary/c/Construction Lead.md glossary/a-c.md#construction-lead -https://tarf.co.uk/Glossary/c/Construction%20Standard/ Glossary/c/Construction Standard.md glossary/a-c.md#construction-standard -https://tarf.co.uk/Glossary/c/Construction%20Test%20Environment/ Glossary/c/Construction Test Environment.md glossary/a-c.md#construction-test-environment -https://tarf.co.uk/Glossary/c/Construction%20Testing/ Glossary/c/Construction Testing.md glossary/a-c.md#construction-testing -https://tarf.co.uk/Glossary/c/Construction/ Glossary/c/Construction.md glossary/a-c.md#construction -https://tarf.co.uk/Glossary/c/Construx%20Knowledge%20Area/ Glossary/c/Construx Knowledge Area.md glossary/s-z.md#software-engineering-body-of-knowledge renamed to "Software Engineering Body of Knowledge" -https://tarf.co.uk/Glossary/c/Continuous%20Deployment/ Glossary/c/Continuous Deployment.md glossary/a-c.md#continuous-deployment -https://tarf.co.uk/Glossary/c/continuous%20integration/ Glossary/c/continuous integration.md glossary/a-c.md#continuous-integration -https://tarf.co.uk/Glossary/c/Control%20Plane/ Glossary/c/Control Plane.md glossary/a-c.md#control-plane -https://tarf.co.uk/Glossary/c/Copyleft/ Glossary/c/Copyleft.md glossary/a-c.md#copyleft -https://tarf.co.uk/Glossary/c/Corrective%20Activity%20Management/ Glossary/c/Corrective Activity Management.md glossary/a-c.md dropped -https://tarf.co.uk/Glossary/c/CoS/ Glossary/c/CoS.md glossary/a-c.md#cos -https://tarf.co.uk/Glossary/c/COTS/ Glossary/c/COTS.md glossary/a-c.md#cots -https://tarf.co.uk/Glossary/c/CPE/ Glossary/c/CPE.md glossary/a-c.md#cpe -https://tarf.co.uk/Glossary/c/CPM/ Glossary/c/CPM.md glossary/a-c.md#cpm -https://tarf.co.uk/Glossary/c/CPS/ Glossary/c/CPS.md glossary/a-c.md#cps -https://tarf.co.uk/Glossary/c/CRC%20Cards/ Glossary/c/CRC Cards.md glossary/a-c.md#crc-cards -https://tarf.co.uk/Glossary/c/CRC/ Glossary/c/CRC.md glossary/a-c.md#crc -https://tarf.co.uk/Glossary/c/CSCF/ Glossary/c/CSCF.md glossary/a-c.md#cscf -https://tarf.co.uk/Glossary/c/csh/ Glossary/c/csh.md glossary/a-c.md#csh -https://tarf.co.uk/Glossary/c/Csharp/ Glossary/c/Csharp.md glossary/a-c.md#c_2 renamed to "C#" -https://tarf.co.uk/Glossary/c/CSI/ Glossary/c/CSI.md glossary/a-c.md#csi -https://tarf.co.uk/Glossary/c/CSMA/ Glossary/c/CSMA.md glossary/a-c.md#csma -https://tarf.co.uk/Glossary/c/CSS/ Glossary/c/CSS.md glossary/a-c.md#css -https://tarf.co.uk/Glossary/c/CUE%20/ Glossary/c/CUE .md glossary/a-c.md#cue -https://tarf.co.uk/Glossary/c/Customer%20Development/ Glossary/c/Customer Development.md glossary/a-c.md#customer-development -https://tarf.co.uk/Glossary/c/CVSD/ Glossary/c/CVSD.md glossary/a-c.md#cvsd -https://tarf.co.uk/Glossary/d/d/ Glossary/d/d.md glossary/d-f.md letter index -https://tarf.co.uk/Glossary/d/DAD/ Glossary/d/DAD.md glossary/d-f.md#dad -https://tarf.co.uk/Glossary/d/Daily%20Meeting/ Glossary/d/Daily Meeting.md glossary/d-f.md#daily-meeting -https://tarf.co.uk/Glossary/d/Daily%20Scrum/ Glossary/d/Daily Scrum.md glossary/d-f.md#daily-scrum -https://tarf.co.uk/Glossary/d/Dark%20Fibre/ Glossary/d/Dark Fibre.md glossary/d-f.md#dark-fibre -https://tarf.co.uk/Glossary/d/DASS1/ Glossary/d/DASS1.md glossary/d-f.md#dass1 -https://tarf.co.uk/Glossary/d/DASS2/ Glossary/d/DASS2.md glossary/d-f.md#dass2 -https://tarf.co.uk/Glossary/d/data%20modeling/ Glossary/d/data modeling.md glossary/d-f.md#data-modelling renamed to "Data modelling" -https://tarf.co.uk/Glossary/d/Data%20Plane/ Glossary/d/Data Plane.md glossary/d-f.md#data-plane -https://tarf.co.uk/Glossary/d/Database%20Lead/ Glossary/d/Database Lead.md glossary/d-f.md#database-lead -https://tarf.co.uk/Glossary/d/Date%20Warehouse/ Glossary/d/Date Warehouse.md glossary/d-f.md#data-warehouse renamed to "Data Warehouse" -https://tarf.co.uk/Glossary/d/DCE/ Glossary/d/DCE.md glossary/d-f.md#dce -https://tarf.co.uk/Glossary/d/DDK/ Glossary/d/DDK.md glossary/d-f.md#ddk -https://tarf.co.uk/Glossary/d/DDOS/ Glossary/d/DDOS.md glossary/d-f.md#ddos -https://tarf.co.uk/Glossary/d/Decomposition%20Estimation/ Glossary/d/Decomposition Estimation.md glossary/d-f.md#decomposition-estimation -https://tarf.co.uk/Glossary/d/Defect%20Management/ Glossary/d/Defect Management.md glossary/d-f.md#defect-management -https://tarf.co.uk/Glossary/d/Defect%20Report/ Glossary/d/Defect Report.md glossary/d-f.md#defect-report -https://tarf.co.uk/Glossary/d/Defect/ Glossary/d/Defect.md glossary/d-f.md#defect -https://tarf.co.uk/Glossary/d/Definition%20of%20Done/ Glossary/d/Definition of Done.md glossary/d-f.md#definition-of-done -https://tarf.co.uk/Glossary/d/Definition%20of%20Ready/ Glossary/d/Definition of Ready.md glossary/d-f.md#definition-of-ready -https://tarf.co.uk/Glossary/d/Deliverable/ Glossary/d/Deliverable.md glossary/d-f.md#deliverable -https://tarf.co.uk/Glossary/d/Deplhi/ Glossary/d/Deplhi.md glossary/d-f.md#delphi renamed to "Delphi" -https://tarf.co.uk/Glossary/d/Deployment%20Lead/ Glossary/d/Deployment Lead.md glossary/d-f.md#deployment-lead -https://tarf.co.uk/Glossary/d/Deployment%20Plan/ Glossary/d/Deployment Plan.md glossary/d-f.md#deployment-plan -https://tarf.co.uk/Glossary/d/Deployment/ Glossary/d/Deployment.md glossary/d-f.md#deployment -https://tarf.co.uk/Glossary/d/DER/ Glossary/d/DER.md glossary/d-f.md#der -https://tarf.co.uk/Glossary/d/DES/ Glossary/d/DES.md glossary/d-f.md#des -https://tarf.co.uk/Glossary/d/Design%20Lead/ Glossary/d/Design Lead.md glossary/d-f.md#design-lead -https://tarf.co.uk/Glossary/d/Design%20Pattern/ Glossary/d/Design Pattern.md glossary/d-f.md#design-pattern -https://tarf.co.uk/Glossary/d/Design%20to%20Tools/ Glossary/d/Design to Tools.md glossary/d-f.md#design-to-tools -https://tarf.co.uk/Glossary/d/Design/ Glossary/d/Design.md glossary/d-f.md#design -https://tarf.co.uk/Glossary/d/Desk%20Check/ Glossary/d/Desk Check.md glossary/d-f.md#desk-check -https://tarf.co.uk/Glossary/d/Detailed%20Design/ Glossary/d/Detailed Design.md glossary/d-f.md#detailed-design -https://tarf.co.uk/Glossary/d/Detailed%20Schedule/ Glossary/d/Detailed Schedule.md glossary/d-f.md#detailed-schedule -https://tarf.co.uk/Glossary/d/Developer%20Integration%20Test/ Glossary/d/Developer Integration Test.md glossary/d-f.md#developer-integration-test -https://tarf.co.uk/Glossary/d/Developer/ Glossary/d/Developer.md glossary/d-f.md#developer -https://tarf.co.uk/Glossary/d/development%20environment/ Glossary/d/development environment.md glossary/d-f.md#development-environment -https://tarf.co.uk/Glossary/d/development%20process/ Glossary/d/development process.md glossary/d-f.md#development-process -https://tarf.co.uk/Glossary/d/DevOps/ Glossary/d/DevOps.md glossary/d-f.md#devops -https://tarf.co.uk/Glossary/d/DHCP/ Glossary/d/DHCP.md glossary/d-f.md#dhcp -https://tarf.co.uk/Glossary/d/Diagram/ Glossary/d/Diagram.md glossary/d-f.md#diagram -https://tarf.co.uk/Glossary/d/DLE/ Glossary/d/DLE.md glossary/d-f.md#dle -https://tarf.co.uk/Glossary/d/DLP/ Glossary/d/DLP.md glossary/d-f.md#dlp -https://tarf.co.uk/Glossary/d/DMZ/ Glossary/d/DMZ.md glossary/d-f.md#dmz -https://tarf.co.uk/Glossary/d/DN/ Glossary/d/DN.md glossary/d-f.md#dn -https://tarf.co.uk/Glossary/d/DNB/ Glossary/d/DNB.md glossary/d-f.md#dnb -https://tarf.co.uk/Glossary/d/DNS%20-%20A/ Glossary/d/DNS - A.md glossary/d-f.md#dns-a -https://tarf.co.uk/Glossary/d/DNS%20-%20AAAA/ Glossary/d/DNS - AAAA.md glossary/d-f.md#dns-aaaa -https://tarf.co.uk/Glossary/d/DNS%20-%20CNAME/ Glossary/d/DNS - CNAME.md glossary/d-f.md#dns-cname -https://tarf.co.uk/Glossary/d/DNS%20-%20DNAME/ Glossary/d/DNS - DNAME.md glossary/d-f.md#dns-dname -https://tarf.co.uk/Glossary/d/DNS%20-%20SRV/ Glossary/d/DNS - SRV.md glossary/d-f.md#dns-srv -https://tarf.co.uk/Glossary/d/DNS%20-%20TXT/ Glossary/d/DNS - TXT.md glossary/d-f.md#dns-txt -https://tarf.co.uk/Glossary/d/DNS%20Spoofing/ Glossary/d/DNS Spoofing.md glossary/d-f.md#dns-spoofing -https://tarf.co.uk/Glossary/d/DNS/ Glossary/d/DNS.md glossary/d-f.md#dns -https://tarf.co.uk/Glossary/d/Docker/ Glossary/d/Docker.md glossary/d-f.md#docker -https://tarf.co.uk/Glossary/d/Document%20Collaboration/ Glossary/d/Document Collaboration.md glossary/d-f.md#document-collaboration -https://tarf.co.uk/Glossary/d/Domain%20Name/ Glossary/d/Domain Name.md glossary/d-f.md#domain-name -https://tarf.co.uk/Glossary/d/Dot1Q%20trunking/ Glossary/d/Dot1Q trunking.md glossary/d-f.md#dot1q-trunking -https://tarf.co.uk/Glossary/d/Dot1Q/ Glossary/d/Dot1Q.md glossary/d-f.md#dot1q -https://tarf.co.uk/Glossary/d/dotNET/ Glossary/d/dotNET.md glossary/s-z.md#net renamed to ".NET" -https://tarf.co.uk/Glossary/d/Downstream/ Glossary/d/Downstream.md glossary/d-f.md#downstream -https://tarf.co.uk/Glossary/d/DP/ Glossary/d/DP.md glossary/d-f.md#dp -https://tarf.co.uk/Glossary/d/DPI/ Glossary/d/DPI.md glossary/d-f.md#dpi -https://tarf.co.uk/Glossary/d/DPNSS/ Glossary/d/DPNSS.md glossary/d-f.md#dpnss -https://tarf.co.uk/Glossary/d/DQ/ Glossary/d/DQ.md glossary/d-f.md#dq -https://tarf.co.uk/Glossary/d/Draft/ Glossary/d/Draft.md glossary/d-f.md#draft -https://tarf.co.uk/Glossary/d/driver/ Glossary/d/driver.md glossary/d-f.md#driver -https://tarf.co.uk/Glossary/d/DSCP/ Glossary/d/DSCP.md glossary/d-f.md#dscp -https://tarf.co.uk/Glossary/d/DSDM/ Glossary/d/DSDM.md glossary/d-f.md#dsdm -https://tarf.co.uk/Glossary/d/DSL/ Glossary/d/DSL.md glossary/d-f.md#dsl -https://tarf.co.uk/Glossary/d/DSLAM/ Glossary/d/DSLAM.md glossary/d-f.md#dslam -https://tarf.co.uk/Glossary/d/DTE/ Glossary/d/DTE.md glossary/d-f.md#dte -https://tarf.co.uk/Glossary/d/DTMF/ Glossary/d/DTMF.md glossary/d-f.md#dtmf -https://tarf.co.uk/Glossary/d/DTT/ Glossary/d/DTT.md glossary/d-f.md#dtt -https://tarf.co.uk/Glossary/d/DTV/ Glossary/d/DTV.md glossary/d-f.md#dtv -https://tarf.co.uk/Glossary/d/Duplex/ Glossary/d/Duplex.md glossary/d-f.md#duplex -https://tarf.co.uk/Glossary/d/DVSR/ Glossary/d/DVSR.md glossary/d-f.md#dvsr -https://tarf.co.uk/Glossary/e/E%20NodeB/ Glossary/e/E NodeB.md glossary/d-f.md#e-nodeb -https://tarf.co.uk/Glossary/e/E-UTRAN/ Glossary/e/E-UTRAN.md glossary/d-f.md#e-utran -https://tarf.co.uk/Glossary/e/E.164/ Glossary/e/E.164.md glossary/d-f.md#e164 -https://tarf.co.uk/Glossary/e/e/ Glossary/e/e.md glossary/d-f.md letter index -https://tarf.co.uk/Glossary/e/EAD/ Glossary/e/EAD.md glossary/d-f.md#ead -https://tarf.co.uk/Glossary/e/EAP/ Glossary/e/EAP.md glossary/d-f.md#eap -https://tarf.co.uk/Glossary/e/Earned%20Value%20Management/ Glossary/e/Earned Value Management.md glossary/d-f.md#earned-value-management -https://tarf.co.uk/Glossary/e/Earned%20Value/ Glossary/e/Earned Value.md glossary/d-f.md#earned-value -https://tarf.co.uk/Glossary/e/Echo%20Cancellation%20-%20G.168/ Glossary/e/Echo Cancellation - G.168.md glossary/d-f.md#echo-cancellation-g168 -https://tarf.co.uk/Glossary/e/Echo%20cancellation/ Glossary/e/Echo cancellation.md glossary/d-f.md#echo-cancellation -https://tarf.co.uk/Glossary/e/Echo/ Glossary/e/Echo.md glossary/d-f.md#echo -https://tarf.co.uk/Glossary/e/EDB/ Glossary/e/EDB.md glossary/d-f.md#edb -https://tarf.co.uk/Glossary/e/EDGE/ Glossary/e/EDGE.md glossary/d-f.md#edge -https://tarf.co.uk/Glossary/e/Editor/ Glossary/e/Editor.md glossary/d-f.md#editor -https://tarf.co.uk/Glossary/e/EEE%20802.1Qay/ Glossary/e/EEE 802.1Qay.md glossary/g-k.md#ieee-8021qay renamed to "IEEE 802.1Qay" -https://tarf.co.uk/Glossary/e/EFM/ Glossary/e/EFM.md glossary/d-f.md#efm -https://tarf.co.uk/Glossary/e/Egress/ Glossary/e/Egress.md glossary/d-f.md#egress -https://tarf.co.uk/Glossary/e/EIA/ Glossary/e/EIA.md glossary/d-f.md#eia -https://tarf.co.uk/Glossary/e/EIGRP/ Glossary/e/EIGRP.md glossary/d-f.md#eigrp -https://tarf.co.uk/Glossary/e/embedded%20systems%20programming/ Glossary/e/embedded systems programming.md glossary/d-f.md#embedded-systems-programming -https://tarf.co.uk/Glossary/e/Engineering%20Discussion/ Glossary/e/Engineering Discussion.md glossary/d-f.md#engineering-discussion -https://tarf.co.uk/Glossary/e/Engineering%20Management/ Glossary/e/Engineering Management.md glossary/d-f.md#engineering-management -https://tarf.co.uk/Glossary/e/Engineering%20Process/ Glossary/e/Engineering Process.md glossary/d-f.md#engineering-process -https://tarf.co.uk/Glossary/e/EoFTTC/ Glossary/e/EoFTTC.md glossary/d-f.md#eofttc -https://tarf.co.uk/Glossary/e/EoS/ Glossary/e/EoS.md glossary/d-f.md#eos -https://tarf.co.uk/Glossary/e/ePDG/ Glossary/e/ePDG.md glossary/d-f.md#epdg -https://tarf.co.uk/Glossary/e/Epic/ Glossary/e/Epic.md glossary/d-f.md#epic -https://tarf.co.uk/Glossary/e/EPWS/ Glossary/e/EPWS.md glossary/d-f.md#epws -https://tarf.co.uk/Glossary/e/ESB/ Glossary/e/ESB.md glossary/d-f.md#esb -https://tarf.co.uk/Glossary/e/Estimate/ Glossary/e/Estimate.md glossary/d-f.md#estimate -https://tarf.co.uk/Glossary/e/Estimation/ Glossary/e/Estimation.md glossary/d-f.md#estimation -https://tarf.co.uk/Glossary/e/Ethernet%20over%20twisted%20pair/ Glossary/e/Ethernet over twisted pair.md glossary/d-f.md#ethernet-over-twisted-pair -https://tarf.co.uk/Glossary/e/Ethical%20Hacking/ Glossary/e/Ethical Hacking.md glossary/d-f.md#ethical-hacking -https://tarf.co.uk/Glossary/e/ETL/ Glossary/e/ETL.md glossary/d-f.md#etl -https://tarf.co.uk/Glossary/e/ETSI/ Glossary/e/ETSI.md glossary/d-f.md#etsi -https://tarf.co.uk/Glossary/e/Evaluation/ Glossary/e/Evaluation.md glossary/d-f.md dropped -https://tarf.co.uk/Glossary/e/EVC/ Glossary/e/EVC.md glossary/d-f.md#evc -https://tarf.co.uk/Glossary/e/Evolutionary%20Delivery%20Lifecycle/ Glossary/e/Evolutionary Delivery Lifecycle.md glossary/d-f.md#evolutionary-delivery-lifecycle -https://tarf.co.uk/Glossary/e/Evolutionary%20Prototyping%20Lifecycle/ Glossary/e/Evolutionary Prototyping Lifecycle.md glossary/d-f.md#evolutionary-prototyping-lifecycle -https://tarf.co.uk/Glossary/e/EVPL/ Glossary/e/EVPL.md glossary/d-f.md#evpl -https://tarf.co.uk/Glossary/e/Expert%20Judgment/ Glossary/e/Expert Judgment.md glossary/d-f.md#expert-judgement renamed to "Expert Judgement" -https://tarf.co.uk/Glossary/e/Explicit%20Change%20Control/ Glossary/e/Explicit Change Control.md glossary/d-f.md#explicit-change-control -https://tarf.co.uk/Glossary/e/Explicit%20Risk%20Management/ Glossary/e/Explicit Risk Management.md glossary/d-f.md#explicit-risk-management -https://tarf.co.uk/Glossary/e/Exploratory%20Testing/ Glossary/e/Exploratory Testing.md glossary/d-f.md#exploratory-testing -https://tarf.co.uk/Glossary/e/Extrinsic%20Risk%20Management/ Glossary/e/Extrinsic Risk Management.md glossary/d-f.md#extrinsic-risk-management -https://tarf.co.uk/Glossary/f/f/ Glossary/f/f.md glossary/d-f.md letter index -https://tarf.co.uk/Glossary/f/Facilitation/ Glossary/f/Facilitation.md glossary/d-f.md#facilitation -https://tarf.co.uk/Glossary/f/FCS/ Glossary/f/FCS.md glossary/d-f.md#fcs -https://tarf.co.uk/Glossary/f/FDDI/ Glossary/f/FDDI.md glossary/d-f.md#fddi -https://tarf.co.uk/Glossary/f/feature%20creep/ Glossary/f/feature creep.md glossary/d-f.md#feature-creep -https://tarf.co.uk/Glossary/f/Feature%20Test/ Glossary/f/Feature Test.md glossary/d-f.md#feature-test -https://tarf.co.uk/Glossary/f/Feature/ Glossary/f/Feature.md glossary/d-f.md#feature -https://tarf.co.uk/Glossary/f/FemtoCell/ Glossary/f/FemtoCell.md glossary/d-f.md#femtocell -https://tarf.co.uk/Glossary/f/FIPS%20140-2/ Glossary/f/FIPS 140-2.md glossary/d-f.md#fips-140-2 -https://tarf.co.uk/Glossary/f/FIPS/ Glossary/f/FIPS.md glossary/d-f.md#fips -https://tarf.co.uk/Glossary/f/Firewall/ Glossary/f/Firewall.md glossary/d-f.md#firewall -https://tarf.co.uk/Glossary/f/FNA/ Glossary/f/FNA.md glossary/d-f.md#fna -https://tarf.co.uk/Glossary/f/Formal%20Testing/ Glossary/f/Formal Testing.md glossary/d-f.md#formal-testing -https://tarf.co.uk/Glossary/f/Fortran/ Glossary/f/Fortran.md glossary/d-f.md#fortran -https://tarf.co.uk/Glossary/f/Forwarding%20plane/ Glossary/f/Forwarding plane.md glossary/d-f.md#forwarding-plane -https://tarf.co.uk/Glossary/f/FQDN/ Glossary/f/FQDN.md glossary/d-f.md#fqdn -https://tarf.co.uk/Glossary/f/Frame%20Relay/ Glossary/f/Frame Relay.md glossary/d-f.md#frame-relay -https://tarf.co.uk/Glossary/f/Frequent%20Releases/ Glossary/f/Frequent Releases.md glossary/d-f.md#frequent-releases -https://tarf.co.uk/Glossary/f/Frequently%20Asked%20Question/ Glossary/f/Frequently Asked Question.md glossary/d-f.md#frequently-asked-question -https://tarf.co.uk/Glossary/f/FSK/ Glossary/f/FSK.md glossary/d-f.md#fsk -https://tarf.co.uk/Glossary/f/FTP/ Glossary/f/FTP.md glossary/d-f.md#ftp -https://tarf.co.uk/Glossary/f/FTPS/ Glossary/f/FTPS.md glossary/d-f.md#ftps -https://tarf.co.uk/Glossary/f/Full%20duplex/ Glossary/f/Full duplex.md glossary/d-f.md#full-duplex -https://tarf.co.uk/Glossary/f/Function%20Point/ Glossary/f/Function Point.md glossary/d-f.md#function-point -https://tarf.co.uk/Glossary/f/Functional%20Requirement/ Glossary/f/Functional Requirement.md glossary/d-f.md#functional-requirement -https://tarf.co.uk/Glossary/f/Functional%20Testing/ Glossary/f/Functional Testing.md glossary/d-f.md#functional-testing -https://tarf.co.uk/Glossary/f/Fuzzy%20Front%20End/ Glossary/f/Fuzzy Front End.md glossary/d-f.md#fuzzy-front-end -https://tarf.co.uk/Glossary/f/Fuzzy%20Logic/ Glossary/f/Fuzzy Logic.md glossary/d-f.md#fuzzy-logic -https://tarf.co.uk/Glossary/f/FXO/ Glossary/f/FXO.md glossary/d-f.md#fxo -https://tarf.co.uk/Glossary/f/FXS/ Glossary/f/FXS.md glossary/d-f.md#fxs -https://tarf.co.uk/Glossary/g/G.711.1/ Glossary/g/G.711.1.md glossary/g-k.md#g7111 -https://tarf.co.uk/Glossary/g/G.711/ Glossary/g/G.711.md glossary/g-k.md#g711 -https://tarf.co.uk/Glossary/g/G.719/ Glossary/g/G.719.md glossary/g-k.md#g719 -https://tarf.co.uk/Glossary/g/G.722/ Glossary/g/G.722.md glossary/g-k.md#g722 -https://tarf.co.uk/Glossary/g/G.723.1/ Glossary/g/G.723.1.md glossary/g-k.md#g7231 -https://tarf.co.uk/Glossary/g/G.726/ Glossary/g/G.726.md glossary/g-k.md#g726 -https://tarf.co.uk/Glossary/g/G.729/ Glossary/g/G.729.md glossary/a-c.md#audio-codecs renamed to "Audio Codecs" -https://tarf.co.uk/Glossary/g/G.Fast/ Glossary/g/G.Fast.md glossary/g-k.md#gfast -https://tarf.co.uk/Glossary/g/g/ Glossary/g/g.md glossary/g-k.md letter index -https://tarf.co.uk/Glossary/g/Gantt%20chart/ Glossary/g/Gantt chart.md glossary/g-k.md#gantt-chart -https://tarf.co.uk/Glossary/g/gap%20analysis/ Glossary/g/gap analysis.md glossary/g-k.md#gap-analysis -https://tarf.co.uk/Glossary/g/Gate%20Checklist/ Glossary/g/Gate Checklist.md glossary/g-k.md#gate-checklist -https://tarf.co.uk/Glossary/g/Gate/ Glossary/g/Gate.md glossary/g-k.md#gate -https://tarf.co.uk/Glossary/g/GBE%20-%20GE/ Glossary/g/GBE - GE.md glossary/g-k.md#gbe-ge renamed to "GBE / GE" -https://tarf.co.uk/Glossary/g/GCHQ/ Glossary/g/GCHQ.md glossary/g-k.md#gchq -https://tarf.co.uk/Glossary/g/Geocast/ Glossary/g/Geocast.md glossary/g-k.md#geocast -https://tarf.co.uk/Glossary/g/GERAN/ Glossary/g/GERAN.md glossary/g-k.md#geran -https://tarf.co.uk/Glossary/g/GGSN/ Glossary/g/GGSN.md glossary/g-k.md#ggsn -https://tarf.co.uk/Glossary/g/Gherkin/ Glossary/g/Gherkin.md glossary/g-k.md#gherkin -https://tarf.co.uk/Glossary/g/Gigabit%20Ethernet/ Glossary/g/Gigabit Ethernet.md glossary/g-k.md#gigabit-ethernet -https://tarf.co.uk/Glossary/g/Git/ Glossary/g/Git.md glossary/g-k.md#git -https://tarf.co.uk/Glossary/g/Given%20When%20Then/ Glossary/g/Given When Then.md glossary/g-k.md#given-when-then -https://tarf.co.uk/Glossary/g/GMSC/ Glossary/g/GMSC.md glossary/g-k.md#gmsc -https://tarf.co.uk/Glossary/g/GNU/ Glossary/g/GNU.md glossary/g-k.md#gnu -https://tarf.co.uk/Glossary/g/Go/ Glossary/g/Go.md glossary/g-k.md#go -https://tarf.co.uk/Glossary/g/Goals/ Glossary/g/Goals.md glossary/g-k.md#goals -https://tarf.co.uk/Glossary/g/GoF/ Glossary/g/GoF.md glossary/g-k.md#gof -https://tarf.co.uk/Glossary/g/Google%20Cloud%20Platform/ Glossary/g/Google Cloud Platform.md glossary/g-k.md#google-cloud-platform -https://tarf.co.uk/Glossary/g/GPL/ Glossary/g/GPL.md glossary/g-k.md#gpl -https://tarf.co.uk/Glossary/g/GPRS/ Glossary/g/GPRS.md glossary/g-k.md#gprs -https://tarf.co.uk/Glossary/g/GSM%20AMR/ Glossary/g/GSM AMR.md glossary/g-k.md#gsm-amr -https://tarf.co.uk/Glossary/g/GSM-FR/ Glossary/g/GSM-FR.md glossary/g-k.md#gsm-fr -https://tarf.co.uk/Glossary/g/GSM-R/ Glossary/g/GSM-R.md glossary/g-k.md#gsm-r -https://tarf.co.uk/Glossary/g/GSM/ Glossary/g/GSM.md glossary/g-k.md#gsm -https://tarf.co.uk/Glossary/g/GSN/ Glossary/g/GSN.md glossary/g-k.md#gsn -https://tarf.co.uk/Glossary/g/The%20first%20example%20has%20two%20steps/ Glossary/g/The first example has two steps.md glossary/g-k.md#gherkin renamed to "Gherkin" -https://tarf.co.uk/Glossary/g/The%20second%20example%20has%20three%20steps/ Glossary/g/The second example has three steps.md glossary/g-k.md#gherkin renamed to "Gherkin" -https://tarf.co.uk/Glossary/h/H.248/ Glossary/h/H.248.md glossary/g-k.md#h248 -https://tarf.co.uk/Glossary/h/H.323/ Glossary/h/H.323.md glossary/g-k.md#h323 -https://tarf.co.uk/Glossary/h/h/ Glossary/h/h.md glossary/g-k.md letter index -https://tarf.co.uk/Glossary/h/Half%20Duplex/ Glossary/h/Half Duplex.md glossary/g-k.md#half-duplex -https://tarf.co.uk/Glossary/h/Happy%20Path/ Glossary/h/Happy Path.md glossary/g-k.md#happy-path -https://tarf.co.uk/Glossary/h/Hash%20Function/ Glossary/h/Hash Function.md glossary/g-k.md#hash-function -https://tarf.co.uk/Glossary/h/HDF/ Glossary/h/HDF.md glossary/g-k.md#hdf -https://tarf.co.uk/Glossary/h/HDLC/ Glossary/h/HDLC.md glossary/g-k.md#hdlc -https://tarf.co.uk/Glossary/h/HDSL/ Glossary/h/HDSL.md glossary/g-k.md#hdsl -https://tarf.co.uk/Glossary/h/Heartbeat%20Retrospective/ Glossary/h/Heartbeat Retrospective.md glossary/g-k.md#heartbeat-retrospective -https://tarf.co.uk/Glossary/h/HiFi%20Prototype/ Glossary/h/HiFi Prototype.md glossary/g-k.md#hifi-prototype -https://tarf.co.uk/Glossary/h/High%20Fidelity%20Prototype/ Glossary/h/High Fidelity Prototype.md glossary/g-k.md#high-fidelity-prototype -https://tarf.co.uk/Glossary/h/High%20Level%20Design/ Glossary/h/High Level Design.md glossary/g-k.md#high-level-design -https://tarf.co.uk/Glossary/h/HLR/ Glossary/h/HLR.md glossary/g-k.md#hlr -https://tarf.co.uk/Glossary/h/Hotfix/ Glossary/h/Hotfix.md glossary/g-k.md#hotfix -https://tarf.co.uk/Glossary/h/How%20Requirement/ Glossary/h/How Requirement.md glossary/g-k.md#how-requirement -https://tarf.co.uk/Glossary/h/How%20Well%20Requirement/ Glossary/h/How Well Requirement.md glossary/g-k.md#how-well-requirement -https://tarf.co.uk/Glossary/h/HSRP/ Glossary/h/HSRP.md glossary/g-k.md#hsrp -https://tarf.co.uk/Glossary/h/HSS/ Glossary/h/HSS.md glossary/g-k.md#hss -https://tarf.co.uk/Glossary/h/HTTP/ Glossary/h/HTTP.md glossary/g-k.md#http -https://tarf.co.uk/Glossary/h/HTTPS/ Glossary/h/HTTPS.md glossary/g-k.md#https -https://tarf.co.uk/Glossary/h/Hub/ Glossary/h/Hub.md glossary/g-k.md#hub -https://tarf.co.uk/Glossary/h/human%20factors/ Glossary/h/human factors.md glossary/g-k.md#human-factors -https://tarf.co.uk/Glossary/h/Hungarian%20notation/ Glossary/h/Hungarian notation.md glossary/g-k.md#hungarian-notation -https://tarf.co.uk/Glossary/i/BRI/ Glossary/i/BRI.md glossary/a-c.md#bri -https://tarf.co.uk/Glossary/i/i/ Glossary/i/i.md glossary/g-k.md letter index -https://tarf.co.uk/Glossary/i/IA/ Glossary/i/IA.md glossary/g-k.md#ia -https://tarf.co.uk/Glossary/i/IaaS/ Glossary/i/IaaS.md glossary/g-k.md#iaas -https://tarf.co.uk/Glossary/i/ICE/ Glossary/i/ICE.md glossary/g-k.md#ice -https://tarf.co.uk/Glossary/i/ICMP/ Glossary/i/ICMP.md glossary/g-k.md#icmp -https://tarf.co.uk/Glossary/i/ICMPv6/ Glossary/i/ICMPv6.md glossary/g-k.md#icmpv6 -https://tarf.co.uk/Glossary/i/IDE/ Glossary/i/IDE.md glossary/g-k.md#ide -https://tarf.co.uk/Glossary/i/IDQ/ Glossary/i/IDQ.md glossary/g-k.md#idq -https://tarf.co.uk/Glossary/i/IEC/ Glossary/i/IEC.md glossary/g-k.md#iec -https://tarf.co.uk/Glossary/i/IEEE%20802.1ah/ Glossary/i/IEEE 802.1ah.md glossary/g-k.md#ieee-8021ah -https://tarf.co.uk/Glossary/i/IEEE/ Glossary/i/IEEE.md glossary/g-k.md#ieee -https://tarf.co.uk/Glossary/i/IETF/ Glossary/i/IETF.md glossary/g-k.md#ietf -https://tarf.co.uk/Glossary/i/IGMP/ Glossary/i/IGMP.md glossary/g-k.md#igmp -https://tarf.co.uk/Glossary/i/IGP/ Glossary/i/IGP.md glossary/g-k.md#igp -https://tarf.co.uk/Glossary/i/IGRP/ Glossary/i/IGRP.md glossary/g-k.md#igrp -https://tarf.co.uk/Glossary/i/IIS/ Glossary/i/IIS.md glossary/g-k.md#iis -https://tarf.co.uk/Glossary/i/IKE/ Glossary/i/IKE.md glossary/g-k.md#ike -https://tarf.co.uk/Glossary/i/iLBC/ Glossary/i/iLBC.md glossary/g-k.md#ilbc -https://tarf.co.uk/Glossary/i/iManager/ Glossary/i/iManager.md glossary/g-k.md#imanager -https://tarf.co.uk/Glossary/i/IMAP/ Glossary/i/IMAP.md glossary/g-k.md#imap -https://tarf.co.uk/Glossary/i/Implementation%20Lead/ Glossary/i/Implementation Lead.md glossary/g-k.md#implementation-lead -https://tarf.co.uk/Glossary/i/Implementation/ Glossary/i/Implementation.md glossary/g-k.md#implementation -https://tarf.co.uk/Glossary/i/Implicit%20Change%20Control/ Glossary/i/Implicit Change Control.md glossary/g-k.md#implicit-change-control -https://tarf.co.uk/Glossary/i/Implicit%20Risk%20Management/ Glossary/i/Implicit Risk Management.md glossary/g-k.md#implicit-risk-management -https://tarf.co.uk/Glossary/i/IMS/ Glossary/i/IMS.md glossary/g-k.md#ims -https://tarf.co.uk/Glossary/i/IMSI/ Glossary/i/IMSI.md glossary/g-k.md#imsi -https://tarf.co.uk/Glossary/i/IN/ Glossary/i/IN.md glossary/g-k.md#in -https://tarf.co.uk/Glossary/i/INAP/ Glossary/i/INAP.md glossary/g-k.md#inap -https://tarf.co.uk/Glossary/i/Inch-Pebble/ Glossary/i/Inch-Pebble.md glossary/g-k.md#inch-pebble -https://tarf.co.uk/Glossary/i/Incremental%20Development/ Glossary/i/Incremental Development.md glossary/g-k.md#incremental-development -https://tarf.co.uk/Glossary/i/Informal%20Review/ Glossary/i/Informal Review.md glossary/g-k.md#informal-review -https://tarf.co.uk/Glossary/i/Informal%20Testing/ Glossary/i/Informal Testing.md glossary/g-k.md#informal-testing -https://tarf.co.uk/Glossary/i/information%20architecture/ Glossary/i/information architecture.md glossary/g-k.md#information-architecture -https://tarf.co.uk/Glossary/i/information%20design/ Glossary/i/information design.md glossary/g-k.md#information-design -https://tarf.co.uk/Glossary/i/Information%20Radiators/ Glossary/i/Information Radiators.md glossary/g-k.md#information-radiators -https://tarf.co.uk/Glossary/i/Ingress/ Glossary/i/Ingress.md glossary/g-k.md#ingress -https://tarf.co.uk/Glossary/i/Inspection/ Glossary/i/Inspection.md glossary/g-k.md#inspection -https://tarf.co.uk/Glossary/i/Inspector/ Glossary/i/Inspector.md glossary/g-k.md#inspector -https://tarf.co.uk/Glossary/i/Institute%20of%20Electrical%20and%20Electronics%20Engineers/ Glossary/i/Institute of Electrical and Electronics Engineers.md glossary/g-k.md#institute-of-electrical-and-electronics-engineers -https://tarf.co.uk/Glossary/i/Integrated%20Development%20Environment/ Glossary/i/Integrated Development Environment.md glossary/g-k.md#integrated-development-environment -https://tarf.co.uk/Glossary/i/Integration%20Test/ Glossary/i/Integration Test.md glossary/g-k.md#integration-test -https://tarf.co.uk/Glossary/i/Integration/ Glossary/i/Integration.md glossary/g-k.md#integration -https://tarf.co.uk/Glossary/i/International%20Organization%20for%20Standardization/ Glossary/i/International Organization for Standardization.md glossary/g-k.md#iso renamed to "ISO" -https://tarf.co.uk/Glossary/i/Internet%20Protocol%20Suite/ Glossary/i/Internet Protocol Suite.md glossary/g-k.md#internet-protocol-suite -https://tarf.co.uk/Glossary/i/Interpreted%20Language/ Glossary/i/Interpreted Language.md glossary/g-k.md#interpreted-language -https://tarf.co.uk/Glossary/i/Intrinsic%20Risk%20Management/ Glossary/i/Intrinsic Risk Management.md glossary/g-k.md#intrinsic-risk-management -https://tarf.co.uk/Glossary/i/INVEST/ Glossary/i/INVEST.md glossary/g-k.md#invest -https://tarf.co.uk/Glossary/i/IOS%20cisco%20OS/ Glossary/i/IOS cisco OS.md glossary/g-k.md#ios renamed to "IOS" -https://tarf.co.uk/Glossary/i/IPS/ Glossary/i/IPS.md glossary/g-k.md#ips -https://tarf.co.uk/Glossary/i/IPSC/ Glossary/i/IPSC.md glossary/g-k.md#ipsc -https://tarf.co.uk/Glossary/i/IPSec/ Glossary/i/IPSec.md glossary/g-k.md#ipsec -https://tarf.co.uk/Glossary/i/IPTV/ Glossary/i/IPTV.md glossary/g-k.md#iptv -https://tarf.co.uk/Glossary/i/IPv4/ Glossary/i/IPv4.md glossary/g-k.md#ipv4 -https://tarf.co.uk/Glossary/i/IPv6/ Glossary/i/IPv6.md glossary/g-k.md#ipv6 -https://tarf.co.uk/Glossary/i/ISDN/ Glossary/i/ISDN.md glossary/g-k.md#isdn -https://tarf.co.uk/Glossary/i/ISDN2/ Glossary/i/ISDN2.md glossary/g-k.md#isdn2 -https://tarf.co.uk/Glossary/i/ISDN30/ Glossary/i/ISDN30.md glossary/g-k.md#isdn30 -https://tarf.co.uk/Glossary/i/ISO/ Glossary/i/ISO.md glossary/g-k.md#iso -https://tarf.co.uk/Glossary/i/ISP/ Glossary/i/ISP.md glossary/g-k.md#isp -https://tarf.co.uk/Glossary/i/Issue%20Management/ Glossary/i/Issue Management.md glossary/g-k.md#issue-management -https://tarf.co.uk/Glossary/i/Issue%20tracker/ Glossary/i/Issue tracker.md glossary/g-k.md#issue-tracker -https://tarf.co.uk/Glossary/i/Issue/ Glossary/i/Issue.md glossary/g-k.md#issue -https://tarf.co.uk/Glossary/i/ISUP/ Glossary/i/ISUP.md glossary/g-k.md#isup -https://tarf.co.uk/Glossary/i/Item/ Glossary/i/Item.md glossary/g-k.md#item -https://tarf.co.uk/Glossary/i/Iteration/ Glossary/i/Iteration.md glossary/g-k.md#iteration -https://tarf.co.uk/Glossary/i/Iterative%20Development/ Glossary/i/Iterative Development.md glossary/g-k.md#iterative-development -https://tarf.co.uk/Glossary/i/ITIL/ Glossary/i/ITIL.md glossary/g-k.md#itil -https://tarf.co.uk/Glossary/i/ITU-T/ Glossary/i/ITU-T.md glossary/g-k.md#itu-t -https://tarf.co.uk/Glossary/i/ITU/ Glossary/i/ITU.md glossary/g-k.md#itu -https://tarf.co.uk/Glossary/i/IUP/ Glossary/i/IUP.md glossary/g-k.md#iup -https://tarf.co.uk/Glossary/i/IVR/ Glossary/i/IVR.md glossary/g-k.md#ivr -https://tarf.co.uk/Glossary/i/PRI/ Glossary/i/PRI.md glossary/o-r.md#pri -https://tarf.co.uk/Glossary/j/j/ Glossary/j/j.md glossary/g-k.md letter index -https://tarf.co.uk/Glossary/j/Java/ Glossary/j/Java.md glossary/g-k.md#java -https://tarf.co.uk/Glossary/j/Javascript/ Glossary/j/Javascript.md glossary/g-k.md#javascript -https://tarf.co.uk/Glossary/j/Jitter%20buffer/ Glossary/j/Jitter buffer.md glossary/g-k.md#jitter-buffer -https://tarf.co.uk/Glossary/j/Jitter/ Glossary/j/Jitter.md glossary/g-k.md#jitter -https://tarf.co.uk/Glossary/j/Joint%20Application%20Development/ Glossary/j/Joint Application Development.md glossary/g-k.md#joint-application-development -https://tarf.co.uk/Glossary/j/JSON-RPC/ Glossary/j/JSON-RPC.md glossary/g-k.md#json-rpc -https://tarf.co.uk/Glossary/j/JSON/ Glossary/j/JSON.md glossary/g-k.md#json -https://tarf.co.uk/Glossary/k/K%20%26%20R%20C/ Glossary/k/K & R C.md glossary/g-k.md#k-r-c -https://tarf.co.uk/Glossary/k/K%20%26%20R%20indent%20style/ Glossary/k/K & R indent style.md glossary/g-k.md#k-r-indent-style -https://tarf.co.uk/Glossary/k/K%20%26%20R/ Glossary/k/K & R.md glossary/g-k.md#k-r -https://tarf.co.uk/Glossary/k/k/ Glossary/k/k.md glossary/g-k.md letter index -https://tarf.co.uk/Glossary/k/Kanban%20Board/ Glossary/k/Kanban Board.md glossary/g-k.md#kanban-board -https://tarf.co.uk/Glossary/k/Kanban/ Glossary/k/Kanban.md glossary/g-k.md#kanban -https://tarf.co.uk/Glossary/k/KDL/ Glossary/k/KDL.md glossary/g-k.md#kdl -https://tarf.co.uk/Glossary/k/Kerberos/ Glossary/k/Kerberos.md glossary/g-k.md#kerberos -https://tarf.co.uk/Glossary/k/Kernel/ Glossary/k/Kernel.md glossary/g-k.md#kernel -https://tarf.co.uk/Glossary/k/KISS%20Principle/ Glossary/k/KISS Principle.md glossary/g-k.md#kiss-principle -https://tarf.co.uk/Glossary/k/Knowledge%20Management/ Glossary/k/Knowledge Management.md glossary/g-k.md#knowledge-management -https://tarf.co.uk/Glossary/k/KPI/ Glossary/k/KPI.md glossary/g-k.md#kpi -https://tarf.co.uk/Glossary/k/ksh/ Glossary/k/ksh.md glossary/g-k.md#ksh -https://tarf.co.uk/Glossary/l/l/ Glossary/l/l.md glossary/l-n.md letter index -https://tarf.co.uk/Glossary/l/L1/ Glossary/l/L1.md glossary/l-n.md#l1 -https://tarf.co.uk/Glossary/l/L2/ Glossary/l/L2.md glossary/l-n.md#l2 -https://tarf.co.uk/Glossary/l/L2F/ Glossary/l/L2F.md glossary/l-n.md#l2f -https://tarf.co.uk/Glossary/l/L2S/ Glossary/l/L2S.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/l/L2TP/ Glossary/l/L2TP.md glossary/l-n.md#l2tp -https://tarf.co.uk/Glossary/l/L2VPN/ Glossary/l/L2VPN.md glossary/l-n.md#l2vpn -https://tarf.co.uk/Glossary/l/L3VPN/ Glossary/l/L3VPN.md glossary/l-n.md#l3vpn -https://tarf.co.uk/Glossary/l/LAC/ Glossary/l/LAC.md glossary/l-n.md#lac -https://tarf.co.uk/Glossary/l/LACP/ Glossary/l/LACP.md glossary/l-n.md#lacp -https://tarf.co.uk/Glossary/l/LAG/ Glossary/l/LAG.md glossary/l-n.md#lag -https://tarf.co.uk/Glossary/l/LAN/ Glossary/l/LAN.md glossary/l-n.md#lan -https://tarf.co.uk/Glossary/l/LANE/ Glossary/l/LANE.md glossary/l-n.md#lane -https://tarf.co.uk/Glossary/l/LAS/ Glossary/l/LAS.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/l/Latency/ Glossary/l/Latency.md glossary/l-n.md#latency -https://tarf.co.uk/Glossary/l/Lawful%20intercept/ Glossary/l/Lawful intercept.md glossary/l-n.md#lawful-intercept -https://tarf.co.uk/Glossary/l/LCR/ Glossary/l/LCR.md glossary/l-n.md#lcr -https://tarf.co.uk/Glossary/l/LDAP/ Glossary/l/LDAP.md glossary/l-n.md#ldap -https://tarf.co.uk/Glossary/l/Lead%20Time/ Glossary/l/Lead Time.md glossary/l-n.md#lead-time -https://tarf.co.uk/Glossary/l/lean%20programming/ Glossary/l/lean programming.md glossary/l-n.md#lean-programming -https://tarf.co.uk/Glossary/l/Lean/ Glossary/l/Lean.md glossary/l-n.md#lean -https://tarf.co.uk/Glossary/l/Leased%20Line/ Glossary/l/Leased Line.md glossary/l-n.md#leased-line -https://tarf.co.uk/Glossary/l/LeSS/ Glossary/l/LeSS.md glossary/l-n.md#less -https://tarf.co.uk/Glossary/l/Lifecycle/ Glossary/l/Lifecycle.md glossary/l-n.md#lifecycle -https://tarf.co.uk/Glossary/l/Lines%20of%20Code/ Glossary/l/Lines of Code.md glossary/l-n.md#lines-of-code -https://tarf.co.uk/Glossary/l/Link/ Glossary/l/Link.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/l/Linux/ Glossary/l/Linux.md glossary/l-n.md#linux -https://tarf.co.uk/Glossary/l/lisp/ Glossary/l/lisp.md glossary/l-n.md#lisp -https://tarf.co.uk/Glossary/l/Lite/ Glossary/l/Lite.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/l/LL/ Glossary/l/LL.md glossary/l-n.md#ll -https://tarf.co.uk/Glossary/l/LNS/ Glossary/l/LNS.md glossary/l-n.md#lns -https://tarf.co.uk/Glossary/l/Local%20Build/ Glossary/l/Local Build.md glossary/l-n.md#local-build -https://tarf.co.uk/Glossary/l/Local%20Development%20Environment/ Glossary/l/Local Development Environment.md glossary/l-n.md#local-development-environment -https://tarf.co.uk/Glossary/l/Local%20Loop/ Glossary/l/Local Loop.md glossary/l-n.md#local-loop -https://tarf.co.uk/Glossary/l/Local%20Test%20Environment/ Glossary/l/Local Test Environment.md glossary/l-n.md#local-test-environment -https://tarf.co.uk/Glossary/l/LoFi%20Prototype/ Glossary/l/LoFi Prototype.md glossary/l-n.md#lofi-prototype -https://tarf.co.uk/Glossary/l/Low%20Fidelity%20Prototype/ Glossary/l/Low Fidelity Prototype.md glossary/l-n.md#low-fidelity-prototype -https://tarf.co.uk/Glossary/l/Low%20Level%20Design/ Glossary/l/Low Level Design.md glossary/l-n.md#low-level-design -https://tarf.co.uk/Glossary/l/LPS/ Glossary/l/LPS.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/l/LSP/ Glossary/l/LSP.md glossary/l-n.md#lsp -https://tarf.co.uk/Glossary/l/LSR/ Glossary/l/LSR.md glossary/l-n.md#lsr -https://tarf.co.uk/Glossary/l/LTE%20%284G%29/ Glossary/l/LTE (4G).md glossary/l-n.md#lte-4g -https://tarf.co.uk/Glossary/l/LTE%20and%20Mobile%20Internet/ Glossary/l/LTE and Mobile Internet.md glossary/l-n.md#lte-4g renamed to "LTE (4G)" -https://tarf.co.uk/Glossary/l/LTS/ Glossary/l/LTS.md glossary/l-n.md#lts -https://tarf.co.uk/Glossary/l/Lua/ Glossary/l/Lua.md glossary/l-n.md#lua -https://tarf.co.uk/Glossary/m/m/ Glossary/m/m.md glossary/l-n.md letter index -https://tarf.co.uk/Glossary/m/M2M/ Glossary/m/M2M.md glossary/l-n.md#m2m -https://tarf.co.uk/Glossary/m/MAC%20Address/ Glossary/m/MAC Address.md glossary/l-n.md#mac-address -https://tarf.co.uk/Glossary/m/MAC%20Flooding/ Glossary/m/MAC Flooding.md glossary/l-n.md#mac-flooding -https://tarf.co.uk/Glossary/m/MAC%20Spoofing/ Glossary/m/MAC Spoofing.md glossary/l-n.md#mac-spoofing -https://tarf.co.uk/Glossary/m/MAC/ Glossary/m/MAC.md glossary/l-n.md#mac -https://tarf.co.uk/Glossary/m/Machine%20code/ Glossary/m/Machine code.md glossary/l-n.md#machine-code -https://tarf.co.uk/Glossary/m/Macro%20Estimation/ Glossary/m/Macro Estimation.md glossary/l-n.md#macro-estimation -https://tarf.co.uk/Glossary/m/MAG/ Glossary/m/MAG.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/m/Maintenance%20Plan/ Glossary/m/Maintenance Plan.md glossary/l-n.md#maintenance-plan -https://tarf.co.uk/Glossary/m/Maintenance/ Glossary/m/Maintenance.md glossary/l-n.md#maintenance -https://tarf.co.uk/Glossary/m/MAN/ Glossary/m/MAN.md glossary/l-n.md#man -https://tarf.co.uk/Glossary/m/Management%20by%20Walking%20Around/ Glossary/m/Management by Walking Around.md glossary/l-n.md#management-by-walking-around -https://tarf.co.uk/Glossary/m/Management%20Plane/ Glossary/m/Management Plane.md glossary/l-n.md#management-plane -https://tarf.co.uk/Glossary/m/MANET/ Glossary/m/MANET.md glossary/l-n.md#manet -https://tarf.co.uk/Glossary/m/Manual%20Testing/ Glossary/m/Manual Testing.md glossary/l-n.md#manual-testing -https://tarf.co.uk/Glossary/m/MAP/ Glossary/m/MAP.md glossary/l-n.md#map -https://tarf.co.uk/Glossary/m/MAS/ Glossary/m/MAS.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/m/Material/ Glossary/m/Material.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/m/MBMS/ Glossary/m/MBMS.md glossary/l-n.md#mbms -https://tarf.co.uk/Glossary/m/MBORC/ Glossary/m/MBORC.md glossary/l-n.md#mborc -https://tarf.co.uk/Glossary/m/MD5/ Glossary/m/MD5.md glossary/l-n.md#md5 -https://tarf.co.uk/Glossary/m/MDF/ Glossary/m/MDF.md glossary/l-n.md#mdf -https://tarf.co.uk/Glossary/m/Mediation/ Glossary/m/Mediation.md glossary/l-n.md#mediation -https://tarf.co.uk/Glossary/m/MELP/ Glossary/m/MELP.md glossary/l-n.md#melp -https://tarf.co.uk/Glossary/m/Metric/ Glossary/m/Metric.md glossary/l-n.md#metric -https://tarf.co.uk/Glossary/m/MGCF/ Glossary/m/MGCF.md glossary/l-n.md#mgcf -https://tarf.co.uk/Glossary/m/MGCP/ Glossary/m/MGCP.md glossary/l-n.md#mgcp -https://tarf.co.uk/Glossary/m/MGW/ Glossary/m/MGW.md glossary/l-n.md#mgw -https://tarf.co.uk/Glossary/m/MIB/ Glossary/m/MIB.md glossary/l-n.md#mib -https://tarf.co.uk/Glossary/m/Micro%20Estimation/ Glossary/m/Micro Estimation.md glossary/l-n.md#micro-estimation -https://tarf.co.uk/Glossary/m/Microservice/ Glossary/m/Microservice.md glossary/l-n.md#microservice -https://tarf.co.uk/Glossary/m/Microstone/ Glossary/m/Microstone.md glossary/l-n.md#microstone -https://tarf.co.uk/Glossary/m/Mil-COTS/ Glossary/m/Mil-COTS.md glossary/l-n.md#mil-cots -https://tarf.co.uk/Glossary/m/Milestone%20Release/ Glossary/m/Milestone Release.md glossary/l-n.md#milestone-release -https://tarf.co.uk/Glossary/m/Milestone%20Retrospective/ Glossary/m/Milestone Retrospective.md glossary/l-n.md#milestone-retrospective -https://tarf.co.uk/Glossary/m/Milestone%20Schedule/ Glossary/m/Milestone Schedule.md glossary/l-n.md#milestone-schedule -https://tarf.co.uk/Glossary/m/Milestone/ Glossary/m/Milestone.md glossary/l-n.md#milestone -https://tarf.co.uk/Glossary/m/MIMO/ Glossary/m/MIMO.md glossary/l-n.md#mimo -https://tarf.co.uk/Glossary/m/Miniature%20Milestone/ Glossary/m/Miniature Milestone.md glossary/l-n.md#miniature-milestone -https://tarf.co.uk/Glossary/m/MME/ Glossary/m/MME.md glossary/l-n.md#mme -https://tarf.co.uk/Glossary/m/MMF/ Glossary/m/MMF.md glossary/l-n.md#mmf -https://tarf.co.uk/Glossary/m/MNC/ Glossary/m/MNC.md glossary/l-n.md#mnc -https://tarf.co.uk/Glossary/m/Mob%20Programming/ Glossary/m/Mob Programming.md glossary/l-n.md#mob-programming -https://tarf.co.uk/Glossary/m/Mobile%20Cell/ Glossary/m/Mobile Cell.md glossary/l-n.md#mobile-cell -https://tarf.co.uk/Glossary/m/Mobile%20Networks%202G-3G/ Glossary/m/Mobile Networks 2G-3G.md glossary/s-z.md#umts renamed to "UMTS" -https://tarf.co.uk/Glossary/m/Mock%20Objects/ Glossary/m/Mock Objects.md glossary/l-n.md#mock-objects -https://tarf.co.uk/Glossary/m/Model/ Glossary/m/Model.md glossary/l-n.md#model -https://tarf.co.uk/Glossary/m/Modeling/ Glossary/m/Modeling.md glossary/l-n.md#modelling renamed to "Modelling" -https://tarf.co.uk/Glossary/m/Moderator/ Glossary/m/Moderator.md glossary/l-n.md#moderator -https://tarf.co.uk/Glossary/m/MOS/ Glossary/m/MOS.md glossary/l-n.md#mos -https://tarf.co.uk/Glossary/m/MP3/ Glossary/m/MP3.md glossary/l-n.md#mp3 -https://tarf.co.uk/Glossary/m/MPEG/ Glossary/m/MPEG.md glossary/l-n.md#mpeg -https://tarf.co.uk/Glossary/m/MPEG4/ Glossary/m/MPEG4.md glossary/l-n.md#mpeg4 -https://tarf.co.uk/Glossary/m/MPF/ Glossary/m/MPF.md glossary/l-n.md#mpf -https://tarf.co.uk/Glossary/m/MPS/ Glossary/m/MPS.md glossary/l-n.md dropped -https://tarf.co.uk/Glossary/m/MQTT/ Glossary/m/MQTT.md glossary/l-n.md#mqtt -https://tarf.co.uk/Glossary/m/MRF/ Glossary/m/MRF.md glossary/l-n.md#mrf -https://tarf.co.uk/Glossary/m/MRS/ Glossary/m/MRS.md glossary/l-n.md#mrs -https://tarf.co.uk/Glossary/m/MRTIE/ Glossary/m/MRTIE.md glossary/l-n.md#mrtie -https://tarf.co.uk/Glossary/m/MSAN/ Glossary/m/MSAN.md glossary/l-n.md#msan -https://tarf.co.uk/Glossary/m/MSC/ Glossary/m/MSC.md glossary/l-n.md#msc -https://tarf.co.uk/Glossary/m/MTP/ Glossary/m/MTP.md glossary/l-n.md#mtp -https://tarf.co.uk/Glossary/m/MTR/ Glossary/m/MTR.md glossary/l-n.md#mtr -https://tarf.co.uk/Glossary/m/MTU/ Glossary/m/MTU.md glossary/l-n.md#mtu -https://tarf.co.uk/Glossary/m/Multicast/ Glossary/m/Multicast.md glossary/l-n.md#multicast -https://tarf.co.uk/Glossary/m/Multiplex/ Glossary/m/Multiplex.md glossary/l-n.md#multiplex -https://tarf.co.uk/Glossary/m/Mutex/ Glossary/m/Mutex.md glossary/l-n.md#mutex -https://tarf.co.uk/Glossary/m/Mutual%20Exclusion/ Glossary/m/Mutual Exclusion.md glossary/l-n.md#mutual-exclusion -https://tarf.co.uk/Glossary/m/MVNE/ Glossary/m/MVNE.md glossary/l-n.md#mvne -https://tarf.co.uk/Glossary/m/MVNO/ Glossary/m/MVNO.md glossary/l-n.md#mvno -https://tarf.co.uk/Glossary/m/MVP/ Glossary/m/MVP.md glossary/l-n.md#mvp -https://tarf.co.uk/Glossary/n/n/ Glossary/n/n.md glossary/l-n.md letter index -https://tarf.co.uk/Glossary/n/NaaS/ Glossary/n/NaaS.md glossary/l-n.md#naas -https://tarf.co.uk/Glossary/n/NAPT/ Glossary/n/NAPT.md glossary/l-n.md#napt -https://tarf.co.uk/Glossary/n/NAS/ Glossary/n/NAS.md glossary/l-n.md#nas -https://tarf.co.uk/Glossary/n/NAT/ Glossary/n/NAT.md glossary/l-n.md#nat -https://tarf.co.uk/Glossary/n/Native%20Risk%20Management/ Glossary/n/Native Risk Management.md glossary/l-n.md#native-risk-management -https://tarf.co.uk/Glossary/n/NBAP/ Glossary/n/NBAP.md glossary/l-n.md#nbap -https://tarf.co.uk/Glossary/n/NCSC/ Glossary/n/NCSC.md glossary/l-n.md#ncsc -https://tarf.co.uk/Glossary/n/Network%20Topologies/ Glossary/n/Network Topologies.md glossary/l-n.md#network-topologies -https://tarf.co.uk/Glossary/n/NFS/ Glossary/n/NFS.md glossary/l-n.md#nfs -https://tarf.co.uk/Glossary/n/NGN/ Glossary/n/NGN.md glossary/l-n.md#ngn -https://tarf.co.uk/Glossary/n/NGWFQ/ Glossary/n/NGWFQ.md glossary/l-n.md#ngwfq -https://tarf.co.uk/Glossary/n/Niko-niko%20Calendar/ Glossary/n/Niko-niko Calendar.md glossary/l-n.md#niko-niko-calendar -https://tarf.co.uk/Glossary/n/NIST/ Glossary/n/NIST.md glossary/l-n.md#nist -https://tarf.co.uk/Glossary/n/NNI/ Glossary/n/NNI.md glossary/l-n.md#nni -https://tarf.co.uk/Glossary/n/NNTP/ Glossary/n/NNTP.md glossary/l-n.md#nntp -https://tarf.co.uk/Glossary/n/NOC/ Glossary/n/NOC.md glossary/l-n.md#noc -https://tarf.co.uk/Glossary/n/Node%20B/ Glossary/n/Node B.md glossary/l-n.md#node-b -https://tarf.co.uk/Glossary/n/Nominal%20Path/ Glossary/n/Nominal Path.md glossary/l-n.md#nominal-path -https://tarf.co.uk/Glossary/n/Non-functional%20Requirement/ Glossary/n/Non-functional Requirement.md glossary/l-n.md#non-functional-requirement -https://tarf.co.uk/Glossary/n/NoSQL/ Glossary/n/NoSQL.md glossary/l-n.md#nosql -https://tarf.co.uk/Glossary/n/NPE/ Glossary/n/NPE.md glossary/l-n.md#npe -https://tarf.co.uk/Glossary/n/NSS/ Glossary/n/NSS.md glossary/l-n.md#nss -https://tarf.co.uk/Glossary/n/NTP/ Glossary/n/NTP.md glossary/l-n.md#ntp -https://tarf.co.uk/Glossary/n/NTS/ Glossary/n/NTS.md glossary/l-n.md#nts -https://tarf.co.uk/Glossary/n/NTU/ Glossary/n/NTU.md glossary/l-n.md#ntu -https://tarf.co.uk/Glossary/n/NYB/ Glossary/n/NYB.md glossary/l-n.md#nyb -https://tarf.co.uk/Glossary/o/o/ Glossary/o/o.md glossary/o-r.md letter index -https://tarf.co.uk/Glossary/o/Objective%20C/ Glossary/o/Objective C.md glossary/o-r.md#objective-c -https://tarf.co.uk/Glossary/o/Objectives/ Glossary/o/Objectives.md glossary/o-r.md#objectives -https://tarf.co.uk/Glossary/o/ODF/ Glossary/o/ODF.md glossary/o-r.md#odf -https://tarf.co.uk/Glossary/o/OFC/ Glossary/o/OFC.md glossary/o-r.md#ofc -https://tarf.co.uk/Glossary/o/Ofcom/ Glossary/o/Ofcom.md glossary/o-r.md#ofcom -https://tarf.co.uk/Glossary/o/OFDMA/ Glossary/o/OFDMA.md glossary/o-r.md#ofdma -https://tarf.co.uk/Glossary/o/OO/ Glossary/o/OO.md glossary/o-r.md#oo -https://tarf.co.uk/Glossary/o/Operations%20Plan/ Glossary/o/Operations Plan.md glossary/o-r.md#operations-plan -https://tarf.co.uk/Glossary/o/OPUS/ Glossary/o/OPUS.md glossary/o-r.md#opus -https://tarf.co.uk/Glossary/o/OS%20X/ Glossary/o/OS X.md glossary/o-r.md#os-x -https://tarf.co.uk/Glossary/o/OSA-FC/ Glossary/o/OSA-FC.md glossary/o-r.md#osa-fc -https://tarf.co.uk/Glossary/o/OSPF/ Glossary/o/OSPF.md glossary/o-r.md#ospf -https://tarf.co.uk/Glossary/o/OSS/ Glossary/o/OSS.md glossary/o-r.md#oss -https://tarf.co.uk/Glossary/o/OTN/ Glossary/o/OTN.md glossary/o-r.md#otn -https://tarf.co.uk/Glossary/o/Out-of-order%20delivery/ Glossary/o/Out-of-order delivery.md glossary/o-r.md#out-of-order-delivery -https://tarf.co.uk/Glossary/o/Overlap%20Dialling/ Glossary/o/Overlap Dialling.md glossary/o-r.md#overlap-dialling -https://tarf.co.uk/Glossary/p/p/ Glossary/p/p.md glossary/o-r.md letter index -https://tarf.co.uk/Glossary/p/PaaS/ Glossary/p/PaaS.md glossary/o-r.md#paas -https://tarf.co.uk/Glossary/p/packet%20sniffing/ Glossary/p/packet sniffing.md glossary/o-r.md#packet-sniffing -https://tarf.co.uk/Glossary/p/Packet%20Switched/ Glossary/p/Packet Switched.md glossary/o-r.md#packet-switched -https://tarf.co.uk/Glossary/p/PAF/ Glossary/p/PAF.md glossary/o-r.md#paf -https://tarf.co.uk/Glossary/p/Pair%20Programming/ Glossary/p/Pair Programming.md glossary/o-r.md#pair-programming -https://tarf.co.uk/Glossary/p/PAP/ Glossary/p/PAP.md glossary/o-r.md#pap -https://tarf.co.uk/Glossary/p/Parametric%20Estimation/ Glossary/p/Parametric Estimation.md glossary/o-r.md#parametric-estimation -https://tarf.co.uk/Glossary/p/Pascal/ Glossary/p/Pascal.md glossary/o-r.md#pascal -https://tarf.co.uk/Glossary/p/Pattern/ Glossary/p/Pattern.md glossary/o-r.md#pattern -https://tarf.co.uk/Glossary/p/PCM/ Glossary/p/PCM.md glossary/o-r.md#pcm -https://tarf.co.uk/Glossary/p/PCP/ Glossary/p/PCP.md glossary/o-r.md#pcp -https://tarf.co.uk/Glossary/p/PCRF/ Glossary/p/PCRF.md glossary/o-r.md#pcrf -https://tarf.co.uk/Glossary/p/PCS/ Glossary/p/PCS.md glossary/o-r.md#pcs -https://tarf.co.uk/Glossary/p/PDCP/ Glossary/p/PDCP.md glossary/o-r.md#pdcp -https://tarf.co.uk/Glossary/p/PDH/ Glossary/p/PDH.md glossary/o-r.md#pdh -https://tarf.co.uk/Glossary/p/PDU/ Glossary/p/PDU.md glossary/o-r.md#pdu -https://tarf.co.uk/Glossary/p/PEAP/ Glossary/p/PEAP.md glossary/o-r.md#peap -https://tarf.co.uk/Glossary/p/peer%20review/ Glossary/p/peer review.md glossary/o-r.md#peer-review -https://tarf.co.uk/Glossary/p/Pen%20testing/ Glossary/p/Pen testing.md glossary/o-r.md#pen-testing -https://tarf.co.uk/Glossary/p/PER/ Glossary/p/PER.md glossary/o-r.md#per -https://tarf.co.uk/Glossary/p/Performance/ Glossary/p/Performance.md glossary/o-r.md#performance -https://tarf.co.uk/Glossary/p/Perl/ Glossary/p/Perl.md glossary/o-r.md#perl -https://tarf.co.uk/Glossary/p/Personas/ Glossary/p/Personas.md glossary/o-r.md#personas -https://tarf.co.uk/Glossary/p/Perspective/ Glossary/p/Perspective.md glossary/o-r.md#perspective -https://tarf.co.uk/Glossary/p/PERT%20chart/ Glossary/p/PERT chart.md glossary/o-r.md#pert-chart -https://tarf.co.uk/Glossary/p/PGP/ Glossary/p/PGP.md glossary/o-r.md#pgp -https://tarf.co.uk/Glossary/p/PGW/ Glossary/p/PGW.md glossary/o-r.md#pgw -https://tarf.co.uk/Glossary/p/Phase/ Glossary/p/Phase.md glossary/o-r.md#phase -https://tarf.co.uk/Glossary/p/Phased%20Estimation/ Glossary/p/Phased Estimation.md glossary/o-r.md#phased-estimation -https://tarf.co.uk/Glossary/p/PHP/ Glossary/p/PHP.md glossary/o-r.md#php -https://tarf.co.uk/Glossary/p/Planning%20and%20Tracking%20Lead/ Glossary/p/Planning and Tracking Lead.md glossary/o-r.md#planning-and-tracking-lead -https://tarf.co.uk/Glossary/p/Planning%20Poker/ Glossary/p/Planning Poker.md glossary/o-r.md#planning-poker -https://tarf.co.uk/Glossary/p/PLL/ Glossary/p/PLL.md glossary/o-r.md#pll -https://tarf.co.uk/Glossary/p/PNT/ Glossary/p/PNT.md glossary/o-r.md#pnt -https://tarf.co.uk/Glossary/p/Points/ Glossary/p/Points.md glossary/o-r.md#points -https://tarf.co.uk/Glossary/p/polymorphism/ Glossary/p/polymorphism.md glossary/o-r.md#polymorphism -https://tarf.co.uk/Glossary/p/POP/ Glossary/p/POP.md glossary/o-r.md#pop -https://tarf.co.uk/Glossary/p/portability/ Glossary/p/portability.md glossary/o-r.md#portability -https://tarf.co.uk/Glossary/p/POSIX/ Glossary/p/POSIX.md glossary/o-r.md#posix -https://tarf.co.uk/Glossary/p/Postmortem/ Glossary/p/Postmortem.md glossary/o-r.md#postmortem -https://tarf.co.uk/Glossary/p/POTS/ Glossary/p/POTS.md glossary/o-r.md#pots -https://tarf.co.uk/Glossary/p/PPP/ Glossary/p/PPP.md glossary/o-r.md#ppp -https://tarf.co.uk/Glossary/p/PPPoE/ Glossary/p/PPPoE.md glossary/o-r.md#pppoe -https://tarf.co.uk/Glossary/p/PPTP/ Glossary/p/PPTP.md glossary/o-r.md#pptp -https://tarf.co.uk/Glossary/p/Preemption/ Glossary/p/Preemption.md glossary/o-r.md#preemption -https://tarf.co.uk/Glossary/p/Preview/ Glossary/p/Preview.md glossary/o-r.md#preview -https://tarf.co.uk/Glossary/p/PRINCE2/ Glossary/p/PRINCE2.md glossary/o-r.md#prince2 -https://tarf.co.uk/Glossary/p/Process%20Flow/ Glossary/p/Process Flow.md glossary/o-r.md#process-flow -https://tarf.co.uk/Glossary/p/Process/ Glossary/p/Process.md glossary/o-r.md#process -https://tarf.co.uk/Glossary/p/Product%20Owner/ Glossary/p/Product Owner.md glossary/o-r.md#product-owner -https://tarf.co.uk/Glossary/p/Product%20Release/ Glossary/p/Product Release.md glossary/o-r.md#product-release -https://tarf.co.uk/Glossary/p/programming%20language/ Glossary/p/programming language.md glossary/o-r.md#programming-language -https://tarf.co.uk/Glossary/p/Programming%20paradigm/ Glossary/p/Programming paradigm.md glossary/o-r.md#programming-paradigm -https://tarf.co.uk/Glossary/p/Progressive%20Elaboration/ Glossary/p/Progressive Elaboration.md glossary/o-r.md#progressive-elaboration -https://tarf.co.uk/Glossary/p/Project%20Build/ Glossary/p/Project Build.md glossary/o-r.md#project-build -https://tarf.co.uk/Glossary/p/Project%20Business%20Manager/ Glossary/p/Project Business Manager.md glossary/o-r.md#project-business-manager -https://tarf.co.uk/Glossary/p/Project%20Charter/ Glossary/p/Project Charter.md glossary/o-r.md#project-charter -https://tarf.co.uk/Glossary/p/Project%20Chartering/ Glossary/p/Project Chartering.md glossary/o-r.md#project-chartering -https://tarf.co.uk/Glossary/p/Project%20Estimate/ Glossary/p/Project Estimate.md glossary/o-r.md#project-estimate -https://tarf.co.uk/Glossary/p/Project%20Headlights/ Glossary/p/Project Headlights.md glossary/o-r.md#project-headlights -https://tarf.co.uk/Glossary/p/Project%20History/ Glossary/p/Project History.md glossary/o-r.md#project-history -https://tarf.co.uk/Glossary/p/Project%20Log/ Glossary/p/Project Log.md glossary/o-r.md#project-log -https://tarf.co.uk/Glossary/p/Project%20Management%20Body%20of%20Knowledge/ Glossary/p/Project Management Body of Knowledge.md glossary/o-r.md#project-management-body-of-knowledge -https://tarf.co.uk/Glossary/p/Project%20Management%20Institute/ Glossary/p/Project Management Institute.md glossary/o-r.md#project-management-institute -https://tarf.co.uk/Glossary/p/Project%20Management/ Glossary/p/Project Management.md glossary/o-r.md#project-management -https://tarf.co.uk/Glossary/p/Project%20Manager/ Glossary/p/Project Manager.md glossary/o-r.md#project-manager -https://tarf.co.uk/Glossary/p/Project%20Plan/ Glossary/p/Project Plan.md glossary/o-r.md#project-plan -https://tarf.co.uk/Glossary/p/Project%20Reviewer/ Glossary/p/Project Reviewer.md glossary/o-r.md#project-reviewer -https://tarf.co.uk/Glossary/p/Project%20Sponsor/ Glossary/p/Project Sponsor.md glossary/o-r.md#project-sponsor -https://tarf.co.uk/Glossary/p/Project/ Glossary/p/Project.md glossary/o-r.md#project -https://tarf.co.uk/Glossary/p/pseudocode/ Glossary/p/pseudocode.md glossary/o-r.md#pseudocode -https://tarf.co.uk/Glossary/p/PSTN/ Glossary/p/PSTN.md glossary/o-r.md#pstn -https://tarf.co.uk/Glossary/p/PTP/ Glossary/p/PTP.md glossary/o-r.md#ptp -https://tarf.co.uk/Glossary/p/PTT/ Glossary/p/PTT.md glossary/o-r.md#ptt -https://tarf.co.uk/Glossary/p/PVC/ Glossary/p/PVC.md glossary/o-r.md#pvc -https://tarf.co.uk/Glossary/p/PW/ Glossary/p/PW.md glossary/o-r.md#pw -https://tarf.co.uk/Glossary/p/PWFQ/ Glossary/p/PWFQ.md glossary/o-r.md#pwfq -https://tarf.co.uk/Glossary/p/Python%20%282.x%20%26%203.x%29/ Glossary/p/Python (2.x & 3.x).md glossary/o-r.md#python renamed to "Python" -https://tarf.co.uk/Glossary/q/Q.931/ Glossary/q/Q.931.md glossary/o-r.md#q931 -https://tarf.co.uk/Glossary/q/q/ Glossary/q/q.md glossary/o-r.md letter index -https://tarf.co.uk/Glossary/q/QA%20Lead/ Glossary/q/QA Lead.md glossary/o-r.md#qa-lead -https://tarf.co.uk/Glossary/q/QAM/ Glossary/q/QAM.md glossary/o-r.md#qam -https://tarf.co.uk/Glossary/q/QoS/ Glossary/q/QoS.md glossary/o-r.md#qos -https://tarf.co.uk/Glossary/q/QSIG/ Glossary/q/QSIG.md glossary/o-r.md#qsig -https://tarf.co.uk/Glossary/q/Quality%20Assurance%20Plan/ Glossary/q/Quality Assurance Plan.md glossary/o-r.md#quality-assurance-plan -https://tarf.co.uk/Glossary/q/Quality%20Assurance/ Glossary/q/Quality Assurance.md glossary/o-r.md#quality-assurance -https://tarf.co.uk/Glossary/q/Quality%20Control/ Glossary/q/Quality Control.md glossary/o-r.md#quality-control -https://tarf.co.uk/Glossary/q/Quality%20Plan/ Glossary/q/Quality Plan.md glossary/o-r.md#quality-plan -https://tarf.co.uk/Glossary/q/Quality/ Glossary/q/Quality.md glossary/o-r.md#quality -https://tarf.co.uk/Glossary/q/Quick%20Design%20Session/ Glossary/q/Quick Design Session.md glossary/o-r.md#quick-design-session -https://tarf.co.uk/Glossary/r/R%20Language/ Glossary/r/R Language.md glossary/o-r.md#r-language -https://tarf.co.uk/Glossary/r/r/ Glossary/r/r.md glossary/o-r.md letter index -https://tarf.co.uk/Glossary/r/RACI/ Glossary/r/RACI.md glossary/o-r.md#raci -https://tarf.co.uk/Glossary/r/RAD%20Environment/ Glossary/r/RAD Environment.md glossary/o-r.md#rad-environment -https://tarf.co.uk/Glossary/r/RAD/ Glossary/r/RAD.md glossary/o-r.md#rad -https://tarf.co.uk/Glossary/r/Radius/ Glossary/r/Radius.md glossary/o-r.md#radius -https://tarf.co.uk/Glossary/r/Rainbow/ Glossary/r/Rainbow.md glossary/o-r.md dropped -https://tarf.co.uk/Glossary/r/RAN/ Glossary/r/RAN.md glossary/o-r.md#ran -https://tarf.co.uk/Glossary/r/RANAP/ Glossary/r/RANAP.md glossary/o-r.md#ranap -https://tarf.co.uk/Glossary/r/RANCID/ Glossary/r/RANCID.md glossary/o-r.md#rancid -https://tarf.co.uk/Glossary/r/Rapid%20Application%20Development/ Glossary/r/Rapid Application Development.md glossary/o-r.md#rapid-application-development -https://tarf.co.uk/Glossary/r/RARP/ Glossary/r/RARP.md glossary/o-r.md#rarp -https://tarf.co.uk/Glossary/r/RAS/ Glossary/r/RAS.md glossary/o-r.md#ras -https://tarf.co.uk/Glossary/r/RDBMS/ Glossary/r/RDBMS.md glossary/o-r.md#rdbms -https://tarf.co.uk/Glossary/r/RDP/ Glossary/r/RDP.md glossary/o-r.md#rdp -https://tarf.co.uk/Glossary/r/Reading%20Inspection/ Glossary/r/Reading Inspection.md glossary/o-r.md#reading-inspection -https://tarf.co.uk/Glossary/r/refactoring/ Glossary/r/refactoring.md glossary/o-r.md#refactoring -https://tarf.co.uk/Glossary/r/Reference/ Glossary/r/Reference.md glossary/o-r.md dropped -https://tarf.co.uk/Glossary/r/regression%20testing/ Glossary/r/regression testing.md glossary/o-r.md#regression-testing -https://tarf.co.uk/Glossary/r/Regular%20expression/ Glossary/r/Regular expression.md glossary/o-r.md#regular-expression -https://tarf.co.uk/Glossary/r/Relative%20Estimation/ Glossary/r/Relative Estimation.md glossary/o-r.md#relative-estimation -https://tarf.co.uk/Glossary/r/Release%20Management/ Glossary/r/Release Management.md glossary/o-r.md#release-management -https://tarf.co.uk/Glossary/r/Release%20Test/ Glossary/r/Release Test.md glossary/o-r.md#release-test -https://tarf.co.uk/Glossary/r/Release/ Glossary/r/Release.md glossary/o-r.md#release -https://tarf.co.uk/Glossary/r/Requirement/ Glossary/r/Requirement.md glossary/o-r.md#requirement -https://tarf.co.uk/Glossary/r/Requirements%20Lead/ Glossary/r/Requirements Lead.md glossary/o-r.md#requirements-lead -https://tarf.co.uk/Glossary/r/Requirements/ Glossary/r/Requirements.md glossary/o-r.md#requirements -https://tarf.co.uk/Glossary/r/REST/ Glossary/r/REST.md glossary/o-r.md#rest -https://tarf.co.uk/Glossary/r/Review/ Glossary/r/Review.md glossary/o-r.md#review -https://tarf.co.uk/Glossary/r/Reviewer/ Glossary/r/Reviewer.md glossary/o-r.md#reviewer -https://tarf.co.uk/Glossary/r/Revision%20Control/ Glossary/r/Revision Control.md glossary/o-r.md#revision-control -https://tarf.co.uk/Glossary/r/Revision/ Glossary/r/Revision.md glossary/o-r.md#revision -https://tarf.co.uk/Glossary/r/Rework/ Glossary/r/Rework.md glossary/o-r.md#rework -https://tarf.co.uk/Glossary/r/RFC/ Glossary/r/RFC.md glossary/o-r.md#rfc -https://tarf.co.uk/Glossary/r/RIP/ Glossary/r/RIP.md glossary/o-r.md#rip -https://tarf.co.uk/Glossary/r/Risk%20Analysis/ Glossary/r/Risk Analysis.md glossary/o-r.md#risk-analysis -https://tarf.co.uk/Glossary/r/Risk%20Identification/ Glossary/r/Risk Identification.md glossary/o-r.md#risk-identification -https://tarf.co.uk/Glossary/r/Risk%20List/ Glossary/r/Risk List.md glossary/o-r.md#risk-list -https://tarf.co.uk/Glossary/r/Risk%20Management/ Glossary/r/Risk Management.md glossary/o-r.md#risk-management -https://tarf.co.uk/Glossary/r/Risk%20Mitigation/ Glossary/r/Risk Mitigation.md glossary/o-r.md#risk-mitigation -https://tarf.co.uk/Glossary/r/Risk%20Reward%20Matric/ Glossary/r/Risk Reward Matric.md glossary/o-r.md#risk-reward-matrix renamed to "Risk Reward Matrix" -https://tarf.co.uk/Glossary/r/Risk/ Glossary/r/Risk.md glossary/o-r.md#risk -https://tarf.co.uk/Glossary/r/RNC/ Glossary/r/RNC.md glossary/o-r.md#rnc -https://tarf.co.uk/Glossary/r/RNSAP/ Glossary/r/RNSAP.md glossary/o-r.md#rnsap -https://tarf.co.uk/Glossary/r/Role-feature-reason/ Glossary/r/Role-feature-reason.md glossary/o-r.md#role-feature-reason -https://tarf.co.uk/Glossary/r/Rolling%20Wave%20Planning/ Glossary/r/Rolling Wave Planning.md glossary/o-r.md#rolling-wave-planning -https://tarf.co.uk/Glossary/r/Round%20Robin/ Glossary/r/Round Robin.md glossary/o-r.md#round-robin -https://tarf.co.uk/Glossary/r/Router/ Glossary/r/Router.md glossary/o-r.md#router -https://tarf.co.uk/Glossary/r/RPC/ Glossary/r/RPC.md glossary/o-r.md#rpc -https://tarf.co.uk/Glossary/r/RRC/ Glossary/r/RRC.md glossary/o-r.md#rrc -https://tarf.co.uk/Glossary/r/RSTP/ Glossary/r/RSTP.md glossary/o-r.md#rstp -https://tarf.co.uk/Glossary/r/RSVP/ Glossary/r/RSVP.md glossary/o-r.md#rsvp -https://tarf.co.uk/Glossary/r/RTCP/ Glossary/r/RTCP.md glossary/o-r.md#rtcp -https://tarf.co.uk/Glossary/r/RTOS/ Glossary/r/RTOS.md glossary/o-r.md#rtos -https://tarf.co.uk/Glossary/r/RTP/ Glossary/r/RTP.md glossary/o-r.md#rtp -https://tarf.co.uk/Glossary/r/RTSP/ Glossary/r/RTSP.md glossary/o-r.md#rtsp -https://tarf.co.uk/Glossary/r/Ruby%20on%20rails/ Glossary/r/Ruby on rails.md glossary/o-r.md#ruby-on-rails -https://tarf.co.uk/Glossary/r/Ruby/ Glossary/r/Ruby.md glossary/o-r.md#ruby -https://tarf.co.uk/Glossary/r/Rule%20of%20Simplicity/ Glossary/r/Rule of Simplicity.md glossary/o-r.md#rule-of-simplicity -https://tarf.co.uk/Glossary/r/RUP/ Glossary/r/RUP.md glossary/o-r.md#rup -https://tarf.co.uk/Glossary/s/S-%20Tag/ Glossary/s/S- Tag.md glossary/s-z.md#s-tag renamed to "S-Tag" -https://tarf.co.uk/Glossary/s/S-GW/ Glossary/s/S-GW.md glossary/s-z.md#s-gw -https://tarf.co.uk/Glossary/s/s/ Glossary/s/s.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/s/SAAL/ Glossary/s/SAAL.md glossary/s-z.md#saal -https://tarf.co.uk/Glossary/s/SaaS/ Glossary/s/SaaS.md glossary/s-z.md#saas -https://tarf.co.uk/Glossary/s/SAE/ Glossary/s/SAE.md glossary/s-z.md#sae -https://tarf.co.uk/Glossary/s/SAFe/ Glossary/s/SAFe.md glossary/s-z.md#safe -https://tarf.co.uk/Glossary/s/SBC/ Glossary/s/SBC.md glossary/s-z.md#sbc -https://tarf.co.uk/Glossary/s/SC-FDMA/ Glossary/s/SC-FDMA.md glossary/s-z.md#sc-fdma -https://tarf.co.uk/Glossary/s/Scala/ Glossary/s/Scala.md glossary/s-z.md#scala -https://tarf.co.uk/Glossary/s/Scaled%20Agile/ Glossary/s/Scaled Agile.md glossary/s-z.md#scaled-agile -https://tarf.co.uk/Glossary/s/SCCP/ Glossary/s/SCCP.md glossary/s-z.md#sccp -https://tarf.co.uk/Glossary/s/Scenario/ Glossary/s/Scenario.md glossary/s-z.md#scenario -https://tarf.co.uk/Glossary/s/Schedule/ Glossary/s/Schedule.md glossary/s-z.md#schedule -https://tarf.co.uk/Glossary/s/Scheduler/ Glossary/s/Scheduler.md glossary/s-z.md#scheduler -https://tarf.co.uk/Glossary/s/SCM/ Glossary/s/SCM.md glossary/s-z.md#scm -https://tarf.co.uk/Glossary/s/Scratch/ Glossary/s/Scratch.md glossary/s-z.md#scratch -https://tarf.co.uk/Glossary/s/Scribe/ Glossary/s/Scribe.md glossary/s-z.md#scribe -https://tarf.co.uk/Glossary/s/Scrum%20Master/ Glossary/s/Scrum Master.md glossary/s-z.md#scrum-master -https://tarf.co.uk/Glossary/s/Scrum%20of%20Scrums/ Glossary/s/Scrum of Scrums.md glossary/s-z.md#scrum-of-scrums -https://tarf.co.uk/Glossary/s/Scrum/ Glossary/s/Scrum.md glossary/s-z.md#scrum -https://tarf.co.uk/Glossary/s/SCTP/ Glossary/s/SCTP.md glossary/s-z.md#sctp -https://tarf.co.uk/Glossary/s/SDES/ Glossary/s/SDES.md glossary/s-z.md#sdes -https://tarf.co.uk/Glossary/s/SDH/ Glossary/s/SDH.md glossary/s-z.md#sdh -https://tarf.co.uk/Glossary/s/SDK/ Glossary/s/SDK.md glossary/s-z.md#sdk -https://tarf.co.uk/Glossary/s/SDLC/ Glossary/s/SDLC.md glossary/s-z.md#sdlc -https://tarf.co.uk/Glossary/s/SDMA/ Glossary/s/SDMA.md glossary/s-z.md#sdma -https://tarf.co.uk/Glossary/s/SDN/ Glossary/s/SDN.md glossary/s-z.md#sdn -https://tarf.co.uk/Glossary/s/SDP/ Glossary/s/SDP.md glossary/s-z.md#sdp -https://tarf.co.uk/Glossary/s/SDSL/ Glossary/s/SDSL.md glossary/s-z.md#sdsl -https://tarf.co.uk/Glossary/s/Sed/ Glossary/s/Sed.md glossary/s-z.md#sed -https://tarf.co.uk/Glossary/s/Semaphore/ Glossary/s/Semaphore.md glossary/s-z.md#semaphore -https://tarf.co.uk/Glossary/s/Sematic%20Versioning/ Glossary/s/Sematic Versioning.md glossary/s-z.md#semantic-versioning renamed to "Semantic Versioning" -https://tarf.co.uk/Glossary/s/Sequence%20Model/ Glossary/s/Sequence Model.md glossary/s-z.md#sequence-model -https://tarf.co.uk/Glossary/s/SF/ Glossary/s/SF.md glossary/s-z.md#sf -https://tarf.co.uk/Glossary/s/SFTP/ Glossary/s/SFTP.md glossary/s-z.md#sftp -https://tarf.co.uk/Glossary/s/SGML/ Glossary/s/SGML.md glossary/s-z.md#sgml -https://tarf.co.uk/Glossary/s/SGSN/ Glossary/s/SGSN.md glossary/s-z.md#sgsn -https://tarf.co.uk/Glossary/s/SGW/ Glossary/s/SGW.md glossary/s-z.md#sgw -https://tarf.co.uk/Glossary/s/Sh/ Glossary/s/Sh.md glossary/s-z.md#sh -https://tarf.co.uk/Glossary/s/SHA/ Glossary/s/SHA.md glossary/s-z.md#sha -https://tarf.co.uk/Glossary/s/Shell%20Scripting/ Glossary/s/Shell Scripting.md glossary/s-z.md#shell-scripting -https://tarf.co.uk/Glossary/s/Sidetone/ Glossary/s/Sidetone.md glossary/s-z.md#sidetone -https://tarf.co.uk/Glossary/s/Sign%20Up/ Glossary/s/Sign Up.md glossary/s-z.md#sign-up -https://tarf.co.uk/Glossary/s/SIM/ Glossary/s/SIM.md glossary/s-z.md#sim -https://tarf.co.uk/Glossary/s/Simple%20Design/ Glossary/s/Simple Design.md glossary/s-z.md#simple-design -https://tarf.co.uk/Glossary/s/SIMPLE/ Glossary/s/SIMPLE.md glossary/s-z.md#simple -https://tarf.co.uk/Glossary/s/Simplex/ Glossary/s/Simplex.md glossary/s-z.md#simplex -https://tarf.co.uk/Glossary/s/Single%20Step%20Test/ Glossary/s/Single Step Test.md glossary/s-z.md#single-step-test -https://tarf.co.uk/Glossary/s/SIP%20Trunk/ Glossary/s/SIP Trunk.md glossary/s-z.md#sip-trunk -https://tarf.co.uk/Glossary/s/SIP/ Glossary/s/SIP.md glossary/s-z.md#sip -https://tarf.co.uk/Glossary/s/SISO/ Glossary/s/SISO.md glossary/s-z.md#siso -https://tarf.co.uk/Glossary/s/Skew/ Glossary/s/Skew.md glossary/s-z.md#skew -https://tarf.co.uk/Glossary/s/SLA/ Glossary/s/SLA.md glossary/s-z.md#sla -https://tarf.co.uk/Glossary/s/SLIP/ Glossary/s/SLIP.md glossary/s-z.md#slip -https://tarf.co.uk/Glossary/s/SME/ Glossary/s/SME.md glossary/s-z.md#sme -https://tarf.co.uk/Glossary/s/Smoke%20Test/ Glossary/s/Smoke Test.md glossary/s-z.md#smoke-test -https://tarf.co.uk/Glossary/s/smoke%20testing/ Glossary/s/smoke testing.md glossary/s-z.md#smoke-testing -https://tarf.co.uk/Glossary/s/SMP/ Glossary/s/SMP.md glossary/s-z.md#smp -https://tarf.co.uk/Glossary/s/SMPF/ Glossary/s/SMPF.md glossary/s-z.md#smpf -https://tarf.co.uk/Glossary/s/SMSC/ Glossary/s/SMSC.md glossary/s-z.md#smsc -https://tarf.co.uk/Glossary/s/Snake%20Case/ Glossary/s/Snake Case.md glossary/s-z.md#snake-case -https://tarf.co.uk/Glossary/s/SNMP/ Glossary/s/SNMP.md glossary/s-z.md#snmp -https://tarf.co.uk/Glossary/s/SNTP/ Glossary/s/SNTP.md glossary/s-z.md#sntp -https://tarf.co.uk/Glossary/s/SOA/ Glossary/s/SOA.md glossary/s-z.md#soa -https://tarf.co.uk/Glossary/s/SOAP/ Glossary/s/SOAP.md glossary/s-z.md#soap -https://tarf.co.uk/Glossary/s/Softswitch/ Glossary/s/Softswitch.md glossary/s-z.md#softswitch -https://tarf.co.uk/Glossary/s/Software%20Change%20Request/ Glossary/s/Software Change Request.md glossary/s-z.md#software-change-request -https://tarf.co.uk/Glossary/s/Software%20Configuration%20Management/ Glossary/s/Software Configuration Management.md glossary/s-z.md#software-configuration-management -https://tarf.co.uk/Glossary/s/Software%20Development%20Plan/ Glossary/s/Software Development Plan.md glossary/s-z.md#software-development-plan -https://tarf.co.uk/Glossary/s/Software%20Engineering%20Body%20of%20Knowledge/ Glossary/s/Software Engineering Body of Knowledge.md glossary/s-z.md#software-engineering-body-of-knowledge -https://tarf.co.uk/Glossary/s/Software%20Engineering%20Lab/ Glossary/s/Software Engineering Lab.md glossary/s-z.md#software-engineering-lab -https://tarf.co.uk/Glossary/s/Software%20Engineering%20Process%20Group/ Glossary/s/Software Engineering Process Group.md glossary/s-z.md#software-engineering-process-group -https://tarf.co.uk/Glossary/s/Software%20Requirements%20Specification/ Glossary/s/Software Requirements Specification.md glossary/s-z.md#software-requirements-specification -https://tarf.co.uk/Glossary/s/SoGEA/ Glossary/s/SoGEA.md glossary/s-z.md#sogea -https://tarf.co.uk/Glossary/s/SONET/ Glossary/s/SONET.md glossary/s-z.md#sonet -https://tarf.co.uk/Glossary/s/Source%20Code/ Glossary/s/Source Code.md glossary/s-z.md#source-code -https://tarf.co.uk/Glossary/s/SPEEX/ Glossary/s/SPEEX.md glossary/s-z.md#speex -https://tarf.co.uk/Glossary/s/SPI/ Glossary/s/SPI.md glossary/s-z.md#spi -https://tarf.co.uk/Glossary/s/Spiral%20Lifecycle/ Glossary/s/Spiral Lifecycle.md glossary/s-z.md#spiral-lifecycle -https://tarf.co.uk/Glossary/s/Sponsor/ Glossary/s/Sponsor.md glossary/s-z.md#sponsor -https://tarf.co.uk/Glossary/s/Sprial%20Model/ Glossary/s/Sprial Model.md glossary/s-z.md#spiral-model renamed to "Spiral Model" -https://tarf.co.uk/Glossary/s/Sprint%20Planning/ Glossary/s/Sprint Planning.md glossary/s-z.md#sprint-planning -https://tarf.co.uk/Glossary/s/Sprint%20retrospective/ Glossary/s/Sprint retrospective.md glossary/s-z.md#sprint-retrospective -https://tarf.co.uk/Glossary/s/Sprints/ Glossary/s/Sprints.md glossary/s-z.md#sprints -https://tarf.co.uk/Glossary/s/SQL/ Glossary/s/SQL.md glossary/s-z.md#sql -https://tarf.co.uk/Glossary/s/SS7/ Glossary/s/SS7.md glossary/s-z.md#ss7 -https://tarf.co.uk/Glossary/s/SSADM/ Glossary/s/SSADM.md glossary/s-z.md#ssadm -https://tarf.co.uk/Glossary/s/SSH/ Glossary/s/SSH.md glossary/s-z.md#ssh -https://tarf.co.uk/Glossary/s/SSID/ Glossary/s/SSID.md glossary/s-z.md#ssid -https://tarf.co.uk/Glossary/s/SSIS/ Glossary/s/SSIS.md glossary/s-z.md#ssis -https://tarf.co.uk/Glossary/s/SSL/ Glossary/s/SSL.md glossary/s-z.md#ssl -https://tarf.co.uk/Glossary/s/Stack/ Glossary/s/Stack.md glossary/s-z.md#stack -https://tarf.co.uk/Glossary/s/Staff%20Days/ Glossary/s/Staff Days.md glossary/s-z.md#staff-days -https://tarf.co.uk/Glossary/s/Stage/ Glossary/s/Stage.md glossary/s-z.md#stage -https://tarf.co.uk/Glossary/s/Staged%20Delivery%20Lifecycle/ Glossary/s/Staged Delivery Lifecycle.md glossary/s-z.md#staged-delivery-lifecycle -https://tarf.co.uk/Glossary/s/Stakeholder/ Glossary/s/Stakeholder.md glossary/s-z.md#stakeholder -https://tarf.co.uk/Glossary/s/Standard/ Glossary/s/Standard.md glossary/s-z.md#standard -https://tarf.co.uk/Glossary/s/Statistical%20Estimation/ Glossary/s/Statistical Estimation.md glossary/s-z.md#statistical-estimation -https://tarf.co.uk/Glossary/s/STB/ Glossary/s/STB.md glossary/s-z.md#stb -https://tarf.co.uk/Glossary/s/Story%20Mapping/ Glossary/s/Story Mapping.md glossary/s-z.md#story-mapping -https://tarf.co.uk/Glossary/s/Story%20Splitting/ Glossary/s/Story Splitting.md glossary/s-z.md#story-splitting -https://tarf.co.uk/Glossary/s/Storyboard/ Glossary/s/Storyboard.md glossary/s-z.md#storyboard -https://tarf.co.uk/Glossary/s/STP/ Glossary/s/STP.md glossary/s-z.md#stp -https://tarf.co.uk/Glossary/s/Strategy/ Glossary/s/Strategy.md glossary/s-z.md#strategy -https://tarf.co.uk/Glossary/s/STUN/ Glossary/s/STUN.md glossary/s-z.md#stun -https://tarf.co.uk/Glossary/s/Subversion/ Glossary/s/Subversion.md glossary/s-z.md#subversion -https://tarf.co.uk/Glossary/s/Sustainable%20Pace/ Glossary/s/Sustainable Pace.md glossary/s-z.md#sustainable-pace -https://tarf.co.uk/Glossary/s/SVC/ Glossary/s/SVC.md glossary/s-z.md#svc -https://tarf.co.uk/Glossary/s/Swift/ Glossary/s/Swift.md glossary/s-z.md#swift -https://tarf.co.uk/Glossary/s/Switch/ Glossary/s/Switch.md glossary/s-z.md#switch -https://tarf.co.uk/Glossary/s/System%20Test%20Environment/ Glossary/s/System Test Environment.md glossary/s-z.md#system-test-environment -https://tarf.co.uk/Glossary/s/System%20Test/ Glossary/s/System Test.md glossary/s-z.md#system-test -https://tarf.co.uk/Glossary/s/System/ Glossary/s/System.md glossary/s-z.md#system -https://tarf.co.uk/Glossary/s/SystemX/ Glossary/s/SystemX.md glossary/s-z.md#systemx -https://tarf.co.uk/Glossary/t/T-Shirt%20Sizing/ Glossary/t/T-Shirt Sizing.md glossary/s-z.md#t-shirt-sizing -https://tarf.co.uk/Glossary/t/t/ Glossary/t/t.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/t/Task%20Board/ Glossary/t/Task Board.md glossary/s-z.md#task-board -https://tarf.co.uk/Glossary/t/Task%20Estimate/ Glossary/t/Task Estimate.md glossary/s-z.md#task-estimate -https://tarf.co.uk/Glossary/t/Task/ Glossary/t/Task.md glossary/s-z.md#task -https://tarf.co.uk/Glossary/t/TCAP/ Glossary/t/TCAP.md glossary/s-z.md#tcap -https://tarf.co.uk/Glossary/t/Tcl-tk/ Glossary/t/Tcl-tk.md glossary/s-z.md#tcltk renamed to "Tcl/tk" -https://tarf.co.uk/Glossary/t/TCP/ Glossary/t/TCP.md glossary/s-z.md#tcp -https://tarf.co.uk/Glossary/t/Tcsh/ Glossary/t/Tcsh.md glossary/s-z.md#tcsh -https://tarf.co.uk/Glossary/t/TDD/ Glossary/t/TDD.md glossary/s-z.md#tdd -https://tarf.co.uk/Glossary/t/TDM/ Glossary/t/TDM.md glossary/s-z.md#tdm -https://tarf.co.uk/Glossary/t/Team%20Room/ Glossary/t/Team Room.md glossary/s-z.md#team-room -https://tarf.co.uk/Glossary/t/Team/ Glossary/t/Team.md glossary/s-z.md#team -https://tarf.co.uk/Glossary/t/Technical%20Debt/ Glossary/t/Technical Debt.md glossary/s-z.md#technical-debt -https://tarf.co.uk/Glossary/t/Template/ Glossary/t/Template.md glossary/s-z.md#template -https://tarf.co.uk/Glossary/t/Test%20Activity/ Glossary/t/Test Activity.md glossary/s-z.md#test-activity -https://tarf.co.uk/Glossary/t/Test%20Case%20Specification/ Glossary/t/Test Case Specification.md glossary/s-z.md#test-case-specification -https://tarf.co.uk/Glossary/t/Test%20Case/ Glossary/t/Test Case.md glossary/s-z.md#test-case -https://tarf.co.uk/Glossary/t/Test%20Design%20Specification/ Glossary/t/Test Design Specification.md glossary/s-z.md#test-design-specification -https://tarf.co.uk/Glossary/t/Test%20Design/ Glossary/t/Test Design.md glossary/s-z.md#test-design -https://tarf.co.uk/Glossary/t/Test%20Level/ Glossary/t/Test Level.md glossary/s-z.md#test-level -https://tarf.co.uk/Glossary/t/Test%20Plan/ Glossary/t/Test Plan.md glossary/s-z.md#test-plan -https://tarf.co.uk/Glossary/t/Test%20Technique/ Glossary/t/Test Technique.md glossary/s-z.md#test-technique -https://tarf.co.uk/Glossary/t/Test%20Type/ Glossary/t/Test Type.md glossary/s-z.md#test-type -https://tarf.co.uk/Glossary/t/Test/ Glossary/t/Test.md glossary/s-z.md#test -https://tarf.co.uk/Glossary/t/Testing/ Glossary/t/Testing.md glossary/s-z.md#testing -https://tarf.co.uk/Glossary/t/Three%20Amigos/ Glossary/t/Three Amigos.md glossary/s-z.md#three-amigos -https://tarf.co.uk/Glossary/t/Three%20C%27s/ Glossary/t/Three C's.md glossary/s-z.md#three-cs -https://tarf.co.uk/Glossary/t/Three%20Questions/ Glossary/t/Three Questions.md glossary/s-z.md#three-questions -https://tarf.co.uk/Glossary/t/TIA/ Glossary/t/TIA.md glossary/s-z.md#tia -https://tarf.co.uk/Glossary/t/Timebox/ Glossary/t/Timebox.md glossary/s-z.md#timebox -https://tarf.co.uk/Glossary/t/TKIP/ Glossary/t/TKIP.md glossary/s-z.md#tkip -https://tarf.co.uk/Glossary/t/TLS/ Glossary/t/TLS.md glossary/s-z.md#tls -https://tarf.co.uk/Glossary/t/TLV/ Glossary/t/TLV.md glossary/s-z.md#tlv -https://tarf.co.uk/Glossary/t/Token%20Ring/ Glossary/t/Token Ring.md glossary/s-z.md#token-ring -https://tarf.co.uk/Glossary/t/TOML/ Glossary/t/TOML.md glossary/s-z.md#toml -https://tarf.co.uk/Glossary/t/Top%20Level%20Schedule/ Glossary/t/Top Level Schedule.md glossary/s-z.md#top-level-schedule -https://tarf.co.uk/Glossary/t/Top-Down%20Estimation/ Glossary/t/Top-Down Estimation.md glossary/s-z.md#top-down-estimation -https://tarf.co.uk/Glossary/t/Triple%20DES/ Glossary/t/Triple DES.md glossary/s-z.md#triple-des -https://tarf.co.uk/Glossary/t/TTY/ Glossary/t/TTY.md glossary/s-z.md#tty -https://tarf.co.uk/Glossary/t/TUP/ Glossary/t/TUP.md glossary/s-z.md#tup -https://tarf.co.uk/Glossary/t/Twisted%20pair/ Glossary/t/Twisted pair.md glossary/s-z.md#twisted-pair -https://tarf.co.uk/Glossary/u/u/ Glossary/u/u.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/u/UA/ Glossary/u/UA.md glossary/s-z.md#ua -https://tarf.co.uk/Glossary/u/Ubiquitous%20Language/ Glossary/u/Ubiquitous Language.md glossary/s-z.md#ubiquitous-language -https://tarf.co.uk/Glossary/u/UDP/ Glossary/u/UDP.md glossary/s-z.md#udp -https://tarf.co.uk/Glossary/u/UER/ Glossary/u/UER.md glossary/s-z.md#uer -https://tarf.co.uk/Glossary/u/UML/ Glossary/u/UML.md glossary/s-z.md#uml -https://tarf.co.uk/Glossary/u/UMTS/ Glossary/u/UMTS.md glossary/s-z.md#umts -https://tarf.co.uk/Glossary/u/UNI/ Glossary/u/UNI.md glossary/s-z.md#uni -https://tarf.co.uk/Glossary/u/unicast/ Glossary/u/unicast.md glossary/s-z.md#unicast -https://tarf.co.uk/Glossary/u/Unicode/ Glossary/u/Unicode.md glossary/s-z.md#unicode -https://tarf.co.uk/Glossary/u/Unified%20Communications/ Glossary/u/Unified Communications.md glossary/s-z.md#unified-communications -https://tarf.co.uk/Glossary/u/Unified%20Modeling%20Language/ Glossary/u/Unified Modeling Language.md glossary/s-z.md#uml renamed to "UML" -https://tarf.co.uk/Glossary/u/Unit%20Test/ Glossary/u/Unit Test.md glossary/s-z.md#unit-test -https://tarf.co.uk/Glossary/u/Unit%20Testing/ Glossary/u/Unit Testing.md glossary/s-z.md#unit-testing -https://tarf.co.uk/Glossary/u/Unix/ Glossary/u/Unix.md glossary/s-z.md#unix -https://tarf.co.uk/Glossary/u/Update/ Glossary/u/Update.md glossary/s-z.md#update -https://tarf.co.uk/Glossary/u/Upstream/ Glossary/u/Upstream.md glossary/s-z.md#upstream -https://tarf.co.uk/Glossary/u/URI/ Glossary/u/URI.md glossary/s-z.md#uri -https://tarf.co.uk/Glossary/u/URL/ Glossary/u/URL.md glossary/s-z.md#url -https://tarf.co.uk/Glossary/u/URN/ Glossary/u/URN.md glossary/s-z.md#urn -https://tarf.co.uk/Glossary/u/Usability%20Testing/ Glossary/u/Usability Testing.md glossary/s-z.md#usability-testing -https://tarf.co.uk/Glossary/u/Use%20Case%20Model/ Glossary/u/Use Case Model.md glossary/s-z.md#use-case-model -https://tarf.co.uk/Glossary/u/Use%20Case%20Specification/ Glossary/u/Use Case Specification.md glossary/s-z.md#use-case-specification -https://tarf.co.uk/Glossary/u/Use%20Case/ Glossary/u/Use Case.md glossary/s-z.md#use-case -https://tarf.co.uk/Glossary/u/User%20Experience/ Glossary/u/User Experience.md glossary/s-z.md#user-experience -https://tarf.co.uk/Glossary/u/User%20Interface%20Design/ Glossary/u/User Interface Design.md glossary/s-z.md#user-interface-design -https://tarf.co.uk/Glossary/u/User%20Interface%20Prototype/ Glossary/u/User Interface Prototype.md glossary/s-z.md#user-interface-prototype -https://tarf.co.uk/Glossary/u/User%20Interface/ Glossary/u/User Interface.md glossary/s-z.md#user-interface -https://tarf.co.uk/Glossary/u/User%20Plane/ Glossary/u/User Plane.md glossary/s-z.md#user-plane -https://tarf.co.uk/Glossary/u/User%20Stories/ Glossary/u/User Stories.md glossary/s-z.md#user-stories -https://tarf.co.uk/Glossary/u/UTC/ Glossary/u/UTC.md glossary/s-z.md#utc -https://tarf.co.uk/Glossary/u/UTF-16/ Glossary/u/UTF-16.md glossary/s-z.md#utf-16 -https://tarf.co.uk/Glossary/u/UTF-8/ Glossary/u/UTF-8.md glossary/s-z.md#utf-8 -https://tarf.co.uk/Glossary/u/UTRAN/ Glossary/u/UTRAN.md glossary/s-z.md#utran -https://tarf.co.uk/Glossary/v/V%20Model/ Glossary/v/V Model.md glossary/s-z.md#v-model -https://tarf.co.uk/Glossary/v/V.90/ Glossary/v/V.90.md glossary/s-z.md#v90 -https://tarf.co.uk/Glossary/v/V.92/ Glossary/v/V.92.md glossary/s-z.md#v92 -https://tarf.co.uk/Glossary/v/v/ Glossary/v/v.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/v/VAD/ Glossary/v/VAD.md glossary/s-z.md#vad -https://tarf.co.uk/Glossary/v/VB/ Glossary/v/VB.md glossary/s-z.md#vb -https://tarf.co.uk/Glossary/v/VBS/ Glossary/v/VBS.md glossary/s-z.md#vbs -https://tarf.co.uk/Glossary/v/VC/ Glossary/v/VC.md glossary/s-z.md#vc -https://tarf.co.uk/Glossary/v/VCI/ Glossary/v/VCI.md glossary/s-z.md#vci -https://tarf.co.uk/Glossary/v/VCID/ Glossary/v/VCID.md glossary/s-z.md#vcid -https://tarf.co.uk/Glossary/v/VDSL/ Glossary/v/VDSL.md glossary/s-z.md#vdsl -https://tarf.co.uk/Glossary/v/Velocity/ Glossary/v/Velocity.md glossary/s-z.md#velocity -https://tarf.co.uk/Glossary/v/Verifier/ Glossary/v/Verifier.md glossary/s-z.md#verifier -https://tarf.co.uk/Glossary/v/Version%20Control/ Glossary/v/Version Control.md glossary/s-z.md#version-control -https://tarf.co.uk/Glossary/v/Version/ Glossary/v/Version.md glossary/s-z.md#version -https://tarf.co.uk/Glossary/v/VHDL/ Glossary/v/VHDL.md glossary/s-z.md#vhdl -https://tarf.co.uk/Glossary/v/VISP/ Glossary/v/VISP.md glossary/s-z.md#visp -https://tarf.co.uk/Glossary/v/VLAN/ Glossary/v/VLAN.md glossary/s-z.md#vlan -https://tarf.co.uk/Glossary/v/VLR/ Glossary/v/VLR.md glossary/s-z.md#vlr -https://tarf.co.uk/Glossary/v/VM/ Glossary/v/VM.md glossary/s-z.md#vm -https://tarf.co.uk/Glossary/v/VOD/ Glossary/v/VOD.md glossary/s-z.md#vod -https://tarf.co.uk/Glossary/v/VoIP/ Glossary/v/VoIP.md glossary/s-z.md#voip -https://tarf.co.uk/Glossary/v/VoLTE/ Glossary/v/VoLTE.md glossary/s-z.md#volte -https://tarf.co.uk/Glossary/v/Vorbis/ Glossary/v/Vorbis.md glossary/s-z.md#vorbis -https://tarf.co.uk/Glossary/v/VPI/ Glossary/v/VPI.md glossary/s-z.md#vpi -https://tarf.co.uk/Glossary/v/VPLS/ Glossary/v/VPLS.md glossary/s-z.md#vpls -https://tarf.co.uk/Glossary/v/VPN/ Glossary/v/VPN.md glossary/s-z.md#vpn -https://tarf.co.uk/Glossary/v/VPWS/ Glossary/v/VPWS.md glossary/s-z.md#vpws -https://tarf.co.uk/Glossary/v/VRF/ Glossary/v/VRF.md glossary/s-z.md#vrf -https://tarf.co.uk/Glossary/w/W-CDMA/ Glossary/w/W-CDMA.md glossary/s-z.md#w-cdma -https://tarf.co.uk/Glossary/w/w/ Glossary/w/w.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/w/W3C/ Glossary/w/W3C.md glossary/s-z.md#w3c -https://tarf.co.uk/Glossary/w/Walkthrough/ Glossary/w/Walkthrough.md glossary/s-z.md#walkthrough -https://tarf.co.uk/Glossary/w/WAN/ Glossary/w/WAN.md glossary/s-z.md#wan -https://tarf.co.uk/Glossary/w/WANET/ Glossary/w/WANET.md glossary/s-z.md#wanet -https://tarf.co.uk/Glossary/w/WAP/ Glossary/w/WAP.md glossary/s-z.md#wap -https://tarf.co.uk/Glossary/w/Waterfall%20Lifecycle/ Glossary/w/Waterfall Lifecycle.md glossary/s-z.md#waterfall-lifecycle -https://tarf.co.uk/Glossary/w/Waterfall/ Glossary/w/Waterfall.md glossary/s-z.md#waterfall -https://tarf.co.uk/Glossary/w/WBC/ Glossary/w/WBC.md glossary/s-z.md#wbc -https://tarf.co.uk/Glossary/w/WBCC/ Glossary/w/WBCC.md glossary/s-z.md#wbcc -https://tarf.co.uk/Glossary/w/WBMC/ Glossary/w/WBMC.md glossary/s-z.md#wbmc -https://tarf.co.uk/Glossary/w/Weekly%20Individual%20Plan/ Glossary/w/Weekly Individual Plan.md glossary/s-z.md#weekly-individual-plan -https://tarf.co.uk/Glossary/w/WEP/ Glossary/w/WEP.md glossary/s-z.md#wep -https://tarf.co.uk/Glossary/w/What%20Requirement/ Glossary/w/What Requirement.md glossary/s-z.md#what-requirement -https://tarf.co.uk/Glossary/w/Why%20Requirement/ Glossary/w/Why Requirement.md glossary/s-z.md#why-requirement -https://tarf.co.uk/Glossary/w/Wi-Fi/ Glossary/w/Wi-Fi.md glossary/s-z.md#wi-fi -https://tarf.co.uk/Glossary/w/Wiki/ Glossary/w/Wiki.md glossary/s-z.md#wiki -https://tarf.co.uk/Glossary/w/WiMAX/ Glossary/w/WiMAX.md glossary/s-z.md#wimax -https://tarf.co.uk/Glossary/w/WIN-T/ Glossary/w/WIN-T.md glossary/s-z.md#win-t -https://tarf.co.uk/Glossary/w/Wireless%20Mesh%20Network/ Glossary/w/Wireless Mesh Network.md glossary/s-z.md#wireless-mesh-network -https://tarf.co.uk/Glossary/w/Wireless%20Repeater/ Glossary/w/Wireless Repeater.md glossary/s-z.md#wireless-repeater -https://tarf.co.uk/Glossary/w/Wireshark/ Glossary/w/Wireshark.md glossary/s-z.md#wireshark -https://tarf.co.uk/Glossary/w/WISP/ Glossary/w/WISP.md glossary/s-z.md#wisp -https://tarf.co.uk/Glossary/w/WLR/ Glossary/w/WLR.md glossary/s-z.md#wlr -https://tarf.co.uk/Glossary/w/WMA/ Glossary/w/WMA.md glossary/s-z.md#wma -https://tarf.co.uk/Glossary/w/Work%20Breakdown%20Structure/ Glossary/w/Work Breakdown Structure.md glossary/s-z.md#work-breakdown-structure -https://tarf.co.uk/Glossary/w/Work%20Item/ Glossary/w/Work Item.md glossary/s-z.md#work-item -https://tarf.co.uk/Glossary/w/Work%20Package/ Glossary/w/Work Package.md glossary/s-z.md#work-package -https://tarf.co.uk/Glossary/w/Work%20Plan/ Glossary/w/Work Plan.md glossary/s-z.md#work-plan -https://tarf.co.uk/Glossary/w/Work%20Product/ Glossary/w/Work Product.md glossary/s-z.md#work-product -https://tarf.co.uk/Glossary/w/WPA2/ Glossary/w/WPA2.md glossary/s-z.md#wpa2 -https://tarf.co.uk/Glossary/w/WPS/ Glossary/w/WPS.md glossary/s-z.md#wps -https://tarf.co.uk/Glossary/w/write-only%20code/ Glossary/w/write-only code.md glossary/s-z.md#write-only-code -https://tarf.co.uk/Glossary/w/WS-BPEL/ Glossary/w/WS-BPEL.md glossary/s-z.md#ws-bpel -https://tarf.co.uk/Glossary/w/WSDL/ Glossary/w/WSDL.md glossary/s-z.md#wsdl -https://tarf.co.uk/Glossary/x/X.25/ Glossary/x/X.25.md glossary/s-z.md#x25 -https://tarf.co.uk/Glossary/x/X.509/ Glossary/x/X.509.md glossary/s-z.md#x509 -https://tarf.co.uk/Glossary/x/x/ Glossary/x/x.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/x/XML%20Schema/ Glossary/x/XML Schema.md glossary/s-z.md#xml-schema -https://tarf.co.uk/Glossary/x/XML-RPC/ Glossary/x/XML-RPC.md glossary/s-z.md#xml-rpc -https://tarf.co.uk/Glossary/x/XML/ Glossary/x/XML.md glossary/s-z.md#xml -https://tarf.co.uk/Glossary/x/XMPP/ Glossary/x/XMPP.md glossary/s-z.md#xmpp -https://tarf.co.uk/Glossary/x/XP/ Glossary/x/XP.md glossary/s-z.md#xp -https://tarf.co.uk/Glossary/x/xsd/ Glossary/x/xsd.md glossary/s-z.md#xsd -https://tarf.co.uk/Glossary/x/XSS/ Glossary/x/XSS.md glossary/s-z.md#xss -https://tarf.co.uk/Glossary/y/y/ Glossary/y/y.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/y/YAML/ Glossary/y/YAML.md glossary/s-z.md#yaml -https://tarf.co.uk/Glossary/z/z/ Glossary/z/z.md glossary/s-z.md letter index -https://tarf.co.uk/Glossary/z/ZigBee/ Glossary/z/ZigBee.md glossary/s-z.md#zigbee -https://tarf.co.uk/Glossary/%CE%BC/%CE%BC-law/ Glossary/μ/μ-law.md glossary/l-n.md#mu-law renamed to "Mu-law" -https://tarf.co.uk/Glossary/%CE%BC/%CE%BC/ Glossary/μ/μ.md glossary/s-z.md letter index +https://tarf.co.uk/Glossary/1/1/ Glossary/1/1.md glossary/numbers.md letter index +https://tarf.co.uk/Glossary/1/100%20Mbps---Fast%20Ethernet/ Glossary/1/100 Mbps---Fast Ethernet.md glossary/numbers.md#100-mbps-fast-ethernet renamed to "100 Mbps (Fast Ethernet)" +https://tarf.co.uk/Glossary/5/5/ Glossary/5/5.md glossary/numbers.md letter index +https://tarf.co.uk/Glossary/5/5ESS/ Glossary/5/5ESS.md glossary/numbers.md#5ess +https://tarf.co.uk/Glossary/5/5G/ Glossary/5/5G.md glossary/numbers.md#5g +https://tarf.co.uk/Glossary/8/8/ Glossary/8/8.md glossary/numbers.md letter index +https://tarf.co.uk/Glossary/8/802.11/ Glossary/8/802.11.md glossary/numbers.md#80211 +https://tarf.co.uk/Glossary/8/802.3%20Physical%20Layer/ Glossary/8/802.3 Physical Layer.md glossary/numbers.md#8023-physical-layer +https://tarf.co.uk/Glossary/a/A-law/ Glossary/a/A-law.md glossary/a.md#a-law +https://tarf.co.uk/Glossary/a/a/ Glossary/a/a.md glossary/a.md letter index +https://tarf.co.uk/Glossary/a/AA/ Glossary/a/AA.md glossary/a.md#aa +https://tarf.co.uk/Glossary/a/AAC/ Glossary/a/AAC.md glossary/a.md#aac +https://tarf.co.uk/Glossary/a/AAL1/ Glossary/a/AAL1.md glossary/a.md#aal1 +https://tarf.co.uk/Glossary/a/AAL2/ Glossary/a/AAL2.md glossary/a.md#aal2 +https://tarf.co.uk/Glossary/a/AAL5/ Glossary/a/AAL5.md glossary/a.md#aal5 +https://tarf.co.uk/Glossary/a/Acceptance%20Test/ Glossary/a/Acceptance Test.md glossary/a.md#acceptance-test +https://tarf.co.uk/Glossary/a/Acceptance%20Testing/ Glossary/a/Acceptance Testing.md glossary/a.md#acceptance-testing +https://tarf.co.uk/Glossary/a/Activity%20Model/ Glossary/a/Activity Model.md glossary/a.md#activity-model +https://tarf.co.uk/Glossary/a/Activity/ Glossary/a/Activity.md glossary/a.md#activity +https://tarf.co.uk/Glossary/a/Ad%20Hoc%20Testing/ Glossary/a/Ad Hoc Testing.md glossary/a.md#ad-hoc-testing +https://tarf.co.uk/Glossary/a/AD/ Glossary/a/AD.md glossary/a.md#ad +https://tarf.co.uk/Glossary/a/Ada/ Glossary/a/Ada.md glossary/a.md#ada +https://tarf.co.uk/Glossary/a/ADSL/ Glossary/a/ADSL.md glossary/a.md#adsl +https://tarf.co.uk/Glossary/a/AES/ Glossary/a/AES.md glossary/a.md#aes +https://tarf.co.uk/Glossary/a/Agent/ Glossary/a/Agent.md glossary/a.md#agent +https://tarf.co.uk/Glossary/a/Agile%20development/ Glossary/a/Agile development.md glossary/a.md#agile-development +https://tarf.co.uk/Glossary/a/Agile%20Manifesto/ Glossary/a/Agile Manifesto.md glossary/a.md#agile-manifesto +https://tarf.co.uk/Glossary/a/Agile/ Glossary/a/Agile.md glossary/a.md#agile +https://tarf.co.uk/Glossary/a/AIS/ Glossary/a/AIS.md glossary/a.md#ais +https://tarf.co.uk/Glossary/a/ALCAP/ Glossary/a/ALCAP.md glossary/a.md#alcap +https://tarf.co.uk/Glossary/a/Amazon%20EC2/ Glossary/a/Amazon EC2.md glossary/a.md#amazon-ec2 +https://tarf.co.uk/Glossary/a/AMP/ Glossary/a/AMP.md glossary/a.md#amp +https://tarf.co.uk/Glossary/a/AMR-NB/ Glossary/a/AMR-NB.md glossary/a.md#amr-nb +https://tarf.co.uk/Glossary/a/AMR-WB/ Glossary/a/AMR-WB.md glossary/a.md#amr-wb +https://tarf.co.uk/Glossary/a/Analogy%20Estimation/ Glossary/a/Analogy Estimation.md glossary/a.md#analogy-estimation +https://tarf.co.uk/Glossary/a/Android/ Glossary/a/Android.md glossary/a.md#android +https://tarf.co.uk/Glossary/a/ANDSF/ Glossary/a/ANDSF.md glossary/a.md#andsf +https://tarf.co.uk/Glossary/a/ANSI/ Glossary/a/ANSI.md glossary/a.md#ansi +https://tarf.co.uk/Glossary/a/Antipattern/ Glossary/a/Antipattern.md glossary/a.md#antipattern +https://tarf.co.uk/Glossary/a/Anycast/ Glossary/a/Anycast.md glossary/a.md#anycast +https://tarf.co.uk/Glossary/a/AOP/ Glossary/a/AOP.md glossary/a.md#aop +https://tarf.co.uk/Glossary/a/API/ Glossary/a/API.md glossary/a.md#api +https://tarf.co.uk/Glossary/a/APN/ Glossary/a/APN.md glossary/a.md#apn +https://tarf.co.uk/Glossary/a/Architecture/ Glossary/a/Architecture.md glossary/a.md#architecture +https://tarf.co.uk/Glossary/a/ARP%20spoofing/ Glossary/a/ARP spoofing.md glossary/a.md#arp-spoofing +https://tarf.co.uk/Glossary/a/Artifact/ Glossary/a/Artifact.md glossary/a.md#artefact renamed to "Artefact" +https://tarf.co.uk/Glossary/a/ASCII/ Glossary/a/ASCII.md glossary/a.md#ascii +https://tarf.co.uk/Glossary/a/ASN.1/ Glossary/a/ASN.1.md glossary/a.md#asn1 +https://tarf.co.uk/Glossary/a/Assessment/ Glossary/a/Assessment.md glossary/a.md#assessment +https://tarf.co.uk/Glossary/a/ATDD/ Glossary/a/ATDD.md glossary/a.md#atdd +https://tarf.co.uk/Glossary/a/ATM/ Glossary/a/ATM.md glossary/a.md#atm +https://tarf.co.uk/Glossary/a/Audio%20Codecs/ Glossary/a/Audio Codecs.md glossary/a.md#audio-codecs +https://tarf.co.uk/Glossary/a/Audit/ Glossary/a/Audit.md glossary/a.md#audit +https://tarf.co.uk/Glossary/a/Author/ Glossary/a/Author.md glossary/a.md#author +https://tarf.co.uk/Glossary/a/Authority/ Glossary/a/Authority.md glossary/a.md#authority +https://tarf.co.uk/Glossary/a/authorriative%20Answer/ Glossary/a/authorriative Answer.md glossary/a.md#authoritative-answer renamed to "Authoritative Answer" +https://tarf.co.uk/Glossary/a/Automated%20Build/ Glossary/a/Automated Build.md glossary/a.md#automated-build +https://tarf.co.uk/Glossary/a/Automated%20Testing/ Glossary/a/Automated Testing.md glossary/a.md#automated-testing +https://tarf.co.uk/Glossary/a/Awk/ Glossary/a/Awk.md glossary/a.md#awk +https://tarf.co.uk/Glossary/a/AWS/ Glossary/a/AWS.md glossary/a.md#aws +https://tarf.co.uk/Glossary/a/Azure/ Glossary/a/Azure.md glossary/a.md#azure +https://tarf.co.uk/Glossary/b/b/ Glossary/b/b.md glossary/b.md letter index +https://tarf.co.uk/Glossary/b/B2B/ Glossary/b/B2B.md glossary/b.md#b2b +https://tarf.co.uk/Glossary/b/B2C/ Glossary/b/B2C.md glossary/b.md#b2c +https://tarf.co.uk/Glossary/b/Backhaul/ Glossary/b/Backhaul.md glossary/b.md#backhaul +https://tarf.co.uk/Glossary/b/Backlog%20Grooming/ Glossary/b/Backlog Grooming.md glossary/b.md#backlog-grooming +https://tarf.co.uk/Glossary/b/Backlog/ Glossary/b/Backlog.md glossary/b.md#backlog +https://tarf.co.uk/Glossary/b/Bandwidth/ Glossary/b/Bandwidth.md glossary/b.md#bandwidth +https://tarf.co.uk/Glossary/b/Baseline/ Glossary/b/Baseline.md glossary/b.md#baseline +https://tarf.co.uk/Glossary/b/Bash/ Glossary/b/Bash.md glossary/b.md#bash +https://tarf.co.uk/Glossary/b/BASIC/ Glossary/b/BASIC.md glossary/b.md#basic +https://tarf.co.uk/Glossary/b/BAU/ Glossary/b/BAU.md glossary/b.md#bau +https://tarf.co.uk/Glossary/b/BDD/ Glossary/b/BDD.md glossary/b.md#bdd +https://tarf.co.uk/Glossary/b/Bench%20Test/ Glossary/b/Bench Test.md glossary/b.md#bench-test +https://tarf.co.uk/Glossary/b/BER/ Glossary/b/BER.md glossary/b.md#ber +https://tarf.co.uk/Glossary/b/Best%20Practice%20Description/ Glossary/b/Best Practice Description.md glossary/b.md#best-practice renamed to "Best Practice" +https://tarf.co.uk/Glossary/b/Best%20Practice/ Glossary/b/Best Practice.md glossary/b.md#best-practice +https://tarf.co.uk/Glossary/b/BGCF/ Glossary/b/BGCF.md glossary/b.md#bgcf +https://tarf.co.uk/Glossary/b/BGP/ Glossary/b/BGP.md glossary/b.md#bgp +https://tarf.co.uk/Glossary/b/Bjarne%20Stroustrup/ Glossary/b/Bjarne Stroustrup.md glossary/b.md#bjarne-stroustrup +https://tarf.co.uk/Glossary/b/Black-Box%20Testing/ Glossary/b/Black-Box Testing.md glossary/b.md#black-box-testing +https://tarf.co.uk/Glossary/b/Bluetooth/ Glossary/b/Bluetooth.md glossary/b.md#bluetooth +https://tarf.co.uk/Glossary/b/BNF/ Glossary/b/BNF.md glossary/b.md#bnf +https://tarf.co.uk/Glossary/b/BNG/ Glossary/b/BNG.md glossary/b.md#bng +https://tarf.co.uk/Glossary/b/BOOTP/ Glossary/b/BOOTP.md glossary/b.md#bootp +https://tarf.co.uk/Glossary/b/Boston%20Matrix/ Glossary/b/Boston Matrix.md glossary/b.md#boston-matrix +https://tarf.co.uk/Glossary/b/Bottom-Up%20Estimation/ Glossary/b/Bottom-Up Estimation.md glossary/b.md#bottom-up-estimation +https://tarf.co.uk/Glossary/b/BPEL/ Glossary/b/BPEL.md glossary/b.md#bpel +https://tarf.co.uk/Glossary/b/BPMN/ Glossary/b/BPMN.md glossary/b.md#bpmn +https://tarf.co.uk/Glossary/b/BRAS/ Glossary/b/BRAS.md glossary/b.md#bras +https://tarf.co.uk/Glossary/b/Broadcast/ Glossary/b/Broadcast.md glossary/b.md#broadcast +https://tarf.co.uk/Glossary/b/BSC/ Glossary/b/BSC.md glossary/b.md#bsc +https://tarf.co.uk/Glossary/b/BSS/ Glossary/b/BSS.md glossary/b.md#bss +https://tarf.co.uk/Glossary/b/BT%2021CN/ Glossary/b/BT 21CN.md glossary/b.md#bt-21cn +https://tarf.co.uk/Glossary/b/BTNUP/ Glossary/b/BTNUP.md glossary/b.md#btnup +https://tarf.co.uk/Glossary/b/BTS/ Glossary/b/BTS.md glossary/b.md#bts +https://tarf.co.uk/Glossary/b/bug/ Glossary/b/bug.md glossary/b.md#bug +https://tarf.co.uk/Glossary/b/Build%20Environment/ Glossary/b/Build Environment.md glossary/b.md#build-environment +https://tarf.co.uk/Glossary/b/Build%20Machine/ Glossary/b/Build Machine.md glossary/b.md#build-machine +https://tarf.co.uk/Glossary/b/build%20tool/ Glossary/b/build tool.md glossary/b.md#build-tool +https://tarf.co.uk/Glossary/b/Build/ Glossary/b/Build.md glossary/b.md#build +https://tarf.co.uk/Glossary/b/Burndown%20Chart/ Glossary/b/Burndown Chart.md glossary/b.md#burndown-chart +https://tarf.co.uk/Glossary/b/Burst/ Glossary/b/Burst.md glossary/b.md#burst +https://tarf.co.uk/Glossary/b/Business%20Agility%20Ownership/ Glossary/b/Business Agility Ownership.md glossary/b.md#business-agility-ownership +https://tarf.co.uk/Glossary/b/Business%20Requirement/ Glossary/b/Business Requirement.md glossary/b.md#business-requirement +https://tarf.co.uk/Glossary/b/Business%20Schedule/ Glossary/b/Business Schedule.md glossary/b.md#business-schedule +https://tarf.co.uk/Glossary/b/BYOD/ Glossary/b/BYOD.md glossary/b.md#byod +https://tarf.co.uk/Glossary/c/C%20Language/ Glossary/c/C Language.md glossary/c.md#c-language +https://tarf.co.uk/Glossary/c/C%23/ Glossary/c/C#.md glossary/c.md#c_2 +https://tarf.co.uk/Glossary/c/C%2B%2B/ Glossary/c/C++.md glossary/c.md#c_1 +https://tarf.co.uk/Glossary/c/C-Tag/ Glossary/c/C-Tag.md glossary/c.md#c-tag +https://tarf.co.uk/Glossary/c/c/ Glossary/c/c.md glossary/c.md letter index +https://tarf.co.uk/Glossary/c/CA/ Glossary/c/CA.md glossary/c.md#ca +https://tarf.co.uk/Glossary/c/Cacti/ Glossary/c/Cacti.md glossary/c.md#cacti +https://tarf.co.uk/Glossary/c/Cadence/ Glossary/c/Cadence.md glossary/c.md#cadence +https://tarf.co.uk/Glossary/c/Calendar%20Days/ Glossary/c/Calendar Days.md glossary/c.md#calendar-days +https://tarf.co.uk/Glossary/c/CAM%20Item/ Glossary/c/CAM Item.md glossary/i.md#item renamed to "Item" +https://tarf.co.uk/Glossary/c/Camel%20Case/ Glossary/c/Camel Case.md glossary/c.md#camel-case +https://tarf.co.uk/Glossary/c/CAMEL/ Glossary/c/CAMEL.md glossary/c.md#camel +https://tarf.co.uk/Glossary/c/CAP/ Glossary/c/CAP.md glossary/c.md#cap +https://tarf.co.uk/Glossary/c/Capability%20Maturity%20Model/ Glossary/c/Capability Maturity Model.md glossary/c.md#capability-maturity-model +https://tarf.co.uk/Glossary/c/Cat%205/ Glossary/c/Cat 5.md glossary/c.md#cat-5 +https://tarf.co.uk/Glossary/c/Cat%205e/ Glossary/c/Cat 5e.md glossary/c.md#cat-5e +https://tarf.co.uk/Glossary/c/Cat%206/ Glossary/c/Cat 6.md glossary/c.md#cat-6 +https://tarf.co.uk/Glossary/c/CCITT/ Glossary/c/CCITT.md glossary/c.md#ccitt +https://tarf.co.uk/Glossary/c/CCS/ Glossary/c/CCS.md glossary/c.md#ccs +https://tarf.co.uk/Glossary/c/CCTA/ Glossary/c/CCTA.md glossary/c.md#ccta +https://tarf.co.uk/Glossary/c/CDMA/ Glossary/c/CDMA.md glossary/c.md#cdma +https://tarf.co.uk/Glossary/c/CDMA2000/ Glossary/c/CDMA2000.md glossary/c.md#cdma2000 +https://tarf.co.uk/Glossary/c/CDR/ Glossary/c/CDR.md glossary/c.md#cdr +https://tarf.co.uk/Glossary/c/CE/ Glossary/c/CE.md glossary/c.md#ce +https://tarf.co.uk/Glossary/c/CER/ Glossary/c/CER.md glossary/c.md#cer +https://tarf.co.uk/Glossary/c/Ceremonies/ Glossary/c/Ceremonies.md glossary/c.md#ceremonies +https://tarf.co.uk/Glossary/c/CESG/ Glossary/c/CESG.md glossary/c.md#cesg +https://tarf.co.uk/Glossary/c/CFi/ Glossary/c/CFi.md glossary/c.md#cfi +https://tarf.co.uk/Glossary/c/CGI/ Glossary/c/CGI.md glossary/c.md#cgi +https://tarf.co.uk/Glossary/c/Change%20Control%20Board/ Glossary/c/Change Control Board.md glossary/c.md#change-control-board +https://tarf.co.uk/Glossary/c/Change%20Control%20Plan/ Glossary/c/Change Control Plan.md glossary/c.md#change-control-plan +https://tarf.co.uk/Glossary/c/Change%20Control/ Glossary/c/Change Control.md glossary/c.md#change-control +https://tarf.co.uk/Glossary/c/Change%20Management/ Glossary/c/Change Management.md glossary/c.md#change-management +https://tarf.co.uk/Glossary/c/Change%20Request/ Glossary/c/Change Request.md glossary/c.md#change-request +https://tarf.co.uk/Glossary/c/CHAP/ Glossary/c/CHAP.md glossary/c.md#chap +https://tarf.co.uk/Glossary/c/Chrome%20OS/ Glossary/c/Chrome OS.md glossary/c.md#chrome-os +https://tarf.co.uk/Glossary/c/Circuit%20Switched/ Glossary/c/Circuit Switched.md glossary/c.md#circuit-switched +https://tarf.co.uk/Glossary/c/Class%204%20Softswitch/ Glossary/c/Class 4 Softswitch.md glossary/c.md#class-4-softswitch +https://tarf.co.uk/Glossary/c/Class%205%20Softswitch/ Glossary/c/Class 5 Softswitch.md glossary/c.md#class-5-softswitch +https://tarf.co.uk/Glossary/c/Class%20Model/ Glossary/c/Class Model.md glossary/c.md#class-model +https://tarf.co.uk/Glossary/c/CLF/ Glossary/c/CLF.md glossary/c.md#clf +https://tarf.co.uk/Glossary/c/CMDB/ Glossary/c/CMDB.md glossary/c.md#cmdb +https://tarf.co.uk/Glossary/c/CNG/ Glossary/c/CNG.md glossary/c.md#cng +https://tarf.co.uk/Glossary/c/Cobol/ Glossary/c/Cobol.md glossary/c.md#cobol +https://tarf.co.uk/Glossary/c/Code%20and%20Fix%20Lifecycle/ Glossary/c/Code and Fix Lifecycle.md glossary/c.md#code-and-fix-lifecycle +https://tarf.co.uk/Glossary/c/Codecs/ Glossary/c/Codecs.md glossary/c.md#codecs +https://tarf.co.uk/Glossary/c/Coding%20Standard/ Glossary/c/Coding Standard.md glossary/c.md#coding-standard +https://tarf.co.uk/Glossary/c/Coding/ Glossary/c/Coding.md glossary/c.md#coding +https://tarf.co.uk/Glossary/c/CoE/ Glossary/c/CoE.md glossary/c.md#coe +https://tarf.co.uk/Glossary/c/COFDM/ Glossary/c/COFDM.md glossary/c.md#cofdm +https://tarf.co.uk/Glossary/c/Collaboration%20Model/ Glossary/c/Collaboration Model.md glossary/c.md#collaboration-model +https://tarf.co.uk/Glossary/c/Collaborative%20Construction/ Glossary/c/Collaborative Construction.md glossary/c.md#collaborative-construction +https://tarf.co.uk/Glossary/c/Collective%20code%20ownership/ Glossary/c/Collective code ownership.md glossary/c.md#collective-code-ownership +https://tarf.co.uk/Glossary/c/COLO/ Glossary/c/COLO.md glossary/c.md#colo +https://tarf.co.uk/Glossary/c/Compatibility%20Test/ Glossary/c/Compatibility Test.md glossary/c.md#compatibility-test +https://tarf.co.uk/Glossary/c/Compiled%20Language/ Glossary/c/Compiled Language.md glossary/c.md#compiled-language +https://tarf.co.uk/Glossary/c/Compiler%20Back%20end/ Glossary/c/Compiler Back end.md glossary/c.md#compiler-back-end +https://tarf.co.uk/Glossary/c/Compiler%20Front%20end/ Glossary/c/Compiler Front end.md glossary/c.md#compiler-front-end +https://tarf.co.uk/Glossary/c/Compiler%20Middle%20end/ Glossary/c/Compiler Middle end.md glossary/c.md#compiler-middle-end +https://tarf.co.uk/Glossary/c/Compiler%20preprocessor/ Glossary/c/Compiler preprocessor.md glossary/c.md#compiler-preprocessor +https://tarf.co.uk/Glossary/c/Complier/ Glossary/c/Complier.md glossary/c.md#compiler renamed to "Compiler" +https://tarf.co.uk/Glossary/c/Component%20Test/ Glossary/c/Component Test.md glossary/c.md#component-test +https://tarf.co.uk/Glossary/c/Component/ Glossary/c/Component.md glossary/c.md#component +https://tarf.co.uk/Glossary/c/Cone%20of%20Uncertainty/ Glossary/c/Cone of Uncertainty.md glossary/c.md#cone-of-uncertainty +https://tarf.co.uk/Glossary/c/Configuration%20Item/ Glossary/c/Configuration Item.md glossary/c.md#configuration-item +https://tarf.co.uk/Glossary/c/Configuration%20Management/ Glossary/c/Configuration Management.md glossary/c.md#configuration-management +https://tarf.co.uk/Glossary/c/Configuration%20Test/ Glossary/c/Configuration Test.md glossary/c.md#configuration-test +https://tarf.co.uk/Glossary/c/Construction%20Environment/ Glossary/c/Construction Environment.md glossary/c.md#construction-environment +https://tarf.co.uk/Glossary/c/Construction%20Lead/ Glossary/c/Construction Lead.md glossary/c.md#construction-lead +https://tarf.co.uk/Glossary/c/Construction%20Standard/ Glossary/c/Construction Standard.md glossary/c.md#construction-standard +https://tarf.co.uk/Glossary/c/Construction%20Test%20Environment/ Glossary/c/Construction Test Environment.md glossary/c.md#construction-test-environment +https://tarf.co.uk/Glossary/c/Construction%20Testing/ Glossary/c/Construction Testing.md glossary/c.md#construction-testing +https://tarf.co.uk/Glossary/c/Construction/ Glossary/c/Construction.md glossary/c.md#construction +https://tarf.co.uk/Glossary/c/Construx%20Knowledge%20Area/ Glossary/c/Construx Knowledge Area.md glossary/s.md#software-engineering-body-of-knowledge renamed to "Software Engineering Body of Knowledge" +https://tarf.co.uk/Glossary/c/Continuous%20Deployment/ Glossary/c/Continuous Deployment.md glossary/c.md#continuous-deployment +https://tarf.co.uk/Glossary/c/continuous%20integration/ Glossary/c/continuous integration.md glossary/c.md#continuous-integration +https://tarf.co.uk/Glossary/c/Control%20Plane/ Glossary/c/Control Plane.md glossary/c.md#control-plane +https://tarf.co.uk/Glossary/c/Copyleft/ Glossary/c/Copyleft.md glossary/c.md#copyleft +https://tarf.co.uk/Glossary/c/Corrective%20Activity%20Management/ Glossary/c/Corrective Activity Management.md glossary/c.md dropped +https://tarf.co.uk/Glossary/c/CoS/ Glossary/c/CoS.md glossary/c.md#cos +https://tarf.co.uk/Glossary/c/COTS/ Glossary/c/COTS.md glossary/c.md#cots +https://tarf.co.uk/Glossary/c/CPE/ Glossary/c/CPE.md glossary/c.md#cpe +https://tarf.co.uk/Glossary/c/CPM/ Glossary/c/CPM.md glossary/c.md#cpm +https://tarf.co.uk/Glossary/c/CPS/ Glossary/c/CPS.md glossary/c.md#cps +https://tarf.co.uk/Glossary/c/CRC%20Cards/ Glossary/c/CRC Cards.md glossary/c.md#crc-cards +https://tarf.co.uk/Glossary/c/CRC/ Glossary/c/CRC.md glossary/c.md#crc +https://tarf.co.uk/Glossary/c/CSCF/ Glossary/c/CSCF.md glossary/c.md#cscf +https://tarf.co.uk/Glossary/c/csh/ Glossary/c/csh.md glossary/c.md#csh +https://tarf.co.uk/Glossary/c/Csharp/ Glossary/c/Csharp.md glossary/c.md#c_2 renamed to "C#" +https://tarf.co.uk/Glossary/c/CSI/ Glossary/c/CSI.md glossary/c.md#csi +https://tarf.co.uk/Glossary/c/CSMA/ Glossary/c/CSMA.md glossary/c.md#csma +https://tarf.co.uk/Glossary/c/CSS/ Glossary/c/CSS.md glossary/c.md#css +https://tarf.co.uk/Glossary/c/CUE%20/ Glossary/c/CUE .md glossary/c.md#cue +https://tarf.co.uk/Glossary/c/Customer%20Development/ Glossary/c/Customer Development.md glossary/c.md#customer-development +https://tarf.co.uk/Glossary/c/CVSD/ Glossary/c/CVSD.md glossary/c.md#cvsd +https://tarf.co.uk/Glossary/d/d/ Glossary/d/d.md glossary/d.md letter index +https://tarf.co.uk/Glossary/d/DAD/ Glossary/d/DAD.md glossary/d.md#dad +https://tarf.co.uk/Glossary/d/Daily%20Meeting/ Glossary/d/Daily Meeting.md glossary/d.md#daily-meeting +https://tarf.co.uk/Glossary/d/Daily%20Scrum/ Glossary/d/Daily Scrum.md glossary/d.md#daily-scrum +https://tarf.co.uk/Glossary/d/Dark%20Fibre/ Glossary/d/Dark Fibre.md glossary/d.md#dark-fibre +https://tarf.co.uk/Glossary/d/DASS1/ Glossary/d/DASS1.md glossary/d.md#dass1 +https://tarf.co.uk/Glossary/d/DASS2/ Glossary/d/DASS2.md glossary/d.md#dass2 +https://tarf.co.uk/Glossary/d/data%20modeling/ Glossary/d/data modeling.md glossary/d.md#data-modelling renamed to "Data modelling" +https://tarf.co.uk/Glossary/d/Data%20Plane/ Glossary/d/Data Plane.md glossary/d.md#data-plane +https://tarf.co.uk/Glossary/d/Database%20Lead/ Glossary/d/Database Lead.md glossary/d.md#database-lead +https://tarf.co.uk/Glossary/d/Date%20Warehouse/ Glossary/d/Date Warehouse.md glossary/d.md#data-warehouse renamed to "Data Warehouse" +https://tarf.co.uk/Glossary/d/DCE/ Glossary/d/DCE.md glossary/d.md#dce +https://tarf.co.uk/Glossary/d/DDK/ Glossary/d/DDK.md glossary/d.md#ddk +https://tarf.co.uk/Glossary/d/DDOS/ Glossary/d/DDOS.md glossary/d.md#ddos +https://tarf.co.uk/Glossary/d/Decomposition%20Estimation/ Glossary/d/Decomposition Estimation.md glossary/d.md#decomposition-estimation +https://tarf.co.uk/Glossary/d/Defect%20Management/ Glossary/d/Defect Management.md glossary/d.md#defect-management +https://tarf.co.uk/Glossary/d/Defect%20Report/ Glossary/d/Defect Report.md glossary/d.md#defect-report +https://tarf.co.uk/Glossary/d/Defect/ Glossary/d/Defect.md glossary/d.md#defect +https://tarf.co.uk/Glossary/d/Definition%20of%20Done/ Glossary/d/Definition of Done.md glossary/d.md#definition-of-done +https://tarf.co.uk/Glossary/d/Definition%20of%20Ready/ Glossary/d/Definition of Ready.md glossary/d.md#definition-of-ready +https://tarf.co.uk/Glossary/d/Deliverable/ Glossary/d/Deliverable.md glossary/d.md#deliverable +https://tarf.co.uk/Glossary/d/Deplhi/ Glossary/d/Deplhi.md glossary/d.md#delphi renamed to "Delphi" +https://tarf.co.uk/Glossary/d/Deployment%20Lead/ Glossary/d/Deployment Lead.md glossary/d.md#deployment-lead +https://tarf.co.uk/Glossary/d/Deployment%20Plan/ Glossary/d/Deployment Plan.md glossary/d.md#deployment-plan +https://tarf.co.uk/Glossary/d/Deployment/ Glossary/d/Deployment.md glossary/d.md#deployment +https://tarf.co.uk/Glossary/d/DER/ Glossary/d/DER.md glossary/d.md#der +https://tarf.co.uk/Glossary/d/DES/ Glossary/d/DES.md glossary/d.md#des +https://tarf.co.uk/Glossary/d/Design%20Lead/ Glossary/d/Design Lead.md glossary/d.md#design-lead +https://tarf.co.uk/Glossary/d/Design%20Pattern/ Glossary/d/Design Pattern.md glossary/d.md#design-pattern +https://tarf.co.uk/Glossary/d/Design%20to%20Tools/ Glossary/d/Design to Tools.md glossary/d.md#design-to-tools +https://tarf.co.uk/Glossary/d/Design/ Glossary/d/Design.md glossary/d.md#design +https://tarf.co.uk/Glossary/d/Desk%20Check/ Glossary/d/Desk Check.md glossary/d.md#desk-check +https://tarf.co.uk/Glossary/d/Detailed%20Design/ Glossary/d/Detailed Design.md glossary/d.md#detailed-design +https://tarf.co.uk/Glossary/d/Detailed%20Schedule/ Glossary/d/Detailed Schedule.md glossary/d.md#detailed-schedule +https://tarf.co.uk/Glossary/d/Developer%20Integration%20Test/ Glossary/d/Developer Integration Test.md glossary/d.md#developer-integration-test +https://tarf.co.uk/Glossary/d/Developer/ Glossary/d/Developer.md glossary/d.md#developer +https://tarf.co.uk/Glossary/d/development%20environment/ Glossary/d/development environment.md glossary/d.md#development-environment +https://tarf.co.uk/Glossary/d/development%20process/ Glossary/d/development process.md glossary/d.md#development-process +https://tarf.co.uk/Glossary/d/DevOps/ Glossary/d/DevOps.md glossary/d.md#devops +https://tarf.co.uk/Glossary/d/DHCP/ Glossary/d/DHCP.md glossary/d.md#dhcp +https://tarf.co.uk/Glossary/d/Diagram/ Glossary/d/Diagram.md glossary/d.md#diagram +https://tarf.co.uk/Glossary/d/DLE/ Glossary/d/DLE.md glossary/d.md#dle +https://tarf.co.uk/Glossary/d/DLP/ Glossary/d/DLP.md glossary/d.md#dlp +https://tarf.co.uk/Glossary/d/DMZ/ Glossary/d/DMZ.md glossary/d.md#dmz +https://tarf.co.uk/Glossary/d/DN/ Glossary/d/DN.md glossary/d.md#dn +https://tarf.co.uk/Glossary/d/DNB/ Glossary/d/DNB.md glossary/d.md#dnb +https://tarf.co.uk/Glossary/d/DNS%20-%20A/ Glossary/d/DNS - A.md glossary/d.md#dns-a +https://tarf.co.uk/Glossary/d/DNS%20-%20AAAA/ Glossary/d/DNS - AAAA.md glossary/d.md#dns-aaaa +https://tarf.co.uk/Glossary/d/DNS%20-%20CNAME/ Glossary/d/DNS - CNAME.md glossary/d.md#dns-cname +https://tarf.co.uk/Glossary/d/DNS%20-%20DNAME/ Glossary/d/DNS - DNAME.md glossary/d.md#dns-dname +https://tarf.co.uk/Glossary/d/DNS%20-%20SRV/ Glossary/d/DNS - SRV.md glossary/d.md#dns-srv +https://tarf.co.uk/Glossary/d/DNS%20-%20TXT/ Glossary/d/DNS - TXT.md glossary/d.md#dns-txt +https://tarf.co.uk/Glossary/d/DNS%20Spoofing/ Glossary/d/DNS Spoofing.md glossary/d.md#dns-spoofing +https://tarf.co.uk/Glossary/d/DNS/ Glossary/d/DNS.md glossary/d.md#dns +https://tarf.co.uk/Glossary/d/Docker/ Glossary/d/Docker.md glossary/d.md#docker +https://tarf.co.uk/Glossary/d/Document%20Collaboration/ Glossary/d/Document Collaboration.md glossary/d.md#document-collaboration +https://tarf.co.uk/Glossary/d/Domain%20Name/ Glossary/d/Domain Name.md glossary/d.md#domain-name +https://tarf.co.uk/Glossary/d/Dot1Q%20trunking/ Glossary/d/Dot1Q trunking.md glossary/d.md#dot1q-trunking +https://tarf.co.uk/Glossary/d/Dot1Q/ Glossary/d/Dot1Q.md glossary/d.md#dot1q +https://tarf.co.uk/Glossary/d/dotNET/ Glossary/d/dotNET.md glossary/symbols.md#net renamed to ".NET" +https://tarf.co.uk/Glossary/d/Downstream/ Glossary/d/Downstream.md glossary/d.md#downstream +https://tarf.co.uk/Glossary/d/DP/ Glossary/d/DP.md glossary/d.md#dp +https://tarf.co.uk/Glossary/d/DPI/ Glossary/d/DPI.md glossary/d.md#dpi +https://tarf.co.uk/Glossary/d/DPNSS/ Glossary/d/DPNSS.md glossary/d.md#dpnss +https://tarf.co.uk/Glossary/d/DQ/ Glossary/d/DQ.md glossary/d.md#dq +https://tarf.co.uk/Glossary/d/Draft/ Glossary/d/Draft.md glossary/d.md#draft +https://tarf.co.uk/Glossary/d/driver/ Glossary/d/driver.md glossary/d.md#driver +https://tarf.co.uk/Glossary/d/DSCP/ Glossary/d/DSCP.md glossary/d.md#dscp +https://tarf.co.uk/Glossary/d/DSDM/ Glossary/d/DSDM.md glossary/d.md#dsdm +https://tarf.co.uk/Glossary/d/DSL/ Glossary/d/DSL.md glossary/d.md#dsl +https://tarf.co.uk/Glossary/d/DSLAM/ Glossary/d/DSLAM.md glossary/d.md#dslam +https://tarf.co.uk/Glossary/d/DTE/ Glossary/d/DTE.md glossary/d.md#dte +https://tarf.co.uk/Glossary/d/DTMF/ Glossary/d/DTMF.md glossary/d.md#dtmf +https://tarf.co.uk/Glossary/d/DTT/ Glossary/d/DTT.md glossary/d.md#dtt +https://tarf.co.uk/Glossary/d/DTV/ Glossary/d/DTV.md glossary/d.md#dtv +https://tarf.co.uk/Glossary/d/Duplex/ Glossary/d/Duplex.md glossary/d.md#duplex +https://tarf.co.uk/Glossary/d/DVSR/ Glossary/d/DVSR.md glossary/d.md#dvsr +https://tarf.co.uk/Glossary/e/E%20NodeB/ Glossary/e/E NodeB.md glossary/e.md#e-nodeb +https://tarf.co.uk/Glossary/e/E-UTRAN/ Glossary/e/E-UTRAN.md glossary/e.md#e-utran +https://tarf.co.uk/Glossary/e/E.164/ Glossary/e/E.164.md glossary/e.md#e164 +https://tarf.co.uk/Glossary/e/e/ Glossary/e/e.md glossary/e.md letter index +https://tarf.co.uk/Glossary/e/EAD/ Glossary/e/EAD.md glossary/e.md#ead +https://tarf.co.uk/Glossary/e/EAP/ Glossary/e/EAP.md glossary/e.md#eap +https://tarf.co.uk/Glossary/e/Earned%20Value%20Management/ Glossary/e/Earned Value Management.md glossary/e.md#earned-value-management +https://tarf.co.uk/Glossary/e/Earned%20Value/ Glossary/e/Earned Value.md glossary/e.md#earned-value +https://tarf.co.uk/Glossary/e/Echo%20Cancellation%20-%20G.168/ Glossary/e/Echo Cancellation - G.168.md glossary/e.md#echo-cancellation-g168 +https://tarf.co.uk/Glossary/e/Echo%20cancellation/ Glossary/e/Echo cancellation.md glossary/e.md#echo-cancellation +https://tarf.co.uk/Glossary/e/Echo/ Glossary/e/Echo.md glossary/e.md#echo +https://tarf.co.uk/Glossary/e/EDB/ Glossary/e/EDB.md glossary/e.md#edb +https://tarf.co.uk/Glossary/e/EDGE/ Glossary/e/EDGE.md glossary/e.md#edge +https://tarf.co.uk/Glossary/e/Editor/ Glossary/e/Editor.md glossary/e.md#editor +https://tarf.co.uk/Glossary/e/EEE%20802.1Qay/ Glossary/e/EEE 802.1Qay.md glossary/i.md#ieee-8021qay renamed to "IEEE 802.1Qay" +https://tarf.co.uk/Glossary/e/EFM/ Glossary/e/EFM.md glossary/e.md#efm +https://tarf.co.uk/Glossary/e/Egress/ Glossary/e/Egress.md glossary/e.md#egress +https://tarf.co.uk/Glossary/e/EIA/ Glossary/e/EIA.md glossary/e.md#eia +https://tarf.co.uk/Glossary/e/EIGRP/ Glossary/e/EIGRP.md glossary/e.md#eigrp +https://tarf.co.uk/Glossary/e/embedded%20systems%20programming/ Glossary/e/embedded systems programming.md glossary/e.md#embedded-systems-programming +https://tarf.co.uk/Glossary/e/Engineering%20Discussion/ Glossary/e/Engineering Discussion.md glossary/e.md#engineering-discussion +https://tarf.co.uk/Glossary/e/Engineering%20Management/ Glossary/e/Engineering Management.md glossary/e.md#engineering-management +https://tarf.co.uk/Glossary/e/Engineering%20Process/ Glossary/e/Engineering Process.md glossary/e.md#engineering-process +https://tarf.co.uk/Glossary/e/EoFTTC/ Glossary/e/EoFTTC.md glossary/e.md#eofttc +https://tarf.co.uk/Glossary/e/EoS/ Glossary/e/EoS.md glossary/e.md#eos +https://tarf.co.uk/Glossary/e/ePDG/ Glossary/e/ePDG.md glossary/e.md#epdg +https://tarf.co.uk/Glossary/e/Epic/ Glossary/e/Epic.md glossary/e.md#epic +https://tarf.co.uk/Glossary/e/EPWS/ Glossary/e/EPWS.md glossary/e.md#epws +https://tarf.co.uk/Glossary/e/ESB/ Glossary/e/ESB.md glossary/e.md#esb +https://tarf.co.uk/Glossary/e/Estimate/ Glossary/e/Estimate.md glossary/e.md#estimate +https://tarf.co.uk/Glossary/e/Estimation/ Glossary/e/Estimation.md glossary/e.md#estimation +https://tarf.co.uk/Glossary/e/Ethernet%20over%20twisted%20pair/ Glossary/e/Ethernet over twisted pair.md glossary/e.md#ethernet-over-twisted-pair +https://tarf.co.uk/Glossary/e/Ethical%20Hacking/ Glossary/e/Ethical Hacking.md glossary/e.md#ethical-hacking +https://tarf.co.uk/Glossary/e/ETL/ Glossary/e/ETL.md glossary/e.md#etl +https://tarf.co.uk/Glossary/e/ETSI/ Glossary/e/ETSI.md glossary/e.md#etsi +https://tarf.co.uk/Glossary/e/Evaluation/ Glossary/e/Evaluation.md glossary/e.md dropped +https://tarf.co.uk/Glossary/e/EVC/ Glossary/e/EVC.md glossary/e.md#evc +https://tarf.co.uk/Glossary/e/Evolutionary%20Delivery%20Lifecycle/ Glossary/e/Evolutionary Delivery Lifecycle.md glossary/e.md#evolutionary-delivery-lifecycle +https://tarf.co.uk/Glossary/e/Evolutionary%20Prototyping%20Lifecycle/ Glossary/e/Evolutionary Prototyping Lifecycle.md glossary/e.md#evolutionary-prototyping-lifecycle +https://tarf.co.uk/Glossary/e/EVPL/ Glossary/e/EVPL.md glossary/e.md#evpl +https://tarf.co.uk/Glossary/e/Expert%20Judgment/ Glossary/e/Expert Judgment.md glossary/e.md#expert-judgement renamed to "Expert Judgement" +https://tarf.co.uk/Glossary/e/Explicit%20Change%20Control/ Glossary/e/Explicit Change Control.md glossary/e.md#explicit-change-control +https://tarf.co.uk/Glossary/e/Explicit%20Risk%20Management/ Glossary/e/Explicit Risk Management.md glossary/e.md#explicit-risk-management +https://tarf.co.uk/Glossary/e/Exploratory%20Testing/ Glossary/e/Exploratory Testing.md glossary/e.md#exploratory-testing +https://tarf.co.uk/Glossary/e/Extrinsic%20Risk%20Management/ Glossary/e/Extrinsic Risk Management.md glossary/e.md#extrinsic-risk-management +https://tarf.co.uk/Glossary/f/f/ Glossary/f/f.md glossary/f.md letter index +https://tarf.co.uk/Glossary/f/Facilitation/ Glossary/f/Facilitation.md glossary/f.md#facilitation +https://tarf.co.uk/Glossary/f/FCS/ Glossary/f/FCS.md glossary/f.md#fcs +https://tarf.co.uk/Glossary/f/FDDI/ Glossary/f/FDDI.md glossary/f.md#fddi +https://tarf.co.uk/Glossary/f/feature%20creep/ Glossary/f/feature creep.md glossary/f.md#feature-creep +https://tarf.co.uk/Glossary/f/Feature%20Test/ Glossary/f/Feature Test.md glossary/f.md#feature-test +https://tarf.co.uk/Glossary/f/Feature/ Glossary/f/Feature.md glossary/f.md#feature +https://tarf.co.uk/Glossary/f/FemtoCell/ Glossary/f/FemtoCell.md glossary/f.md#femtocell +https://tarf.co.uk/Glossary/f/FIPS%20140-2/ Glossary/f/FIPS 140-2.md glossary/f.md#fips-140-2 +https://tarf.co.uk/Glossary/f/FIPS/ Glossary/f/FIPS.md glossary/f.md#fips +https://tarf.co.uk/Glossary/f/Firewall/ Glossary/f/Firewall.md glossary/f.md#firewall +https://tarf.co.uk/Glossary/f/FNA/ Glossary/f/FNA.md glossary/f.md#fna +https://tarf.co.uk/Glossary/f/Formal%20Testing/ Glossary/f/Formal Testing.md glossary/f.md#formal-testing +https://tarf.co.uk/Glossary/f/Fortran/ Glossary/f/Fortran.md glossary/f.md#fortran +https://tarf.co.uk/Glossary/f/Forwarding%20plane/ Glossary/f/Forwarding plane.md glossary/f.md#forwarding-plane +https://tarf.co.uk/Glossary/f/FQDN/ Glossary/f/FQDN.md glossary/f.md#fqdn +https://tarf.co.uk/Glossary/f/Frame%20Relay/ Glossary/f/Frame Relay.md glossary/f.md#frame-relay +https://tarf.co.uk/Glossary/f/Frequent%20Releases/ Glossary/f/Frequent Releases.md glossary/f.md#frequent-releases +https://tarf.co.uk/Glossary/f/Frequently%20Asked%20Question/ Glossary/f/Frequently Asked Question.md glossary/f.md#frequently-asked-question +https://tarf.co.uk/Glossary/f/FSK/ Glossary/f/FSK.md glossary/f.md#fsk +https://tarf.co.uk/Glossary/f/FTP/ Glossary/f/FTP.md glossary/f.md#ftp +https://tarf.co.uk/Glossary/f/FTPS/ Glossary/f/FTPS.md glossary/f.md#ftps +https://tarf.co.uk/Glossary/f/Full%20duplex/ Glossary/f/Full duplex.md glossary/f.md#full-duplex +https://tarf.co.uk/Glossary/f/Function%20Point/ Glossary/f/Function Point.md glossary/f.md#function-point +https://tarf.co.uk/Glossary/f/Functional%20Requirement/ Glossary/f/Functional Requirement.md glossary/f.md#functional-requirement +https://tarf.co.uk/Glossary/f/Functional%20Testing/ Glossary/f/Functional Testing.md glossary/f.md#functional-testing +https://tarf.co.uk/Glossary/f/Fuzzy%20Front%20End/ Glossary/f/Fuzzy Front End.md glossary/f.md#fuzzy-front-end +https://tarf.co.uk/Glossary/f/Fuzzy%20Logic/ Glossary/f/Fuzzy Logic.md glossary/f.md#fuzzy-logic +https://tarf.co.uk/Glossary/f/FXO/ Glossary/f/FXO.md glossary/f.md#fxo +https://tarf.co.uk/Glossary/f/FXS/ Glossary/f/FXS.md glossary/f.md#fxs +https://tarf.co.uk/Glossary/g/G.711.1/ Glossary/g/G.711.1.md glossary/g.md#g7111 +https://tarf.co.uk/Glossary/g/G.711/ Glossary/g/G.711.md glossary/g.md#g711 +https://tarf.co.uk/Glossary/g/G.719/ Glossary/g/G.719.md glossary/g.md#g719 +https://tarf.co.uk/Glossary/g/G.722/ Glossary/g/G.722.md glossary/g.md#g722 +https://tarf.co.uk/Glossary/g/G.723.1/ Glossary/g/G.723.1.md glossary/g.md#g7231 +https://tarf.co.uk/Glossary/g/G.726/ Glossary/g/G.726.md glossary/g.md#g726 +https://tarf.co.uk/Glossary/g/G.729/ Glossary/g/G.729.md glossary/a.md#audio-codecs renamed to "Audio Codecs" +https://tarf.co.uk/Glossary/g/G.Fast/ Glossary/g/G.Fast.md glossary/g.md#gfast +https://tarf.co.uk/Glossary/g/g/ Glossary/g/g.md glossary/g.md letter index +https://tarf.co.uk/Glossary/g/Gantt%20chart/ Glossary/g/Gantt chart.md glossary/g.md#gantt-chart +https://tarf.co.uk/Glossary/g/gap%20analysis/ Glossary/g/gap analysis.md glossary/g.md#gap-analysis +https://tarf.co.uk/Glossary/g/Gate%20Checklist/ Glossary/g/Gate Checklist.md glossary/g.md#gate-checklist +https://tarf.co.uk/Glossary/g/Gate/ Glossary/g/Gate.md glossary/g.md#gate +https://tarf.co.uk/Glossary/g/GBE%20-%20GE/ Glossary/g/GBE - GE.md glossary/g.md#gbe-ge renamed to "GBE / GE" +https://tarf.co.uk/Glossary/g/GCHQ/ Glossary/g/GCHQ.md glossary/g.md#gchq +https://tarf.co.uk/Glossary/g/Geocast/ Glossary/g/Geocast.md glossary/g.md#geocast +https://tarf.co.uk/Glossary/g/GERAN/ Glossary/g/GERAN.md glossary/g.md#geran +https://tarf.co.uk/Glossary/g/GGSN/ Glossary/g/GGSN.md glossary/g.md#ggsn +https://tarf.co.uk/Glossary/g/Gherkin/ Glossary/g/Gherkin.md glossary/g.md#gherkin +https://tarf.co.uk/Glossary/g/Gigabit%20Ethernet/ Glossary/g/Gigabit Ethernet.md glossary/g.md#gigabit-ethernet +https://tarf.co.uk/Glossary/g/Git/ Glossary/g/Git.md glossary/g.md#git +https://tarf.co.uk/Glossary/g/Given%20When%20Then/ Glossary/g/Given When Then.md glossary/g.md#given-when-then +https://tarf.co.uk/Glossary/g/GMSC/ Glossary/g/GMSC.md glossary/g.md#gmsc +https://tarf.co.uk/Glossary/g/GNU/ Glossary/g/GNU.md glossary/g.md#gnu +https://tarf.co.uk/Glossary/g/Go/ Glossary/g/Go.md glossary/g.md#go +https://tarf.co.uk/Glossary/g/Goals/ Glossary/g/Goals.md glossary/g.md#goals +https://tarf.co.uk/Glossary/g/GoF/ Glossary/g/GoF.md glossary/g.md#gof +https://tarf.co.uk/Glossary/g/Google%20Cloud%20Platform/ Glossary/g/Google Cloud Platform.md glossary/g.md#google-cloud-platform +https://tarf.co.uk/Glossary/g/GPL/ Glossary/g/GPL.md glossary/g.md#gpl +https://tarf.co.uk/Glossary/g/GPRS/ Glossary/g/GPRS.md glossary/g.md#gprs +https://tarf.co.uk/Glossary/g/GSM%20AMR/ Glossary/g/GSM AMR.md glossary/g.md#gsm-amr +https://tarf.co.uk/Glossary/g/GSM-FR/ Glossary/g/GSM-FR.md glossary/g.md#gsm-fr +https://tarf.co.uk/Glossary/g/GSM-R/ Glossary/g/GSM-R.md glossary/g.md#gsm-r +https://tarf.co.uk/Glossary/g/GSM/ Glossary/g/GSM.md glossary/g.md#gsm +https://tarf.co.uk/Glossary/g/GSN/ Glossary/g/GSN.md glossary/g.md#gsn +https://tarf.co.uk/Glossary/g/The%20first%20example%20has%20two%20steps/ Glossary/g/The first example has two steps.md glossary/g.md#gherkin renamed to "Gherkin" +https://tarf.co.uk/Glossary/g/The%20second%20example%20has%20three%20steps/ Glossary/g/The second example has three steps.md glossary/g.md#gherkin renamed to "Gherkin" +https://tarf.co.uk/Glossary/h/H.248/ Glossary/h/H.248.md glossary/h.md#h248 +https://tarf.co.uk/Glossary/h/H.323/ Glossary/h/H.323.md glossary/h.md#h323 +https://tarf.co.uk/Glossary/h/h/ Glossary/h/h.md glossary/h.md letter index +https://tarf.co.uk/Glossary/h/Half%20Duplex/ Glossary/h/Half Duplex.md glossary/h.md#half-duplex +https://tarf.co.uk/Glossary/h/Happy%20Path/ Glossary/h/Happy Path.md glossary/h.md#happy-path +https://tarf.co.uk/Glossary/h/Hash%20Function/ Glossary/h/Hash Function.md glossary/h.md#hash-function +https://tarf.co.uk/Glossary/h/HDF/ Glossary/h/HDF.md glossary/h.md#hdf +https://tarf.co.uk/Glossary/h/HDLC/ Glossary/h/HDLC.md glossary/h.md#hdlc +https://tarf.co.uk/Glossary/h/HDSL/ Glossary/h/HDSL.md glossary/h.md#hdsl +https://tarf.co.uk/Glossary/h/Heartbeat%20Retrospective/ Glossary/h/Heartbeat Retrospective.md glossary/h.md#heartbeat-retrospective +https://tarf.co.uk/Glossary/h/HiFi%20Prototype/ Glossary/h/HiFi Prototype.md glossary/h.md#hifi-prototype +https://tarf.co.uk/Glossary/h/High%20Fidelity%20Prototype/ Glossary/h/High Fidelity Prototype.md glossary/h.md#high-fidelity-prototype +https://tarf.co.uk/Glossary/h/High%20Level%20Design/ Glossary/h/High Level Design.md glossary/h.md#high-level-design +https://tarf.co.uk/Glossary/h/HLR/ Glossary/h/HLR.md glossary/h.md#hlr +https://tarf.co.uk/Glossary/h/Hotfix/ Glossary/h/Hotfix.md glossary/h.md#hotfix +https://tarf.co.uk/Glossary/h/How%20Requirement/ Glossary/h/How Requirement.md glossary/h.md#how-requirement +https://tarf.co.uk/Glossary/h/How%20Well%20Requirement/ Glossary/h/How Well Requirement.md glossary/h.md#how-well-requirement +https://tarf.co.uk/Glossary/h/HSRP/ Glossary/h/HSRP.md glossary/h.md#hsrp +https://tarf.co.uk/Glossary/h/HSS/ Glossary/h/HSS.md glossary/h.md#hss +https://tarf.co.uk/Glossary/h/HTTP/ Glossary/h/HTTP.md glossary/h.md#http +https://tarf.co.uk/Glossary/h/HTTPS/ Glossary/h/HTTPS.md glossary/h.md#https +https://tarf.co.uk/Glossary/h/Hub/ Glossary/h/Hub.md glossary/h.md#hub +https://tarf.co.uk/Glossary/h/human%20factors/ Glossary/h/human factors.md glossary/h.md#human-factors +https://tarf.co.uk/Glossary/h/Hungarian%20notation/ Glossary/h/Hungarian notation.md glossary/h.md#hungarian-notation +https://tarf.co.uk/Glossary/i/BRI/ Glossary/i/BRI.md glossary/b.md#bri +https://tarf.co.uk/Glossary/i/i/ Glossary/i/i.md glossary/i.md letter index +https://tarf.co.uk/Glossary/i/IA/ Glossary/i/IA.md glossary/i.md#ia +https://tarf.co.uk/Glossary/i/IaaS/ Glossary/i/IaaS.md glossary/i.md#iaas +https://tarf.co.uk/Glossary/i/ICE/ Glossary/i/ICE.md glossary/i.md#ice +https://tarf.co.uk/Glossary/i/ICMP/ Glossary/i/ICMP.md glossary/i.md#icmp +https://tarf.co.uk/Glossary/i/ICMPv6/ Glossary/i/ICMPv6.md glossary/i.md#icmpv6 +https://tarf.co.uk/Glossary/i/IDE/ Glossary/i/IDE.md glossary/i.md#ide +https://tarf.co.uk/Glossary/i/IDQ/ Glossary/i/IDQ.md glossary/i.md#idq +https://tarf.co.uk/Glossary/i/IEC/ Glossary/i/IEC.md glossary/i.md#iec +https://tarf.co.uk/Glossary/i/IEEE%20802.1ah/ Glossary/i/IEEE 802.1ah.md glossary/i.md#ieee-8021ah +https://tarf.co.uk/Glossary/i/IEEE/ Glossary/i/IEEE.md glossary/i.md#ieee +https://tarf.co.uk/Glossary/i/IETF/ Glossary/i/IETF.md glossary/i.md#ietf +https://tarf.co.uk/Glossary/i/IGMP/ Glossary/i/IGMP.md glossary/i.md#igmp +https://tarf.co.uk/Glossary/i/IGP/ Glossary/i/IGP.md glossary/i.md#igp +https://tarf.co.uk/Glossary/i/IGRP/ Glossary/i/IGRP.md glossary/i.md#igrp +https://tarf.co.uk/Glossary/i/IIS/ Glossary/i/IIS.md glossary/i.md#iis +https://tarf.co.uk/Glossary/i/IKE/ Glossary/i/IKE.md glossary/i.md#ike +https://tarf.co.uk/Glossary/i/iLBC/ Glossary/i/iLBC.md glossary/i.md#ilbc +https://tarf.co.uk/Glossary/i/iManager/ Glossary/i/iManager.md glossary/i.md#imanager +https://tarf.co.uk/Glossary/i/IMAP/ Glossary/i/IMAP.md glossary/i.md#imap +https://tarf.co.uk/Glossary/i/Implementation%20Lead/ Glossary/i/Implementation Lead.md glossary/i.md#implementation-lead +https://tarf.co.uk/Glossary/i/Implementation/ Glossary/i/Implementation.md glossary/i.md#implementation +https://tarf.co.uk/Glossary/i/Implicit%20Change%20Control/ Glossary/i/Implicit Change Control.md glossary/i.md#implicit-change-control +https://tarf.co.uk/Glossary/i/Implicit%20Risk%20Management/ Glossary/i/Implicit Risk Management.md glossary/i.md#implicit-risk-management +https://tarf.co.uk/Glossary/i/IMS/ Glossary/i/IMS.md glossary/i.md#ims +https://tarf.co.uk/Glossary/i/IMSI/ Glossary/i/IMSI.md glossary/i.md#imsi +https://tarf.co.uk/Glossary/i/IN/ Glossary/i/IN.md glossary/i.md#in +https://tarf.co.uk/Glossary/i/INAP/ Glossary/i/INAP.md glossary/i.md#inap +https://tarf.co.uk/Glossary/i/Inch-Pebble/ Glossary/i/Inch-Pebble.md glossary/i.md#inch-pebble +https://tarf.co.uk/Glossary/i/Incremental%20Development/ Glossary/i/Incremental Development.md glossary/i.md#incremental-development +https://tarf.co.uk/Glossary/i/Informal%20Review/ Glossary/i/Informal Review.md glossary/i.md#informal-review +https://tarf.co.uk/Glossary/i/Informal%20Testing/ Glossary/i/Informal Testing.md glossary/i.md#informal-testing +https://tarf.co.uk/Glossary/i/information%20architecture/ Glossary/i/information architecture.md glossary/i.md#information-architecture +https://tarf.co.uk/Glossary/i/information%20design/ Glossary/i/information design.md glossary/i.md#information-design +https://tarf.co.uk/Glossary/i/Information%20Radiators/ Glossary/i/Information Radiators.md glossary/i.md#information-radiators +https://tarf.co.uk/Glossary/i/Ingress/ Glossary/i/Ingress.md glossary/i.md#ingress +https://tarf.co.uk/Glossary/i/Inspection/ Glossary/i/Inspection.md glossary/i.md#inspection +https://tarf.co.uk/Glossary/i/Inspector/ Glossary/i/Inspector.md glossary/i.md#inspector +https://tarf.co.uk/Glossary/i/Institute%20of%20Electrical%20and%20Electronics%20Engineers/ Glossary/i/Institute of Electrical and Electronics Engineers.md glossary/i.md#institute-of-electrical-and-electronics-engineers +https://tarf.co.uk/Glossary/i/Integrated%20Development%20Environment/ Glossary/i/Integrated Development Environment.md glossary/i.md#integrated-development-environment +https://tarf.co.uk/Glossary/i/Integration%20Test/ Glossary/i/Integration Test.md glossary/i.md#integration-test +https://tarf.co.uk/Glossary/i/Integration/ Glossary/i/Integration.md glossary/i.md#integration +https://tarf.co.uk/Glossary/i/International%20Organization%20for%20Standardization/ Glossary/i/International Organization for Standardization.md glossary/i.md#iso renamed to "ISO" +https://tarf.co.uk/Glossary/i/Internet%20Protocol%20Suite/ Glossary/i/Internet Protocol Suite.md glossary/i.md#internet-protocol-suite +https://tarf.co.uk/Glossary/i/Interpreted%20Language/ Glossary/i/Interpreted Language.md glossary/i.md#interpreted-language +https://tarf.co.uk/Glossary/i/Intrinsic%20Risk%20Management/ Glossary/i/Intrinsic Risk Management.md glossary/i.md#intrinsic-risk-management +https://tarf.co.uk/Glossary/i/INVEST/ Glossary/i/INVEST.md glossary/i.md#invest +https://tarf.co.uk/Glossary/i/IOS%20cisco%20OS/ Glossary/i/IOS cisco OS.md glossary/i.md#ios renamed to "IOS" +https://tarf.co.uk/Glossary/i/IPS/ Glossary/i/IPS.md glossary/i.md#ips +https://tarf.co.uk/Glossary/i/IPSC/ Glossary/i/IPSC.md glossary/i.md#ipsc +https://tarf.co.uk/Glossary/i/IPSec/ Glossary/i/IPSec.md glossary/i.md#ipsec +https://tarf.co.uk/Glossary/i/IPTV/ Glossary/i/IPTV.md glossary/i.md#iptv +https://tarf.co.uk/Glossary/i/IPv4/ Glossary/i/IPv4.md glossary/i.md#ipv4 +https://tarf.co.uk/Glossary/i/IPv6/ Glossary/i/IPv6.md glossary/i.md#ipv6 +https://tarf.co.uk/Glossary/i/ISDN/ Glossary/i/ISDN.md glossary/i.md#isdn +https://tarf.co.uk/Glossary/i/ISDN2/ Glossary/i/ISDN2.md glossary/i.md#isdn2 +https://tarf.co.uk/Glossary/i/ISDN30/ Glossary/i/ISDN30.md glossary/i.md#isdn30 +https://tarf.co.uk/Glossary/i/ISO/ Glossary/i/ISO.md glossary/i.md#iso +https://tarf.co.uk/Glossary/i/ISP/ Glossary/i/ISP.md glossary/i.md#isp +https://tarf.co.uk/Glossary/i/Issue%20Management/ Glossary/i/Issue Management.md glossary/i.md#issue-management +https://tarf.co.uk/Glossary/i/Issue%20tracker/ Glossary/i/Issue tracker.md glossary/i.md#issue-tracker +https://tarf.co.uk/Glossary/i/Issue/ Glossary/i/Issue.md glossary/i.md#issue +https://tarf.co.uk/Glossary/i/ISUP/ Glossary/i/ISUP.md glossary/i.md#isup +https://tarf.co.uk/Glossary/i/Item/ Glossary/i/Item.md glossary/i.md#item +https://tarf.co.uk/Glossary/i/Iteration/ Glossary/i/Iteration.md glossary/i.md#iteration +https://tarf.co.uk/Glossary/i/Iterative%20Development/ Glossary/i/Iterative Development.md glossary/i.md#iterative-development +https://tarf.co.uk/Glossary/i/ITIL/ Glossary/i/ITIL.md glossary/i.md#itil +https://tarf.co.uk/Glossary/i/ITU-T/ Glossary/i/ITU-T.md glossary/i.md#itu-t +https://tarf.co.uk/Glossary/i/ITU/ Glossary/i/ITU.md glossary/i.md#itu +https://tarf.co.uk/Glossary/i/IUP/ Glossary/i/IUP.md glossary/i.md#iup +https://tarf.co.uk/Glossary/i/IVR/ Glossary/i/IVR.md glossary/i.md#ivr +https://tarf.co.uk/Glossary/i/PRI/ Glossary/i/PRI.md glossary/p.md#pri +https://tarf.co.uk/Glossary/j/j/ Glossary/j/j.md glossary/j.md letter index +https://tarf.co.uk/Glossary/j/Java/ Glossary/j/Java.md glossary/j.md#java +https://tarf.co.uk/Glossary/j/Javascript/ Glossary/j/Javascript.md glossary/j.md#javascript +https://tarf.co.uk/Glossary/j/Jitter%20buffer/ Glossary/j/Jitter buffer.md glossary/j.md#jitter-buffer +https://tarf.co.uk/Glossary/j/Jitter/ Glossary/j/Jitter.md glossary/j.md#jitter +https://tarf.co.uk/Glossary/j/Joint%20Application%20Development/ Glossary/j/Joint Application Development.md glossary/j.md#joint-application-development +https://tarf.co.uk/Glossary/j/JSON-RPC/ Glossary/j/JSON-RPC.md glossary/j.md#json-rpc +https://tarf.co.uk/Glossary/j/JSON/ Glossary/j/JSON.md glossary/j.md#json +https://tarf.co.uk/Glossary/k/K%20%26%20R%20C/ Glossary/k/K & R C.md glossary/k.md#k-r-c +https://tarf.co.uk/Glossary/k/K%20%26%20R%20indent%20style/ Glossary/k/K & R indent style.md glossary/k.md#k-r-indent-style +https://tarf.co.uk/Glossary/k/K%20%26%20R/ Glossary/k/K & R.md glossary/k.md#k-r +https://tarf.co.uk/Glossary/k/k/ Glossary/k/k.md glossary/k.md letter index +https://tarf.co.uk/Glossary/k/Kanban%20Board/ Glossary/k/Kanban Board.md glossary/k.md#kanban-board +https://tarf.co.uk/Glossary/k/Kanban/ Glossary/k/Kanban.md glossary/k.md#kanban +https://tarf.co.uk/Glossary/k/KDL/ Glossary/k/KDL.md glossary/k.md#kdl +https://tarf.co.uk/Glossary/k/Kerberos/ Glossary/k/Kerberos.md glossary/k.md#kerberos +https://tarf.co.uk/Glossary/k/Kernel/ Glossary/k/Kernel.md glossary/k.md#kernel +https://tarf.co.uk/Glossary/k/KISS%20Principle/ Glossary/k/KISS Principle.md glossary/k.md#kiss-principle +https://tarf.co.uk/Glossary/k/Knowledge%20Management/ Glossary/k/Knowledge Management.md glossary/k.md#knowledge-management +https://tarf.co.uk/Glossary/k/KPI/ Glossary/k/KPI.md glossary/k.md#kpi +https://tarf.co.uk/Glossary/k/ksh/ Glossary/k/ksh.md glossary/k.md#ksh +https://tarf.co.uk/Glossary/l/l/ Glossary/l/l.md glossary/l.md letter index +https://tarf.co.uk/Glossary/l/L1/ Glossary/l/L1.md glossary/l.md#l1 +https://tarf.co.uk/Glossary/l/L2/ Glossary/l/L2.md glossary/l.md#l2 +https://tarf.co.uk/Glossary/l/L2F/ Glossary/l/L2F.md glossary/l.md#l2f +https://tarf.co.uk/Glossary/l/L2S/ Glossary/l/L2S.md glossary/l.md dropped +https://tarf.co.uk/Glossary/l/L2TP/ Glossary/l/L2TP.md glossary/l.md#l2tp +https://tarf.co.uk/Glossary/l/L2VPN/ Glossary/l/L2VPN.md glossary/l.md#l2vpn +https://tarf.co.uk/Glossary/l/L3VPN/ Glossary/l/L3VPN.md glossary/l.md#l3vpn +https://tarf.co.uk/Glossary/l/LAC/ Glossary/l/LAC.md glossary/l.md#lac +https://tarf.co.uk/Glossary/l/LACP/ Glossary/l/LACP.md glossary/l.md#lacp +https://tarf.co.uk/Glossary/l/LAG/ Glossary/l/LAG.md glossary/l.md#lag +https://tarf.co.uk/Glossary/l/LAN/ Glossary/l/LAN.md glossary/l.md#lan +https://tarf.co.uk/Glossary/l/LANE/ Glossary/l/LANE.md glossary/l.md#lane +https://tarf.co.uk/Glossary/l/LAS/ Glossary/l/LAS.md glossary/l.md dropped +https://tarf.co.uk/Glossary/l/Latency/ Glossary/l/Latency.md glossary/l.md#latency +https://tarf.co.uk/Glossary/l/Lawful%20intercept/ Glossary/l/Lawful intercept.md glossary/l.md#lawful-intercept +https://tarf.co.uk/Glossary/l/LCR/ Glossary/l/LCR.md glossary/l.md#lcr +https://tarf.co.uk/Glossary/l/LDAP/ Glossary/l/LDAP.md glossary/l.md#ldap +https://tarf.co.uk/Glossary/l/Lead%20Time/ Glossary/l/Lead Time.md glossary/l.md#lead-time +https://tarf.co.uk/Glossary/l/lean%20programming/ Glossary/l/lean programming.md glossary/l.md#lean-programming +https://tarf.co.uk/Glossary/l/Lean/ Glossary/l/Lean.md glossary/l.md#lean +https://tarf.co.uk/Glossary/l/Leased%20Line/ Glossary/l/Leased Line.md glossary/l.md#leased-line +https://tarf.co.uk/Glossary/l/LeSS/ Glossary/l/LeSS.md glossary/l.md#less +https://tarf.co.uk/Glossary/l/Lifecycle/ Glossary/l/Lifecycle.md glossary/l.md#lifecycle +https://tarf.co.uk/Glossary/l/Lines%20of%20Code/ Glossary/l/Lines of Code.md glossary/l.md#lines-of-code +https://tarf.co.uk/Glossary/l/Link/ Glossary/l/Link.md glossary/l.md dropped +https://tarf.co.uk/Glossary/l/Linux/ Glossary/l/Linux.md glossary/l.md#linux +https://tarf.co.uk/Glossary/l/lisp/ Glossary/l/lisp.md glossary/l.md#lisp +https://tarf.co.uk/Glossary/l/Lite/ Glossary/l/Lite.md glossary/l.md dropped +https://tarf.co.uk/Glossary/l/LL/ Glossary/l/LL.md glossary/l.md#ll +https://tarf.co.uk/Glossary/l/LNS/ Glossary/l/LNS.md glossary/l.md#lns +https://tarf.co.uk/Glossary/l/Local%20Build/ Glossary/l/Local Build.md glossary/l.md#local-build +https://tarf.co.uk/Glossary/l/Local%20Development%20Environment/ Glossary/l/Local Development Environment.md glossary/l.md#local-development-environment +https://tarf.co.uk/Glossary/l/Local%20Loop/ Glossary/l/Local Loop.md glossary/l.md#local-loop +https://tarf.co.uk/Glossary/l/Local%20Test%20Environment/ Glossary/l/Local Test Environment.md glossary/l.md#local-test-environment +https://tarf.co.uk/Glossary/l/LoFi%20Prototype/ Glossary/l/LoFi Prototype.md glossary/l.md#lofi-prototype +https://tarf.co.uk/Glossary/l/Low%20Fidelity%20Prototype/ Glossary/l/Low Fidelity Prototype.md glossary/l.md#low-fidelity-prototype +https://tarf.co.uk/Glossary/l/Low%20Level%20Design/ Glossary/l/Low Level Design.md glossary/l.md#low-level-design +https://tarf.co.uk/Glossary/l/LPS/ Glossary/l/LPS.md glossary/l.md dropped +https://tarf.co.uk/Glossary/l/LSP/ Glossary/l/LSP.md glossary/l.md#lsp +https://tarf.co.uk/Glossary/l/LSR/ Glossary/l/LSR.md glossary/l.md#lsr +https://tarf.co.uk/Glossary/l/LTE%20%284G%29/ Glossary/l/LTE (4G).md glossary/l.md#lte-4g +https://tarf.co.uk/Glossary/l/LTE%20and%20Mobile%20Internet/ Glossary/l/LTE and Mobile Internet.md glossary/l.md#lte-4g renamed to "LTE (4G)" +https://tarf.co.uk/Glossary/l/LTS/ Glossary/l/LTS.md glossary/l.md#lts +https://tarf.co.uk/Glossary/l/Lua/ Glossary/l/Lua.md glossary/l.md#lua +https://tarf.co.uk/Glossary/m/m/ Glossary/m/m.md glossary/m.md letter index +https://tarf.co.uk/Glossary/m/M2M/ Glossary/m/M2M.md glossary/m.md#m2m +https://tarf.co.uk/Glossary/m/MAC%20Address/ Glossary/m/MAC Address.md glossary/m.md#mac-address +https://tarf.co.uk/Glossary/m/MAC%20Flooding/ Glossary/m/MAC Flooding.md glossary/m.md#mac-flooding +https://tarf.co.uk/Glossary/m/MAC%20Spoofing/ Glossary/m/MAC Spoofing.md glossary/m.md#mac-spoofing +https://tarf.co.uk/Glossary/m/MAC/ Glossary/m/MAC.md glossary/m.md#mac +https://tarf.co.uk/Glossary/m/Machine%20code/ Glossary/m/Machine code.md glossary/m.md#machine-code +https://tarf.co.uk/Glossary/m/Macro%20Estimation/ Glossary/m/Macro Estimation.md glossary/m.md#macro-estimation +https://tarf.co.uk/Glossary/m/MAG/ Glossary/m/MAG.md glossary/m.md dropped +https://tarf.co.uk/Glossary/m/Maintenance%20Plan/ Glossary/m/Maintenance Plan.md glossary/m.md#maintenance-plan +https://tarf.co.uk/Glossary/m/Maintenance/ Glossary/m/Maintenance.md glossary/m.md#maintenance +https://tarf.co.uk/Glossary/m/MAN/ Glossary/m/MAN.md glossary/m.md#man +https://tarf.co.uk/Glossary/m/Management%20by%20Walking%20Around/ Glossary/m/Management by Walking Around.md glossary/m.md#management-by-walking-around +https://tarf.co.uk/Glossary/m/Management%20Plane/ Glossary/m/Management Plane.md glossary/m.md#management-plane +https://tarf.co.uk/Glossary/m/MANET/ Glossary/m/MANET.md glossary/m.md#manet +https://tarf.co.uk/Glossary/m/Manual%20Testing/ Glossary/m/Manual Testing.md glossary/m.md#manual-testing +https://tarf.co.uk/Glossary/m/MAP/ Glossary/m/MAP.md glossary/m.md#map +https://tarf.co.uk/Glossary/m/MAS/ Glossary/m/MAS.md glossary/m.md dropped +https://tarf.co.uk/Glossary/m/Material/ Glossary/m/Material.md glossary/m.md dropped +https://tarf.co.uk/Glossary/m/MBMS/ Glossary/m/MBMS.md glossary/m.md#mbms +https://tarf.co.uk/Glossary/m/MBORC/ Glossary/m/MBORC.md glossary/m.md#mborc +https://tarf.co.uk/Glossary/m/MD5/ Glossary/m/MD5.md glossary/m.md#md5 +https://tarf.co.uk/Glossary/m/MDF/ Glossary/m/MDF.md glossary/m.md#mdf +https://tarf.co.uk/Glossary/m/Mediation/ Glossary/m/Mediation.md glossary/m.md#mediation +https://tarf.co.uk/Glossary/m/MELP/ Glossary/m/MELP.md glossary/m.md#melp +https://tarf.co.uk/Glossary/m/Metric/ Glossary/m/Metric.md glossary/m.md#metric +https://tarf.co.uk/Glossary/m/MGCF/ Glossary/m/MGCF.md glossary/m.md#mgcf +https://tarf.co.uk/Glossary/m/MGCP/ Glossary/m/MGCP.md glossary/m.md#mgcp +https://tarf.co.uk/Glossary/m/MGW/ Glossary/m/MGW.md glossary/m.md#mgw +https://tarf.co.uk/Glossary/m/MIB/ Glossary/m/MIB.md glossary/m.md#mib +https://tarf.co.uk/Glossary/m/Micro%20Estimation/ Glossary/m/Micro Estimation.md glossary/m.md#micro-estimation +https://tarf.co.uk/Glossary/m/Microservice/ Glossary/m/Microservice.md glossary/m.md#microservice +https://tarf.co.uk/Glossary/m/Microstone/ Glossary/m/Microstone.md glossary/m.md#microstone +https://tarf.co.uk/Glossary/m/Mil-COTS/ Glossary/m/Mil-COTS.md glossary/m.md#mil-cots +https://tarf.co.uk/Glossary/m/Milestone%20Release/ Glossary/m/Milestone Release.md glossary/m.md#milestone-release +https://tarf.co.uk/Glossary/m/Milestone%20Retrospective/ Glossary/m/Milestone Retrospective.md glossary/m.md#milestone-retrospective +https://tarf.co.uk/Glossary/m/Milestone%20Schedule/ Glossary/m/Milestone Schedule.md glossary/m.md#milestone-schedule +https://tarf.co.uk/Glossary/m/Milestone/ Glossary/m/Milestone.md glossary/m.md#milestone +https://tarf.co.uk/Glossary/m/MIMO/ Glossary/m/MIMO.md glossary/m.md#mimo +https://tarf.co.uk/Glossary/m/Miniature%20Milestone/ Glossary/m/Miniature Milestone.md glossary/m.md#miniature-milestone +https://tarf.co.uk/Glossary/m/MME/ Glossary/m/MME.md glossary/m.md#mme +https://tarf.co.uk/Glossary/m/MMF/ Glossary/m/MMF.md glossary/m.md#mmf +https://tarf.co.uk/Glossary/m/MNC/ Glossary/m/MNC.md glossary/m.md#mnc +https://tarf.co.uk/Glossary/m/Mob%20Programming/ Glossary/m/Mob Programming.md glossary/m.md#mob-programming +https://tarf.co.uk/Glossary/m/Mobile%20Cell/ Glossary/m/Mobile Cell.md glossary/m.md#mobile-cell +https://tarf.co.uk/Glossary/m/Mobile%20Networks%202G-3G/ Glossary/m/Mobile Networks 2G-3G.md glossary/u.md#umts renamed to "UMTS" +https://tarf.co.uk/Glossary/m/Mock%20Objects/ Glossary/m/Mock Objects.md glossary/m.md#mock-objects +https://tarf.co.uk/Glossary/m/Model/ Glossary/m/Model.md glossary/m.md#model +https://tarf.co.uk/Glossary/m/Modeling/ Glossary/m/Modeling.md glossary/m.md#modelling renamed to "Modelling" +https://tarf.co.uk/Glossary/m/Moderator/ Glossary/m/Moderator.md glossary/m.md#moderator +https://tarf.co.uk/Glossary/m/MOS/ Glossary/m/MOS.md glossary/m.md#mos +https://tarf.co.uk/Glossary/m/MP3/ Glossary/m/MP3.md glossary/m.md#mp3 +https://tarf.co.uk/Glossary/m/MPEG/ Glossary/m/MPEG.md glossary/m.md#mpeg +https://tarf.co.uk/Glossary/m/MPEG4/ Glossary/m/MPEG4.md glossary/m.md#mpeg4 +https://tarf.co.uk/Glossary/m/MPF/ Glossary/m/MPF.md glossary/m.md#mpf +https://tarf.co.uk/Glossary/m/MPS/ Glossary/m/MPS.md glossary/m.md dropped +https://tarf.co.uk/Glossary/m/MQTT/ Glossary/m/MQTT.md glossary/m.md#mqtt +https://tarf.co.uk/Glossary/m/MRF/ Glossary/m/MRF.md glossary/m.md#mrf +https://tarf.co.uk/Glossary/m/MRS/ Glossary/m/MRS.md glossary/m.md#mrs +https://tarf.co.uk/Glossary/m/MRTIE/ Glossary/m/MRTIE.md glossary/m.md#mrtie +https://tarf.co.uk/Glossary/m/MSAN/ Glossary/m/MSAN.md glossary/m.md#msan +https://tarf.co.uk/Glossary/m/MSC/ Glossary/m/MSC.md glossary/m.md#msc +https://tarf.co.uk/Glossary/m/MTP/ Glossary/m/MTP.md glossary/m.md#mtp +https://tarf.co.uk/Glossary/m/MTR/ Glossary/m/MTR.md glossary/m.md#mtr +https://tarf.co.uk/Glossary/m/MTU/ Glossary/m/MTU.md glossary/m.md#mtu +https://tarf.co.uk/Glossary/m/Multicast/ Glossary/m/Multicast.md glossary/m.md#multicast +https://tarf.co.uk/Glossary/m/Multiplex/ Glossary/m/Multiplex.md glossary/m.md#multiplex +https://tarf.co.uk/Glossary/m/Mutex/ Glossary/m/Mutex.md glossary/m.md#mutex +https://tarf.co.uk/Glossary/m/Mutual%20Exclusion/ Glossary/m/Mutual Exclusion.md glossary/m.md#mutual-exclusion +https://tarf.co.uk/Glossary/m/MVNE/ Glossary/m/MVNE.md glossary/m.md#mvne +https://tarf.co.uk/Glossary/m/MVNO/ Glossary/m/MVNO.md glossary/m.md#mvno +https://tarf.co.uk/Glossary/m/MVP/ Glossary/m/MVP.md glossary/m.md#mvp +https://tarf.co.uk/Glossary/n/n/ Glossary/n/n.md glossary/n.md letter index +https://tarf.co.uk/Glossary/n/NaaS/ Glossary/n/NaaS.md glossary/n.md#naas +https://tarf.co.uk/Glossary/n/NAPT/ Glossary/n/NAPT.md glossary/n.md#napt +https://tarf.co.uk/Glossary/n/NAS/ Glossary/n/NAS.md glossary/n.md#nas +https://tarf.co.uk/Glossary/n/NAT/ Glossary/n/NAT.md glossary/n.md#nat +https://tarf.co.uk/Glossary/n/Native%20Risk%20Management/ Glossary/n/Native Risk Management.md glossary/n.md#native-risk-management +https://tarf.co.uk/Glossary/n/NBAP/ Glossary/n/NBAP.md glossary/n.md#nbap +https://tarf.co.uk/Glossary/n/NCSC/ Glossary/n/NCSC.md glossary/n.md#ncsc +https://tarf.co.uk/Glossary/n/Network%20Topologies/ Glossary/n/Network Topologies.md glossary/n.md#network-topologies +https://tarf.co.uk/Glossary/n/NFS/ Glossary/n/NFS.md glossary/n.md#nfs +https://tarf.co.uk/Glossary/n/NGN/ Glossary/n/NGN.md glossary/n.md#ngn +https://tarf.co.uk/Glossary/n/NGWFQ/ Glossary/n/NGWFQ.md glossary/n.md#ngwfq +https://tarf.co.uk/Glossary/n/Niko-niko%20Calendar/ Glossary/n/Niko-niko Calendar.md glossary/n.md#niko-niko-calendar +https://tarf.co.uk/Glossary/n/NIST/ Glossary/n/NIST.md glossary/n.md#nist +https://tarf.co.uk/Glossary/n/NNI/ Glossary/n/NNI.md glossary/n.md#nni +https://tarf.co.uk/Glossary/n/NNTP/ Glossary/n/NNTP.md glossary/n.md#nntp +https://tarf.co.uk/Glossary/n/NOC/ Glossary/n/NOC.md glossary/n.md#noc +https://tarf.co.uk/Glossary/n/Node%20B/ Glossary/n/Node B.md glossary/n.md#node-b +https://tarf.co.uk/Glossary/n/Nominal%20Path/ Glossary/n/Nominal Path.md glossary/n.md#nominal-path +https://tarf.co.uk/Glossary/n/Non-functional%20Requirement/ Glossary/n/Non-functional Requirement.md glossary/n.md#non-functional-requirement +https://tarf.co.uk/Glossary/n/NoSQL/ Glossary/n/NoSQL.md glossary/n.md#nosql +https://tarf.co.uk/Glossary/n/NPE/ Glossary/n/NPE.md glossary/n.md#npe +https://tarf.co.uk/Glossary/n/NSS/ Glossary/n/NSS.md glossary/n.md#nss +https://tarf.co.uk/Glossary/n/NTP/ Glossary/n/NTP.md glossary/n.md#ntp +https://tarf.co.uk/Glossary/n/NTS/ Glossary/n/NTS.md glossary/n.md#nts +https://tarf.co.uk/Glossary/n/NTU/ Glossary/n/NTU.md glossary/n.md#ntu +https://tarf.co.uk/Glossary/n/NYB/ Glossary/n/NYB.md glossary/n.md#nyb +https://tarf.co.uk/Glossary/o/o/ Glossary/o/o.md glossary/o.md letter index +https://tarf.co.uk/Glossary/o/Objective%20C/ Glossary/o/Objective C.md glossary/o.md#objective-c +https://tarf.co.uk/Glossary/o/Objectives/ Glossary/o/Objectives.md glossary/o.md#objectives +https://tarf.co.uk/Glossary/o/ODF/ Glossary/o/ODF.md glossary/o.md#odf +https://tarf.co.uk/Glossary/o/OFC/ Glossary/o/OFC.md glossary/o.md#ofc +https://tarf.co.uk/Glossary/o/Ofcom/ Glossary/o/Ofcom.md glossary/o.md#ofcom +https://tarf.co.uk/Glossary/o/OFDMA/ Glossary/o/OFDMA.md glossary/o.md#ofdma +https://tarf.co.uk/Glossary/o/OO/ Glossary/o/OO.md glossary/o.md#oo +https://tarf.co.uk/Glossary/o/Operations%20Plan/ Glossary/o/Operations Plan.md glossary/o.md#operations-plan +https://tarf.co.uk/Glossary/o/OPUS/ Glossary/o/OPUS.md glossary/o.md#opus +https://tarf.co.uk/Glossary/o/OS%20X/ Glossary/o/OS X.md glossary/o.md#os-x +https://tarf.co.uk/Glossary/o/OSA-FC/ Glossary/o/OSA-FC.md glossary/o.md#osa-fc +https://tarf.co.uk/Glossary/o/OSPF/ Glossary/o/OSPF.md glossary/o.md#ospf +https://tarf.co.uk/Glossary/o/OSS/ Glossary/o/OSS.md glossary/o.md#oss +https://tarf.co.uk/Glossary/o/OTN/ Glossary/o/OTN.md glossary/o.md#otn +https://tarf.co.uk/Glossary/o/Out-of-order%20delivery/ Glossary/o/Out-of-order delivery.md glossary/o.md#out-of-order-delivery +https://tarf.co.uk/Glossary/o/Overlap%20Dialling/ Glossary/o/Overlap Dialling.md glossary/o.md#overlap-dialling +https://tarf.co.uk/Glossary/p/p/ Glossary/p/p.md glossary/p.md letter index +https://tarf.co.uk/Glossary/p/PaaS/ Glossary/p/PaaS.md glossary/p.md#paas +https://tarf.co.uk/Glossary/p/packet%20sniffing/ Glossary/p/packet sniffing.md glossary/p.md#packet-sniffing +https://tarf.co.uk/Glossary/p/Packet%20Switched/ Glossary/p/Packet Switched.md glossary/p.md#packet-switched +https://tarf.co.uk/Glossary/p/PAF/ Glossary/p/PAF.md glossary/p.md#paf +https://tarf.co.uk/Glossary/p/Pair%20Programming/ Glossary/p/Pair Programming.md glossary/p.md#pair-programming +https://tarf.co.uk/Glossary/p/PAP/ Glossary/p/PAP.md glossary/p.md#pap +https://tarf.co.uk/Glossary/p/Parametric%20Estimation/ Glossary/p/Parametric Estimation.md glossary/p.md#parametric-estimation +https://tarf.co.uk/Glossary/p/Pascal/ Glossary/p/Pascal.md glossary/p.md#pascal +https://tarf.co.uk/Glossary/p/Pattern/ Glossary/p/Pattern.md glossary/p.md#pattern +https://tarf.co.uk/Glossary/p/PCM/ Glossary/p/PCM.md glossary/p.md#pcm +https://tarf.co.uk/Glossary/p/PCP/ Glossary/p/PCP.md glossary/p.md#pcp +https://tarf.co.uk/Glossary/p/PCRF/ Glossary/p/PCRF.md glossary/p.md#pcrf +https://tarf.co.uk/Glossary/p/PCS/ Glossary/p/PCS.md glossary/p.md#pcs +https://tarf.co.uk/Glossary/p/PDCP/ Glossary/p/PDCP.md glossary/p.md#pdcp +https://tarf.co.uk/Glossary/p/PDH/ Glossary/p/PDH.md glossary/p.md#pdh +https://tarf.co.uk/Glossary/p/PDU/ Glossary/p/PDU.md glossary/p.md#pdu +https://tarf.co.uk/Glossary/p/PEAP/ Glossary/p/PEAP.md glossary/p.md#peap +https://tarf.co.uk/Glossary/p/peer%20review/ Glossary/p/peer review.md glossary/p.md#peer-review +https://tarf.co.uk/Glossary/p/Pen%20testing/ Glossary/p/Pen testing.md glossary/p.md#pen-testing +https://tarf.co.uk/Glossary/p/PER/ Glossary/p/PER.md glossary/p.md#per +https://tarf.co.uk/Glossary/p/Performance/ Glossary/p/Performance.md glossary/p.md#performance +https://tarf.co.uk/Glossary/p/Perl/ Glossary/p/Perl.md glossary/p.md#perl +https://tarf.co.uk/Glossary/p/Personas/ Glossary/p/Personas.md glossary/p.md#personas +https://tarf.co.uk/Glossary/p/Perspective/ Glossary/p/Perspective.md glossary/p.md#perspective +https://tarf.co.uk/Glossary/p/PERT%20chart/ Glossary/p/PERT chart.md glossary/p.md#pert-chart +https://tarf.co.uk/Glossary/p/PGP/ Glossary/p/PGP.md glossary/p.md#pgp +https://tarf.co.uk/Glossary/p/PGW/ Glossary/p/PGW.md glossary/p.md#pgw +https://tarf.co.uk/Glossary/p/Phase/ Glossary/p/Phase.md glossary/p.md#phase +https://tarf.co.uk/Glossary/p/Phased%20Estimation/ Glossary/p/Phased Estimation.md glossary/p.md#phased-estimation +https://tarf.co.uk/Glossary/p/PHP/ Glossary/p/PHP.md glossary/p.md#php +https://tarf.co.uk/Glossary/p/Planning%20and%20Tracking%20Lead/ Glossary/p/Planning and Tracking Lead.md glossary/p.md#planning-and-tracking-lead +https://tarf.co.uk/Glossary/p/Planning%20Poker/ Glossary/p/Planning Poker.md glossary/p.md#planning-poker +https://tarf.co.uk/Glossary/p/PLL/ Glossary/p/PLL.md glossary/p.md#pll +https://tarf.co.uk/Glossary/p/PNT/ Glossary/p/PNT.md glossary/p.md#pnt +https://tarf.co.uk/Glossary/p/Points/ Glossary/p/Points.md glossary/p.md#points +https://tarf.co.uk/Glossary/p/polymorphism/ Glossary/p/polymorphism.md glossary/p.md#polymorphism +https://tarf.co.uk/Glossary/p/POP/ Glossary/p/POP.md glossary/p.md#pop +https://tarf.co.uk/Glossary/p/portability/ Glossary/p/portability.md glossary/p.md#portability +https://tarf.co.uk/Glossary/p/POSIX/ Glossary/p/POSIX.md glossary/p.md#posix +https://tarf.co.uk/Glossary/p/Postmortem/ Glossary/p/Postmortem.md glossary/p.md#postmortem +https://tarf.co.uk/Glossary/p/POTS/ Glossary/p/POTS.md glossary/p.md#pots +https://tarf.co.uk/Glossary/p/PPP/ Glossary/p/PPP.md glossary/p.md#ppp +https://tarf.co.uk/Glossary/p/PPPoE/ Glossary/p/PPPoE.md glossary/p.md#pppoe +https://tarf.co.uk/Glossary/p/PPTP/ Glossary/p/PPTP.md glossary/p.md#pptp +https://tarf.co.uk/Glossary/p/Preemption/ Glossary/p/Preemption.md glossary/p.md#preemption +https://tarf.co.uk/Glossary/p/Preview/ Glossary/p/Preview.md glossary/p.md#preview +https://tarf.co.uk/Glossary/p/PRINCE2/ Glossary/p/PRINCE2.md glossary/p.md#prince2 +https://tarf.co.uk/Glossary/p/Process%20Flow/ Glossary/p/Process Flow.md glossary/p.md#process-flow +https://tarf.co.uk/Glossary/p/Process/ Glossary/p/Process.md glossary/p.md#process +https://tarf.co.uk/Glossary/p/Product%20Owner/ Glossary/p/Product Owner.md glossary/p.md#product-owner +https://tarf.co.uk/Glossary/p/Product%20Release/ Glossary/p/Product Release.md glossary/p.md#product-release +https://tarf.co.uk/Glossary/p/programming%20language/ Glossary/p/programming language.md glossary/p.md#programming-language +https://tarf.co.uk/Glossary/p/Programming%20paradigm/ Glossary/p/Programming paradigm.md glossary/p.md#programming-paradigm +https://tarf.co.uk/Glossary/p/Progressive%20Elaboration/ Glossary/p/Progressive Elaboration.md glossary/p.md#progressive-elaboration +https://tarf.co.uk/Glossary/p/Project%20Build/ Glossary/p/Project Build.md glossary/p.md#project-build +https://tarf.co.uk/Glossary/p/Project%20Business%20Manager/ Glossary/p/Project Business Manager.md glossary/p.md#project-business-manager +https://tarf.co.uk/Glossary/p/Project%20Charter/ Glossary/p/Project Charter.md glossary/p.md#project-charter +https://tarf.co.uk/Glossary/p/Project%20Chartering/ Glossary/p/Project Chartering.md glossary/p.md#project-chartering +https://tarf.co.uk/Glossary/p/Project%20Estimate/ Glossary/p/Project Estimate.md glossary/p.md#project-estimate +https://tarf.co.uk/Glossary/p/Project%20Headlights/ Glossary/p/Project Headlights.md glossary/p.md#project-headlights +https://tarf.co.uk/Glossary/p/Project%20History/ Glossary/p/Project History.md glossary/p.md#project-history +https://tarf.co.uk/Glossary/p/Project%20Log/ Glossary/p/Project Log.md glossary/p.md#project-log +https://tarf.co.uk/Glossary/p/Project%20Management%20Body%20of%20Knowledge/ Glossary/p/Project Management Body of Knowledge.md glossary/p.md#project-management-body-of-knowledge +https://tarf.co.uk/Glossary/p/Project%20Management%20Institute/ Glossary/p/Project Management Institute.md glossary/p.md#project-management-institute +https://tarf.co.uk/Glossary/p/Project%20Management/ Glossary/p/Project Management.md glossary/p.md#project-management +https://tarf.co.uk/Glossary/p/Project%20Manager/ Glossary/p/Project Manager.md glossary/p.md#project-manager +https://tarf.co.uk/Glossary/p/Project%20Plan/ Glossary/p/Project Plan.md glossary/p.md#project-plan +https://tarf.co.uk/Glossary/p/Project%20Reviewer/ Glossary/p/Project Reviewer.md glossary/p.md#project-reviewer +https://tarf.co.uk/Glossary/p/Project%20Sponsor/ Glossary/p/Project Sponsor.md glossary/p.md#project-sponsor +https://tarf.co.uk/Glossary/p/Project/ Glossary/p/Project.md glossary/p.md#project +https://tarf.co.uk/Glossary/p/pseudocode/ Glossary/p/pseudocode.md glossary/p.md#pseudocode +https://tarf.co.uk/Glossary/p/PSTN/ Glossary/p/PSTN.md glossary/p.md#pstn +https://tarf.co.uk/Glossary/p/PTP/ Glossary/p/PTP.md glossary/p.md#ptp +https://tarf.co.uk/Glossary/p/PTT/ Glossary/p/PTT.md glossary/p.md#ptt +https://tarf.co.uk/Glossary/p/PVC/ Glossary/p/PVC.md glossary/p.md#pvc +https://tarf.co.uk/Glossary/p/PW/ Glossary/p/PW.md glossary/p.md#pw +https://tarf.co.uk/Glossary/p/PWFQ/ Glossary/p/PWFQ.md glossary/p.md#pwfq +https://tarf.co.uk/Glossary/p/Python%20%282.x%20%26%203.x%29/ Glossary/p/Python (2.x & 3.x).md glossary/p.md#python renamed to "Python" +https://tarf.co.uk/Glossary/q/Q.931/ Glossary/q/Q.931.md glossary/q.md#q931 +https://tarf.co.uk/Glossary/q/q/ Glossary/q/q.md glossary/q.md letter index +https://tarf.co.uk/Glossary/q/QA%20Lead/ Glossary/q/QA Lead.md glossary/q.md#qa-lead +https://tarf.co.uk/Glossary/q/QAM/ Glossary/q/QAM.md glossary/q.md#qam +https://tarf.co.uk/Glossary/q/QoS/ Glossary/q/QoS.md glossary/q.md#qos +https://tarf.co.uk/Glossary/q/QSIG/ Glossary/q/QSIG.md glossary/q.md#qsig +https://tarf.co.uk/Glossary/q/Quality%20Assurance%20Plan/ Glossary/q/Quality Assurance Plan.md glossary/q.md#quality-assurance-plan +https://tarf.co.uk/Glossary/q/Quality%20Assurance/ Glossary/q/Quality Assurance.md glossary/q.md#quality-assurance +https://tarf.co.uk/Glossary/q/Quality%20Control/ Glossary/q/Quality Control.md glossary/q.md#quality-control +https://tarf.co.uk/Glossary/q/Quality%20Plan/ Glossary/q/Quality Plan.md glossary/q.md#quality-plan +https://tarf.co.uk/Glossary/q/Quality/ Glossary/q/Quality.md glossary/q.md#quality +https://tarf.co.uk/Glossary/q/Quick%20Design%20Session/ Glossary/q/Quick Design Session.md glossary/q.md#quick-design-session +https://tarf.co.uk/Glossary/r/R%20Language/ Glossary/r/R Language.md glossary/r.md#r-language +https://tarf.co.uk/Glossary/r/r/ Glossary/r/r.md glossary/r.md letter index +https://tarf.co.uk/Glossary/r/RACI/ Glossary/r/RACI.md glossary/r.md#raci +https://tarf.co.uk/Glossary/r/RAD%20Environment/ Glossary/r/RAD Environment.md glossary/r.md#rad-environment +https://tarf.co.uk/Glossary/r/RAD/ Glossary/r/RAD.md glossary/r.md#rad +https://tarf.co.uk/Glossary/r/Radius/ Glossary/r/Radius.md glossary/r.md#radius +https://tarf.co.uk/Glossary/r/Rainbow/ Glossary/r/Rainbow.md glossary/r.md dropped +https://tarf.co.uk/Glossary/r/RAN/ Glossary/r/RAN.md glossary/r.md#ran +https://tarf.co.uk/Glossary/r/RANAP/ Glossary/r/RANAP.md glossary/r.md#ranap +https://tarf.co.uk/Glossary/r/RANCID/ Glossary/r/RANCID.md glossary/r.md#rancid +https://tarf.co.uk/Glossary/r/Rapid%20Application%20Development/ Glossary/r/Rapid Application Development.md glossary/r.md#rapid-application-development +https://tarf.co.uk/Glossary/r/RARP/ Glossary/r/RARP.md glossary/r.md#rarp +https://tarf.co.uk/Glossary/r/RAS/ Glossary/r/RAS.md glossary/r.md#ras +https://tarf.co.uk/Glossary/r/RDBMS/ Glossary/r/RDBMS.md glossary/r.md#rdbms +https://tarf.co.uk/Glossary/r/RDP/ Glossary/r/RDP.md glossary/r.md#rdp +https://tarf.co.uk/Glossary/r/Reading%20Inspection/ Glossary/r/Reading Inspection.md glossary/r.md#reading-inspection +https://tarf.co.uk/Glossary/r/refactoring/ Glossary/r/refactoring.md glossary/r.md#refactoring +https://tarf.co.uk/Glossary/r/Reference/ Glossary/r/Reference.md glossary/r.md dropped +https://tarf.co.uk/Glossary/r/regression%20testing/ Glossary/r/regression testing.md glossary/r.md#regression-testing +https://tarf.co.uk/Glossary/r/Regular%20expression/ Glossary/r/Regular expression.md glossary/r.md#regular-expression +https://tarf.co.uk/Glossary/r/Relative%20Estimation/ Glossary/r/Relative Estimation.md glossary/r.md#relative-estimation +https://tarf.co.uk/Glossary/r/Release%20Management/ Glossary/r/Release Management.md glossary/r.md#release-management +https://tarf.co.uk/Glossary/r/Release%20Test/ Glossary/r/Release Test.md glossary/r.md#release-test +https://tarf.co.uk/Glossary/r/Release/ Glossary/r/Release.md glossary/r.md#release +https://tarf.co.uk/Glossary/r/Requirement/ Glossary/r/Requirement.md glossary/r.md#requirement +https://tarf.co.uk/Glossary/r/Requirements%20Lead/ Glossary/r/Requirements Lead.md glossary/r.md#requirements-lead +https://tarf.co.uk/Glossary/r/Requirements/ Glossary/r/Requirements.md glossary/r.md#requirements +https://tarf.co.uk/Glossary/r/REST/ Glossary/r/REST.md glossary/r.md#rest +https://tarf.co.uk/Glossary/r/Review/ Glossary/r/Review.md glossary/r.md#review +https://tarf.co.uk/Glossary/r/Reviewer/ Glossary/r/Reviewer.md glossary/r.md#reviewer +https://tarf.co.uk/Glossary/r/Revision%20Control/ Glossary/r/Revision Control.md glossary/r.md#revision-control +https://tarf.co.uk/Glossary/r/Revision/ Glossary/r/Revision.md glossary/r.md#revision +https://tarf.co.uk/Glossary/r/Rework/ Glossary/r/Rework.md glossary/r.md#rework +https://tarf.co.uk/Glossary/r/RFC/ Glossary/r/RFC.md glossary/r.md#rfc +https://tarf.co.uk/Glossary/r/RIP/ Glossary/r/RIP.md glossary/r.md#rip +https://tarf.co.uk/Glossary/r/Risk%20Analysis/ Glossary/r/Risk Analysis.md glossary/r.md#risk-analysis +https://tarf.co.uk/Glossary/r/Risk%20Identification/ Glossary/r/Risk Identification.md glossary/r.md#risk-identification +https://tarf.co.uk/Glossary/r/Risk%20List/ Glossary/r/Risk List.md glossary/r.md#risk-list +https://tarf.co.uk/Glossary/r/Risk%20Management/ Glossary/r/Risk Management.md glossary/r.md#risk-management +https://tarf.co.uk/Glossary/r/Risk%20Mitigation/ Glossary/r/Risk Mitigation.md glossary/r.md#risk-mitigation +https://tarf.co.uk/Glossary/r/Risk%20Reward%20Matric/ Glossary/r/Risk Reward Matric.md glossary/r.md#risk-reward-matrix renamed to "Risk Reward Matrix" +https://tarf.co.uk/Glossary/r/Risk/ Glossary/r/Risk.md glossary/r.md#risk +https://tarf.co.uk/Glossary/r/RNC/ Glossary/r/RNC.md glossary/r.md#rnc +https://tarf.co.uk/Glossary/r/RNSAP/ Glossary/r/RNSAP.md glossary/r.md#rnsap +https://tarf.co.uk/Glossary/r/Role-feature-reason/ Glossary/r/Role-feature-reason.md glossary/r.md#role-feature-reason +https://tarf.co.uk/Glossary/r/Rolling%20Wave%20Planning/ Glossary/r/Rolling Wave Planning.md glossary/r.md#rolling-wave-planning +https://tarf.co.uk/Glossary/r/Round%20Robin/ Glossary/r/Round Robin.md glossary/r.md#round-robin +https://tarf.co.uk/Glossary/r/Router/ Glossary/r/Router.md glossary/r.md#router +https://tarf.co.uk/Glossary/r/RPC/ Glossary/r/RPC.md glossary/r.md#rpc +https://tarf.co.uk/Glossary/r/RRC/ Glossary/r/RRC.md glossary/r.md#rrc +https://tarf.co.uk/Glossary/r/RSTP/ Glossary/r/RSTP.md glossary/r.md#rstp +https://tarf.co.uk/Glossary/r/RSVP/ Glossary/r/RSVP.md glossary/r.md#rsvp +https://tarf.co.uk/Glossary/r/RTCP/ Glossary/r/RTCP.md glossary/r.md#rtcp +https://tarf.co.uk/Glossary/r/RTOS/ Glossary/r/RTOS.md glossary/r.md#rtos +https://tarf.co.uk/Glossary/r/RTP/ Glossary/r/RTP.md glossary/r.md#rtp +https://tarf.co.uk/Glossary/r/RTSP/ Glossary/r/RTSP.md glossary/r.md#rtsp +https://tarf.co.uk/Glossary/r/Ruby%20on%20rails/ Glossary/r/Ruby on rails.md glossary/r.md#ruby-on-rails +https://tarf.co.uk/Glossary/r/Ruby/ Glossary/r/Ruby.md glossary/r.md#ruby +https://tarf.co.uk/Glossary/r/Rule%20of%20Simplicity/ Glossary/r/Rule of Simplicity.md glossary/r.md#rule-of-simplicity +https://tarf.co.uk/Glossary/r/RUP/ Glossary/r/RUP.md glossary/r.md#rup +https://tarf.co.uk/Glossary/s/S-%20Tag/ Glossary/s/S- Tag.md glossary/s.md#s-tag renamed to "S-Tag" +https://tarf.co.uk/Glossary/s/S-GW/ Glossary/s/S-GW.md glossary/s.md#s-gw +https://tarf.co.uk/Glossary/s/s/ Glossary/s/s.md glossary/s.md letter index +https://tarf.co.uk/Glossary/s/SAAL/ Glossary/s/SAAL.md glossary/s.md#saal +https://tarf.co.uk/Glossary/s/SaaS/ Glossary/s/SaaS.md glossary/s.md#saas +https://tarf.co.uk/Glossary/s/SAE/ Glossary/s/SAE.md glossary/s.md#sae +https://tarf.co.uk/Glossary/s/SAFe/ Glossary/s/SAFe.md glossary/s.md#safe +https://tarf.co.uk/Glossary/s/SBC/ Glossary/s/SBC.md glossary/s.md#sbc +https://tarf.co.uk/Glossary/s/SC-FDMA/ Glossary/s/SC-FDMA.md glossary/s.md#sc-fdma +https://tarf.co.uk/Glossary/s/Scala/ Glossary/s/Scala.md glossary/s.md#scala +https://tarf.co.uk/Glossary/s/Scaled%20Agile/ Glossary/s/Scaled Agile.md glossary/s.md#scaled-agile +https://tarf.co.uk/Glossary/s/SCCP/ Glossary/s/SCCP.md glossary/s.md#sccp +https://tarf.co.uk/Glossary/s/Scenario/ Glossary/s/Scenario.md glossary/s.md#scenario +https://tarf.co.uk/Glossary/s/Schedule/ Glossary/s/Schedule.md glossary/s.md#schedule +https://tarf.co.uk/Glossary/s/Scheduler/ Glossary/s/Scheduler.md glossary/s.md#scheduler +https://tarf.co.uk/Glossary/s/SCM/ Glossary/s/SCM.md glossary/s.md#scm +https://tarf.co.uk/Glossary/s/Scratch/ Glossary/s/Scratch.md glossary/s.md#scratch +https://tarf.co.uk/Glossary/s/Scribe/ Glossary/s/Scribe.md glossary/s.md#scribe +https://tarf.co.uk/Glossary/s/Scrum%20Master/ Glossary/s/Scrum Master.md glossary/s.md#scrum-master +https://tarf.co.uk/Glossary/s/Scrum%20of%20Scrums/ Glossary/s/Scrum of Scrums.md glossary/s.md#scrum-of-scrums +https://tarf.co.uk/Glossary/s/Scrum/ Glossary/s/Scrum.md glossary/s.md#scrum +https://tarf.co.uk/Glossary/s/SCTP/ Glossary/s/SCTP.md glossary/s.md#sctp +https://tarf.co.uk/Glossary/s/SDES/ Glossary/s/SDES.md glossary/s.md#sdes +https://tarf.co.uk/Glossary/s/SDH/ Glossary/s/SDH.md glossary/s.md#sdh +https://tarf.co.uk/Glossary/s/SDK/ Glossary/s/SDK.md glossary/s.md#sdk +https://tarf.co.uk/Glossary/s/SDLC/ Glossary/s/SDLC.md glossary/s.md#sdlc +https://tarf.co.uk/Glossary/s/SDMA/ Glossary/s/SDMA.md glossary/s.md#sdma +https://tarf.co.uk/Glossary/s/SDN/ Glossary/s/SDN.md glossary/s.md#sdn +https://tarf.co.uk/Glossary/s/SDP/ Glossary/s/SDP.md glossary/s.md#sdp +https://tarf.co.uk/Glossary/s/SDSL/ Glossary/s/SDSL.md glossary/s.md#sdsl +https://tarf.co.uk/Glossary/s/Sed/ Glossary/s/Sed.md glossary/s.md#sed +https://tarf.co.uk/Glossary/s/Semaphore/ Glossary/s/Semaphore.md glossary/s.md#semaphore +https://tarf.co.uk/Glossary/s/Sematic%20Versioning/ Glossary/s/Sematic Versioning.md glossary/s.md#semantic-versioning renamed to "Semantic Versioning" +https://tarf.co.uk/Glossary/s/Sequence%20Model/ Glossary/s/Sequence Model.md glossary/s.md#sequence-model +https://tarf.co.uk/Glossary/s/SF/ Glossary/s/SF.md glossary/s.md#sf +https://tarf.co.uk/Glossary/s/SFTP/ Glossary/s/SFTP.md glossary/s.md#sftp +https://tarf.co.uk/Glossary/s/SGML/ Glossary/s/SGML.md glossary/s.md#sgml +https://tarf.co.uk/Glossary/s/SGSN/ Glossary/s/SGSN.md glossary/s.md#sgsn +https://tarf.co.uk/Glossary/s/SGW/ Glossary/s/SGW.md glossary/s.md#sgw +https://tarf.co.uk/Glossary/s/Sh/ Glossary/s/Sh.md glossary/s.md#sh +https://tarf.co.uk/Glossary/s/SHA/ Glossary/s/SHA.md glossary/s.md#sha +https://tarf.co.uk/Glossary/s/Shell%20Scripting/ Glossary/s/Shell Scripting.md glossary/s.md#shell-scripting +https://tarf.co.uk/Glossary/s/Sidetone/ Glossary/s/Sidetone.md glossary/s.md#sidetone +https://tarf.co.uk/Glossary/s/Sign%20Up/ Glossary/s/Sign Up.md glossary/s.md#sign-up +https://tarf.co.uk/Glossary/s/SIM/ Glossary/s/SIM.md glossary/s.md#sim +https://tarf.co.uk/Glossary/s/Simple%20Design/ Glossary/s/Simple Design.md glossary/s.md#simple-design +https://tarf.co.uk/Glossary/s/SIMPLE/ Glossary/s/SIMPLE.md glossary/s.md#simple +https://tarf.co.uk/Glossary/s/Simplex/ Glossary/s/Simplex.md glossary/s.md#simplex +https://tarf.co.uk/Glossary/s/Single%20Step%20Test/ Glossary/s/Single Step Test.md glossary/s.md#single-step-test +https://tarf.co.uk/Glossary/s/SIP%20Trunk/ Glossary/s/SIP Trunk.md glossary/s.md#sip-trunk +https://tarf.co.uk/Glossary/s/SIP/ Glossary/s/SIP.md glossary/s.md#sip +https://tarf.co.uk/Glossary/s/SISO/ Glossary/s/SISO.md glossary/s.md#siso +https://tarf.co.uk/Glossary/s/Skew/ Glossary/s/Skew.md glossary/s.md#skew +https://tarf.co.uk/Glossary/s/SLA/ Glossary/s/SLA.md glossary/s.md#sla +https://tarf.co.uk/Glossary/s/SLIP/ Glossary/s/SLIP.md glossary/s.md#slip +https://tarf.co.uk/Glossary/s/SME/ Glossary/s/SME.md glossary/s.md#sme +https://tarf.co.uk/Glossary/s/Smoke%20Test/ Glossary/s/Smoke Test.md glossary/s.md#smoke-test +https://tarf.co.uk/Glossary/s/smoke%20testing/ Glossary/s/smoke testing.md glossary/s.md#smoke-testing +https://tarf.co.uk/Glossary/s/SMP/ Glossary/s/SMP.md glossary/s.md#smp +https://tarf.co.uk/Glossary/s/SMPF/ Glossary/s/SMPF.md glossary/s.md#smpf +https://tarf.co.uk/Glossary/s/SMSC/ Glossary/s/SMSC.md glossary/s.md#smsc +https://tarf.co.uk/Glossary/s/Snake%20Case/ Glossary/s/Snake Case.md glossary/s.md#snake-case +https://tarf.co.uk/Glossary/s/SNMP/ Glossary/s/SNMP.md glossary/s.md#snmp +https://tarf.co.uk/Glossary/s/SNTP/ Glossary/s/SNTP.md glossary/s.md#sntp +https://tarf.co.uk/Glossary/s/SOA/ Glossary/s/SOA.md glossary/s.md#soa +https://tarf.co.uk/Glossary/s/SOAP/ Glossary/s/SOAP.md glossary/s.md#soap +https://tarf.co.uk/Glossary/s/Softswitch/ Glossary/s/Softswitch.md glossary/s.md#softswitch +https://tarf.co.uk/Glossary/s/Software%20Change%20Request/ Glossary/s/Software Change Request.md glossary/s.md#software-change-request +https://tarf.co.uk/Glossary/s/Software%20Configuration%20Management/ Glossary/s/Software Configuration Management.md glossary/s.md#software-configuration-management +https://tarf.co.uk/Glossary/s/Software%20Development%20Plan/ Glossary/s/Software Development Plan.md glossary/s.md#software-development-plan +https://tarf.co.uk/Glossary/s/Software%20Engineering%20Body%20of%20Knowledge/ Glossary/s/Software Engineering Body of Knowledge.md glossary/s.md#software-engineering-body-of-knowledge +https://tarf.co.uk/Glossary/s/Software%20Engineering%20Lab/ Glossary/s/Software Engineering Lab.md glossary/s.md#software-engineering-lab +https://tarf.co.uk/Glossary/s/Software%20Engineering%20Process%20Group/ Glossary/s/Software Engineering Process Group.md glossary/s.md#software-engineering-process-group +https://tarf.co.uk/Glossary/s/Software%20Requirements%20Specification/ Glossary/s/Software Requirements Specification.md glossary/s.md#software-requirements-specification +https://tarf.co.uk/Glossary/s/SoGEA/ Glossary/s/SoGEA.md glossary/s.md#sogea +https://tarf.co.uk/Glossary/s/SONET/ Glossary/s/SONET.md glossary/s.md#sonet +https://tarf.co.uk/Glossary/s/Source%20Code/ Glossary/s/Source Code.md glossary/s.md#source-code +https://tarf.co.uk/Glossary/s/SPEEX/ Glossary/s/SPEEX.md glossary/s.md#speex +https://tarf.co.uk/Glossary/s/SPI/ Glossary/s/SPI.md glossary/s.md#spi +https://tarf.co.uk/Glossary/s/Spiral%20Lifecycle/ Glossary/s/Spiral Lifecycle.md glossary/s.md#spiral-lifecycle +https://tarf.co.uk/Glossary/s/Sponsor/ Glossary/s/Sponsor.md glossary/s.md#sponsor +https://tarf.co.uk/Glossary/s/Sprial%20Model/ Glossary/s/Sprial Model.md glossary/s.md#spiral-model renamed to "Spiral Model" +https://tarf.co.uk/Glossary/s/Sprint%20Planning/ Glossary/s/Sprint Planning.md glossary/s.md#sprint-planning +https://tarf.co.uk/Glossary/s/Sprint%20retrospective/ Glossary/s/Sprint retrospective.md glossary/s.md#sprint-retrospective +https://tarf.co.uk/Glossary/s/Sprints/ Glossary/s/Sprints.md glossary/s.md#sprints +https://tarf.co.uk/Glossary/s/SQL/ Glossary/s/SQL.md glossary/s.md#sql +https://tarf.co.uk/Glossary/s/SS7/ Glossary/s/SS7.md glossary/s.md#ss7 +https://tarf.co.uk/Glossary/s/SSADM/ Glossary/s/SSADM.md glossary/s.md#ssadm +https://tarf.co.uk/Glossary/s/SSH/ Glossary/s/SSH.md glossary/s.md#ssh +https://tarf.co.uk/Glossary/s/SSID/ Glossary/s/SSID.md glossary/s.md#ssid +https://tarf.co.uk/Glossary/s/SSIS/ Glossary/s/SSIS.md glossary/s.md#ssis +https://tarf.co.uk/Glossary/s/SSL/ Glossary/s/SSL.md glossary/s.md#ssl +https://tarf.co.uk/Glossary/s/Stack/ Glossary/s/Stack.md glossary/s.md#stack +https://tarf.co.uk/Glossary/s/Staff%20Days/ Glossary/s/Staff Days.md glossary/s.md#staff-days +https://tarf.co.uk/Glossary/s/Stage/ Glossary/s/Stage.md glossary/s.md#stage +https://tarf.co.uk/Glossary/s/Staged%20Delivery%20Lifecycle/ Glossary/s/Staged Delivery Lifecycle.md glossary/s.md#staged-delivery-lifecycle +https://tarf.co.uk/Glossary/s/Stakeholder/ Glossary/s/Stakeholder.md glossary/s.md#stakeholder +https://tarf.co.uk/Glossary/s/Standard/ Glossary/s/Standard.md glossary/s.md#standard +https://tarf.co.uk/Glossary/s/Statistical%20Estimation/ Glossary/s/Statistical Estimation.md glossary/s.md#statistical-estimation +https://tarf.co.uk/Glossary/s/STB/ Glossary/s/STB.md glossary/s.md#stb +https://tarf.co.uk/Glossary/s/Story%20Mapping/ Glossary/s/Story Mapping.md glossary/s.md#story-mapping +https://tarf.co.uk/Glossary/s/Story%20Splitting/ Glossary/s/Story Splitting.md glossary/s.md#story-splitting +https://tarf.co.uk/Glossary/s/Storyboard/ Glossary/s/Storyboard.md glossary/s.md#storyboard +https://tarf.co.uk/Glossary/s/STP/ Glossary/s/STP.md glossary/s.md#stp +https://tarf.co.uk/Glossary/s/Strategy/ Glossary/s/Strategy.md glossary/s.md#strategy +https://tarf.co.uk/Glossary/s/STUN/ Glossary/s/STUN.md glossary/s.md#stun +https://tarf.co.uk/Glossary/s/Subversion/ Glossary/s/Subversion.md glossary/s.md#subversion +https://tarf.co.uk/Glossary/s/Sustainable%20Pace/ Glossary/s/Sustainable Pace.md glossary/s.md#sustainable-pace +https://tarf.co.uk/Glossary/s/SVC/ Glossary/s/SVC.md glossary/s.md#svc +https://tarf.co.uk/Glossary/s/Swift/ Glossary/s/Swift.md glossary/s.md#swift +https://tarf.co.uk/Glossary/s/Switch/ Glossary/s/Switch.md glossary/s.md#switch +https://tarf.co.uk/Glossary/s/System%20Test%20Environment/ Glossary/s/System Test Environment.md glossary/s.md#system-test-environment +https://tarf.co.uk/Glossary/s/System%20Test/ Glossary/s/System Test.md glossary/s.md#system-test +https://tarf.co.uk/Glossary/s/System/ Glossary/s/System.md glossary/s.md#system +https://tarf.co.uk/Glossary/s/SystemX/ Glossary/s/SystemX.md glossary/s.md#systemx +https://tarf.co.uk/Glossary/t/T-Shirt%20Sizing/ Glossary/t/T-Shirt Sizing.md glossary/t.md#t-shirt-sizing +https://tarf.co.uk/Glossary/t/t/ Glossary/t/t.md glossary/t.md letter index +https://tarf.co.uk/Glossary/t/Task%20Board/ Glossary/t/Task Board.md glossary/t.md#task-board +https://tarf.co.uk/Glossary/t/Task%20Estimate/ Glossary/t/Task Estimate.md glossary/t.md#task-estimate +https://tarf.co.uk/Glossary/t/Task/ Glossary/t/Task.md glossary/t.md#task +https://tarf.co.uk/Glossary/t/TCAP/ Glossary/t/TCAP.md glossary/t.md#tcap +https://tarf.co.uk/Glossary/t/Tcl-tk/ Glossary/t/Tcl-tk.md glossary/t.md#tcltk renamed to "Tcl/tk" +https://tarf.co.uk/Glossary/t/TCP/ Glossary/t/TCP.md glossary/t.md#tcp +https://tarf.co.uk/Glossary/t/Tcsh/ Glossary/t/Tcsh.md glossary/t.md#tcsh +https://tarf.co.uk/Glossary/t/TDD/ Glossary/t/TDD.md glossary/t.md#tdd +https://tarf.co.uk/Glossary/t/TDM/ Glossary/t/TDM.md glossary/t.md#tdm +https://tarf.co.uk/Glossary/t/Team%20Room/ Glossary/t/Team Room.md glossary/t.md#team-room +https://tarf.co.uk/Glossary/t/Team/ Glossary/t/Team.md glossary/t.md#team +https://tarf.co.uk/Glossary/t/Technical%20Debt/ Glossary/t/Technical Debt.md glossary/t.md#technical-debt +https://tarf.co.uk/Glossary/t/Template/ Glossary/t/Template.md glossary/t.md#template +https://tarf.co.uk/Glossary/t/Test%20Activity/ Glossary/t/Test Activity.md glossary/t.md#test-activity +https://tarf.co.uk/Glossary/t/Test%20Case%20Specification/ Glossary/t/Test Case Specification.md glossary/t.md#test-case-specification +https://tarf.co.uk/Glossary/t/Test%20Case/ Glossary/t/Test Case.md glossary/t.md#test-case +https://tarf.co.uk/Glossary/t/Test%20Design%20Specification/ Glossary/t/Test Design Specification.md glossary/t.md#test-design-specification +https://tarf.co.uk/Glossary/t/Test%20Design/ Glossary/t/Test Design.md glossary/t.md#test-design +https://tarf.co.uk/Glossary/t/Test%20Level/ Glossary/t/Test Level.md glossary/t.md#test-level +https://tarf.co.uk/Glossary/t/Test%20Plan/ Glossary/t/Test Plan.md glossary/t.md#test-plan +https://tarf.co.uk/Glossary/t/Test%20Technique/ Glossary/t/Test Technique.md glossary/t.md#test-technique +https://tarf.co.uk/Glossary/t/Test%20Type/ Glossary/t/Test Type.md glossary/t.md#test-type +https://tarf.co.uk/Glossary/t/Test/ Glossary/t/Test.md glossary/t.md#test +https://tarf.co.uk/Glossary/t/Testing/ Glossary/t/Testing.md glossary/t.md#testing +https://tarf.co.uk/Glossary/t/Three%20Amigos/ Glossary/t/Three Amigos.md glossary/t.md#three-amigos +https://tarf.co.uk/Glossary/t/Three%20C%27s/ Glossary/t/Three C's.md glossary/t.md#three-cs +https://tarf.co.uk/Glossary/t/Three%20Questions/ Glossary/t/Three Questions.md glossary/t.md#three-questions +https://tarf.co.uk/Glossary/t/TIA/ Glossary/t/TIA.md glossary/t.md#tia +https://tarf.co.uk/Glossary/t/Timebox/ Glossary/t/Timebox.md glossary/t.md#timebox +https://tarf.co.uk/Glossary/t/TKIP/ Glossary/t/TKIP.md glossary/t.md#tkip +https://tarf.co.uk/Glossary/t/TLS/ Glossary/t/TLS.md glossary/t.md#tls +https://tarf.co.uk/Glossary/t/TLV/ Glossary/t/TLV.md glossary/t.md#tlv +https://tarf.co.uk/Glossary/t/Token%20Ring/ Glossary/t/Token Ring.md glossary/t.md#token-ring +https://tarf.co.uk/Glossary/t/TOML/ Glossary/t/TOML.md glossary/t.md#toml +https://tarf.co.uk/Glossary/t/Top%20Level%20Schedule/ Glossary/t/Top Level Schedule.md glossary/t.md#top-level-schedule +https://tarf.co.uk/Glossary/t/Top-Down%20Estimation/ Glossary/t/Top-Down Estimation.md glossary/t.md#top-down-estimation +https://tarf.co.uk/Glossary/t/Triple%20DES/ Glossary/t/Triple DES.md glossary/t.md#triple-des +https://tarf.co.uk/Glossary/t/TTY/ Glossary/t/TTY.md glossary/t.md#tty +https://tarf.co.uk/Glossary/t/TUP/ Glossary/t/TUP.md glossary/t.md#tup +https://tarf.co.uk/Glossary/t/Twisted%20pair/ Glossary/t/Twisted pair.md glossary/t.md#twisted-pair +https://tarf.co.uk/Glossary/u/u/ Glossary/u/u.md glossary/u.md letter index +https://tarf.co.uk/Glossary/u/UA/ Glossary/u/UA.md glossary/u.md#ua +https://tarf.co.uk/Glossary/u/Ubiquitous%20Language/ Glossary/u/Ubiquitous Language.md glossary/u.md#ubiquitous-language +https://tarf.co.uk/Glossary/u/UDP/ Glossary/u/UDP.md glossary/u.md#udp +https://tarf.co.uk/Glossary/u/UER/ Glossary/u/UER.md glossary/u.md#uer +https://tarf.co.uk/Glossary/u/UML/ Glossary/u/UML.md glossary/u.md#uml +https://tarf.co.uk/Glossary/u/UMTS/ Glossary/u/UMTS.md glossary/u.md#umts +https://tarf.co.uk/Glossary/u/UNI/ Glossary/u/UNI.md glossary/u.md#uni +https://tarf.co.uk/Glossary/u/unicast/ Glossary/u/unicast.md glossary/u.md#unicast +https://tarf.co.uk/Glossary/u/Unicode/ Glossary/u/Unicode.md glossary/u.md#unicode +https://tarf.co.uk/Glossary/u/Unified%20Communications/ Glossary/u/Unified Communications.md glossary/u.md#unified-communications +https://tarf.co.uk/Glossary/u/Unified%20Modeling%20Language/ Glossary/u/Unified Modeling Language.md glossary/u.md#uml renamed to "UML" +https://tarf.co.uk/Glossary/u/Unit%20Test/ Glossary/u/Unit Test.md glossary/u.md#unit-test +https://tarf.co.uk/Glossary/u/Unit%20Testing/ Glossary/u/Unit Testing.md glossary/u.md#unit-testing +https://tarf.co.uk/Glossary/u/Unix/ Glossary/u/Unix.md glossary/u.md#unix +https://tarf.co.uk/Glossary/u/Update/ Glossary/u/Update.md glossary/u.md#update +https://tarf.co.uk/Glossary/u/Upstream/ Glossary/u/Upstream.md glossary/u.md#upstream +https://tarf.co.uk/Glossary/u/URI/ Glossary/u/URI.md glossary/u.md#uri +https://tarf.co.uk/Glossary/u/URL/ Glossary/u/URL.md glossary/u.md#url +https://tarf.co.uk/Glossary/u/URN/ Glossary/u/URN.md glossary/u.md#urn +https://tarf.co.uk/Glossary/u/Usability%20Testing/ Glossary/u/Usability Testing.md glossary/u.md#usability-testing +https://tarf.co.uk/Glossary/u/Use%20Case%20Model/ Glossary/u/Use Case Model.md glossary/u.md#use-case-model +https://tarf.co.uk/Glossary/u/Use%20Case%20Specification/ Glossary/u/Use Case Specification.md glossary/u.md#use-case-specification +https://tarf.co.uk/Glossary/u/Use%20Case/ Glossary/u/Use Case.md glossary/u.md#use-case +https://tarf.co.uk/Glossary/u/User%20Experience/ Glossary/u/User Experience.md glossary/u.md#user-experience +https://tarf.co.uk/Glossary/u/User%20Interface%20Design/ Glossary/u/User Interface Design.md glossary/u.md#user-interface-design +https://tarf.co.uk/Glossary/u/User%20Interface%20Prototype/ Glossary/u/User Interface Prototype.md glossary/u.md#user-interface-prototype +https://tarf.co.uk/Glossary/u/User%20Interface/ Glossary/u/User Interface.md glossary/u.md#user-interface +https://tarf.co.uk/Glossary/u/User%20Plane/ Glossary/u/User Plane.md glossary/u.md#user-plane +https://tarf.co.uk/Glossary/u/User%20Stories/ Glossary/u/User Stories.md glossary/u.md#user-stories +https://tarf.co.uk/Glossary/u/UTC/ Glossary/u/UTC.md glossary/u.md#utc +https://tarf.co.uk/Glossary/u/UTF-16/ Glossary/u/UTF-16.md glossary/u.md#utf-16 +https://tarf.co.uk/Glossary/u/UTF-8/ Glossary/u/UTF-8.md glossary/u.md#utf-8 +https://tarf.co.uk/Glossary/u/UTRAN/ Glossary/u/UTRAN.md glossary/u.md#utran +https://tarf.co.uk/Glossary/v/V%20Model/ Glossary/v/V Model.md glossary/v.md#v-model +https://tarf.co.uk/Glossary/v/V.90/ Glossary/v/V.90.md glossary/v.md#v90 +https://tarf.co.uk/Glossary/v/V.92/ Glossary/v/V.92.md glossary/v.md#v92 +https://tarf.co.uk/Glossary/v/v/ Glossary/v/v.md glossary/v.md letter index +https://tarf.co.uk/Glossary/v/VAD/ Glossary/v/VAD.md glossary/v.md#vad +https://tarf.co.uk/Glossary/v/VB/ Glossary/v/VB.md glossary/v.md#vb +https://tarf.co.uk/Glossary/v/VBS/ Glossary/v/VBS.md glossary/v.md#vbs +https://tarf.co.uk/Glossary/v/VC/ Glossary/v/VC.md glossary/v.md#vc +https://tarf.co.uk/Glossary/v/VCI/ Glossary/v/VCI.md glossary/v.md#vci +https://tarf.co.uk/Glossary/v/VCID/ Glossary/v/VCID.md glossary/v.md#vcid +https://tarf.co.uk/Glossary/v/VDSL/ Glossary/v/VDSL.md glossary/v.md#vdsl +https://tarf.co.uk/Glossary/v/Velocity/ Glossary/v/Velocity.md glossary/v.md#velocity +https://tarf.co.uk/Glossary/v/Verifier/ Glossary/v/Verifier.md glossary/v.md#verifier +https://tarf.co.uk/Glossary/v/Version%20Control/ Glossary/v/Version Control.md glossary/v.md#version-control +https://tarf.co.uk/Glossary/v/Version/ Glossary/v/Version.md glossary/v.md#version +https://tarf.co.uk/Glossary/v/VHDL/ Glossary/v/VHDL.md glossary/v.md#vhdl +https://tarf.co.uk/Glossary/v/VISP/ Glossary/v/VISP.md glossary/v.md#visp +https://tarf.co.uk/Glossary/v/VLAN/ Glossary/v/VLAN.md glossary/v.md#vlan +https://tarf.co.uk/Glossary/v/VLR/ Glossary/v/VLR.md glossary/v.md#vlr +https://tarf.co.uk/Glossary/v/VM/ Glossary/v/VM.md glossary/v.md#vm +https://tarf.co.uk/Glossary/v/VOD/ Glossary/v/VOD.md glossary/v.md#vod +https://tarf.co.uk/Glossary/v/VoIP/ Glossary/v/VoIP.md glossary/v.md#voip +https://tarf.co.uk/Glossary/v/VoLTE/ Glossary/v/VoLTE.md glossary/v.md#volte +https://tarf.co.uk/Glossary/v/Vorbis/ Glossary/v/Vorbis.md glossary/v.md#vorbis +https://tarf.co.uk/Glossary/v/VPI/ Glossary/v/VPI.md glossary/v.md#vpi +https://tarf.co.uk/Glossary/v/VPLS/ Glossary/v/VPLS.md glossary/v.md#vpls +https://tarf.co.uk/Glossary/v/VPN/ Glossary/v/VPN.md glossary/v.md#vpn +https://tarf.co.uk/Glossary/v/VPWS/ Glossary/v/VPWS.md glossary/v.md#vpws +https://tarf.co.uk/Glossary/v/VRF/ Glossary/v/VRF.md glossary/v.md#vrf +https://tarf.co.uk/Glossary/w/W-CDMA/ Glossary/w/W-CDMA.md glossary/w.md#w-cdma +https://tarf.co.uk/Glossary/w/w/ Glossary/w/w.md glossary/w.md letter index +https://tarf.co.uk/Glossary/w/W3C/ Glossary/w/W3C.md glossary/w.md#w3c +https://tarf.co.uk/Glossary/w/Walkthrough/ Glossary/w/Walkthrough.md glossary/w.md#walkthrough +https://tarf.co.uk/Glossary/w/WAN/ Glossary/w/WAN.md glossary/w.md#wan +https://tarf.co.uk/Glossary/w/WANET/ Glossary/w/WANET.md glossary/w.md#wanet +https://tarf.co.uk/Glossary/w/WAP/ Glossary/w/WAP.md glossary/w.md#wap +https://tarf.co.uk/Glossary/w/Waterfall%20Lifecycle/ Glossary/w/Waterfall Lifecycle.md glossary/w.md#waterfall-lifecycle +https://tarf.co.uk/Glossary/w/Waterfall/ Glossary/w/Waterfall.md glossary/w.md#waterfall +https://tarf.co.uk/Glossary/w/WBC/ Glossary/w/WBC.md glossary/w.md#wbc +https://tarf.co.uk/Glossary/w/WBCC/ Glossary/w/WBCC.md glossary/w.md#wbcc +https://tarf.co.uk/Glossary/w/WBMC/ Glossary/w/WBMC.md glossary/w.md#wbmc +https://tarf.co.uk/Glossary/w/Weekly%20Individual%20Plan/ Glossary/w/Weekly Individual Plan.md glossary/w.md#weekly-individual-plan +https://tarf.co.uk/Glossary/w/WEP/ Glossary/w/WEP.md glossary/w.md#wep +https://tarf.co.uk/Glossary/w/What%20Requirement/ Glossary/w/What Requirement.md glossary/w.md#what-requirement +https://tarf.co.uk/Glossary/w/Why%20Requirement/ Glossary/w/Why Requirement.md glossary/w.md#why-requirement +https://tarf.co.uk/Glossary/w/Wi-Fi/ Glossary/w/Wi-Fi.md glossary/w.md#wi-fi +https://tarf.co.uk/Glossary/w/Wiki/ Glossary/w/Wiki.md glossary/w.md#wiki +https://tarf.co.uk/Glossary/w/WiMAX/ Glossary/w/WiMAX.md glossary/w.md#wimax +https://tarf.co.uk/Glossary/w/WIN-T/ Glossary/w/WIN-T.md glossary/w.md#win-t +https://tarf.co.uk/Glossary/w/Wireless%20Mesh%20Network/ Glossary/w/Wireless Mesh Network.md glossary/w.md#wireless-mesh-network +https://tarf.co.uk/Glossary/w/Wireless%20Repeater/ Glossary/w/Wireless Repeater.md glossary/w.md#wireless-repeater +https://tarf.co.uk/Glossary/w/Wireshark/ Glossary/w/Wireshark.md glossary/w.md#wireshark +https://tarf.co.uk/Glossary/w/WISP/ Glossary/w/WISP.md glossary/w.md#wisp +https://tarf.co.uk/Glossary/w/WLR/ Glossary/w/WLR.md glossary/w.md#wlr +https://tarf.co.uk/Glossary/w/WMA/ Glossary/w/WMA.md glossary/w.md#wma +https://tarf.co.uk/Glossary/w/Work%20Breakdown%20Structure/ Glossary/w/Work Breakdown Structure.md glossary/w.md#work-breakdown-structure +https://tarf.co.uk/Glossary/w/Work%20Item/ Glossary/w/Work Item.md glossary/w.md#work-item +https://tarf.co.uk/Glossary/w/Work%20Package/ Glossary/w/Work Package.md glossary/w.md#work-package +https://tarf.co.uk/Glossary/w/Work%20Plan/ Glossary/w/Work Plan.md glossary/w.md#work-plan +https://tarf.co.uk/Glossary/w/Work%20Product/ Glossary/w/Work Product.md glossary/w.md#work-product +https://tarf.co.uk/Glossary/w/WPA2/ Glossary/w/WPA2.md glossary/w.md#wpa2 +https://tarf.co.uk/Glossary/w/WPS/ Glossary/w/WPS.md glossary/w.md#wps +https://tarf.co.uk/Glossary/w/write-only%20code/ Glossary/w/write-only code.md glossary/w.md#write-only-code +https://tarf.co.uk/Glossary/w/WS-BPEL/ Glossary/w/WS-BPEL.md glossary/w.md#ws-bpel +https://tarf.co.uk/Glossary/w/WSDL/ Glossary/w/WSDL.md glossary/w.md#wsdl +https://tarf.co.uk/Glossary/x/X.25/ Glossary/x/X.25.md glossary/x.md#x25 +https://tarf.co.uk/Glossary/x/X.509/ Glossary/x/X.509.md glossary/x.md#x509 +https://tarf.co.uk/Glossary/x/x/ Glossary/x/x.md glossary/x.md letter index +https://tarf.co.uk/Glossary/x/XML%20Schema/ Glossary/x/XML Schema.md glossary/x.md#xml-schema +https://tarf.co.uk/Glossary/x/XML-RPC/ Glossary/x/XML-RPC.md glossary/x.md#xml-rpc +https://tarf.co.uk/Glossary/x/XML/ Glossary/x/XML.md glossary/x.md#xml +https://tarf.co.uk/Glossary/x/XMPP/ Glossary/x/XMPP.md glossary/x.md#xmpp +https://tarf.co.uk/Glossary/x/XP/ Glossary/x/XP.md glossary/x.md#xp +https://tarf.co.uk/Glossary/x/xsd/ Glossary/x/xsd.md glossary/x.md#xsd +https://tarf.co.uk/Glossary/x/XSS/ Glossary/x/XSS.md glossary/x.md#xss +https://tarf.co.uk/Glossary/y/y/ Glossary/y/y.md glossary/y.md letter index +https://tarf.co.uk/Glossary/y/YAML/ Glossary/y/YAML.md glossary/y.md#yaml +https://tarf.co.uk/Glossary/z/z/ Glossary/z/z.md glossary/z.md letter index +https://tarf.co.uk/Glossary/z/ZigBee/ Glossary/z/ZigBee.md glossary/z.md#zigbee +https://tarf.co.uk/Glossary/%CE%BC/%CE%BC-law/ Glossary/μ/μ-law.md glossary/m.md#mu-law renamed to "Mu-law" +https://tarf.co.uk/Glossary/%CE%BC/%CE%BC/ Glossary/μ/μ.md glossary/index.md letter index https://markbac.github.io/technical-documentation/glossary/ glossary/index.md technical-documentation site; redirect lives in that repository -https://markbac.github.io/technical-documentation/glossary/a-c/ glossary/a-c.md same anchors except the rows below -https://markbac.github.io/technical-documentation/glossary/d-f/ glossary/d-f.md same anchors except the rows below -https://markbac.github.io/technical-documentation/glossary/g-k/ glossary/g-k.md same anchors except the rows below -https://markbac.github.io/technical-documentation/glossary/l-n/ glossary/l-n.md same anchors except the rows below -https://markbac.github.io/technical-documentation/glossary/o-r/ glossary/o-r.md same anchors except the rows below -https://markbac.github.io/technical-documentation/glossary/s-z/ glossary/s-z.md same anchors except the rows below -https://markbac.github.io/technical-documentation/glossary/a-c/#artifact glossary/a-c.md#artefact renamed to "Artefact" -https://markbac.github.io/technical-documentation/glossary/a-c/#best-practice-description glossary/a-c.md#best-practice renamed to "Best Practice" -https://markbac.github.io/technical-documentation/glossary/a-c/#cam-item glossary/g-k.md#item renamed to "Item" -https://markbac.github.io/technical-documentation/glossary/a-c/#complier glossary/a-c.md#compiler renamed to "Compiler" -https://markbac.github.io/technical-documentation/glossary/a-c/#construx-knowledge-area glossary/s-z.md#software-engineering-body-of-knowledge renamed to "Software Engineering Body of Knowledge" -https://markbac.github.io/technical-documentation/glossary/a-c/#corrective-activity-management glossary/a-c.md dropped -https://markbac.github.io/technical-documentation/glossary/d-f/#data-modeling glossary/d-f.md#data-modelling renamed to "Data modelling" -https://markbac.github.io/technical-documentation/glossary/d-f/#date-warehouse glossary/d-f.md#data-warehouse renamed to "Data Warehouse" -https://markbac.github.io/technical-documentation/glossary/d-f/#deplhi glossary/d-f.md#delphi renamed to "Delphi" -https://markbac.github.io/technical-documentation/glossary/d-f/#eee-8021qay glossary/g-k.md#ieee-8021qay renamed to "IEEE 802.1Qay" -https://markbac.github.io/technical-documentation/glossary/d-f/#evaluation glossary/d-f.md dropped -https://markbac.github.io/technical-documentation/glossary/d-f/#expert-judgment glossary/d-f.md#expert-judgement renamed to "Expert Judgement" -https://markbac.github.io/technical-documentation/glossary/g-k/#international-organization-for-standardization glossary/g-k.md#iso renamed to "ISO" -https://markbac.github.io/technical-documentation/glossary/g-k/#ios-cisco-os glossary/g-k.md#ios renamed to "IOS" -https://markbac.github.io/technical-documentation/glossary/l-n/#l2s glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#las glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#link glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#lite glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#lps glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#lte-and-mobile-internet glossary/l-n.md#lte-4g renamed to "LTE (4G)" -https://markbac.github.io/technical-documentation/glossary/l-n/#mag glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#mas glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#material glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/l-n/#mobile-networks-2g3g glossary/s-z.md#umts renamed to "UMTS" -https://markbac.github.io/technical-documentation/glossary/l-n/#modeling glossary/l-n.md#modelling renamed to "Modelling" -https://markbac.github.io/technical-documentation/glossary/l-n/#mps glossary/l-n.md dropped -https://markbac.github.io/technical-documentation/glossary/o-r/#python-2x-3x glossary/o-r.md#python renamed to "Python" -https://markbac.github.io/technical-documentation/glossary/o-r/#rainbow glossary/o-r.md dropped -https://markbac.github.io/technical-documentation/glossary/o-r/#reference glossary/o-r.md dropped -https://markbac.github.io/technical-documentation/glossary/o-r/#risk-reward-matric glossary/o-r.md#risk-reward-matrix renamed to "Risk Reward Matrix" -https://markbac.github.io/technical-documentation/glossary/s-z/#sematic-versioning glossary/s-z.md#semantic-versioning renamed to "Semantic Versioning" -https://markbac.github.io/technical-documentation/glossary/s-z/#sprial-model glossary/s-z.md#spiral-model renamed to "Spiral Model" -https://markbac.github.io/technical-documentation/glossary/s-z/#the-first-example-has-two-steps glossary/g-k.md#gherkin renamed to "Gherkin" -https://markbac.github.io/technical-documentation/glossary/s-z/#the-second-example-has-three-steps glossary/g-k.md#gherkin renamed to "Gherkin" -https://markbac.github.io/technical-documentation/glossary/s-z/#unified-modeling-language glossary/s-z.md#uml renamed to "UML" -https://markbac.github.io/technical-documentation/glossary/s-z/#-law glossary/l-n.md#mu-law renamed to "Mu-law" +https://markbac.github.io/technical-documentation/glossary/a-c/ glossary/index.md same anchors except the rows below +https://markbac.github.io/technical-documentation/glossary/d-f/ glossary/index.md same anchors except the rows below +https://markbac.github.io/technical-documentation/glossary/g-k/ glossary/index.md same anchors except the rows below +https://markbac.github.io/technical-documentation/glossary/l-n/ glossary/index.md same anchors except the rows below +https://markbac.github.io/technical-documentation/glossary/o-r/ glossary/index.md same anchors except the rows below +https://markbac.github.io/technical-documentation/glossary/s-z/ glossary/index.md same anchors except the rows below +https://markbac.github.io/technical-documentation/glossary/a-c/#artifact glossary/a.md#artefact renamed to "Artefact" +https://markbac.github.io/technical-documentation/glossary/a-c/#best-practice-description glossary/b.md#best-practice renamed to "Best Practice" +https://markbac.github.io/technical-documentation/glossary/a-c/#cam-item glossary/i.md#item renamed to "Item" +https://markbac.github.io/technical-documentation/glossary/a-c/#complier glossary/c.md#compiler renamed to "Compiler" +https://markbac.github.io/technical-documentation/glossary/a-c/#construx-knowledge-area glossary/s.md#software-engineering-body-of-knowledge renamed to "Software Engineering Body of Knowledge" +https://markbac.github.io/technical-documentation/glossary/a-c/#corrective-activity-management glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/d-f/#data-modeling glossary/d.md#data-modelling renamed to "Data modelling" +https://markbac.github.io/technical-documentation/glossary/d-f/#date-warehouse glossary/d.md#data-warehouse renamed to "Data Warehouse" +https://markbac.github.io/technical-documentation/glossary/d-f/#deplhi glossary/d.md#delphi renamed to "Delphi" +https://markbac.github.io/technical-documentation/glossary/d-f/#eee-8021qay glossary/i.md#ieee-8021qay renamed to "IEEE 802.1Qay" +https://markbac.github.io/technical-documentation/glossary/d-f/#evaluation glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/d-f/#expert-judgment glossary/e.md#expert-judgement renamed to "Expert Judgement" +https://markbac.github.io/technical-documentation/glossary/g-k/#international-organization-for-standardization glossary/i.md#iso renamed to "ISO" +https://markbac.github.io/technical-documentation/glossary/g-k/#ios-cisco-os glossary/i.md#ios renamed to "IOS" +https://markbac.github.io/technical-documentation/glossary/l-n/#l2s glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#las glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#link glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#lite glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#lps glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#lte-and-mobile-internet glossary/l.md#lte-4g renamed to "LTE (4G)" +https://markbac.github.io/technical-documentation/glossary/l-n/#mag glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#mas glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#material glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/l-n/#mobile-networks-2g3g glossary/u.md#umts renamed to "UMTS" +https://markbac.github.io/technical-documentation/glossary/l-n/#modeling glossary/m.md#modelling renamed to "Modelling" +https://markbac.github.io/technical-documentation/glossary/l-n/#mps glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/o-r/#python-2x-3x glossary/p.md#python renamed to "Python" +https://markbac.github.io/technical-documentation/glossary/o-r/#rainbow glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/o-r/#reference glossary/index.md dropped +https://markbac.github.io/technical-documentation/glossary/o-r/#risk-reward-matric glossary/r.md#risk-reward-matrix renamed to "Risk Reward Matrix" +https://markbac.github.io/technical-documentation/glossary/s-z/#sematic-versioning glossary/s.md#semantic-versioning renamed to "Semantic Versioning" +https://markbac.github.io/technical-documentation/glossary/s-z/#sprial-model glossary/s.md#spiral-model renamed to "Spiral Model" +https://markbac.github.io/technical-documentation/glossary/s-z/#the-first-example-has-two-steps glossary/g.md#gherkin renamed to "Gherkin" +https://markbac.github.io/technical-documentation/glossary/s-z/#the-second-example-has-three-steps glossary/g.md#gherkin renamed to "Gherkin" +https://markbac.github.io/technical-documentation/glossary/s-z/#unified-modeling-language glossary/u.md#uml renamed to "UML" +https://markbac.github.io/technical-documentation/glossary/s-z/#-law glossary/m.md#mu-law renamed to "Mu-law" + +https://markbac.github.io/glossary/a-c/ glossary/a-c.md glossary/index.md letter-group page, replaced by one page per letter +https://markbac.github.io/glossary/d-f/ glossary/d-f.md glossary/index.md letter-group page, replaced by one page per letter +https://markbac.github.io/glossary/g-k/ glossary/g-k.md glossary/index.md letter-group page, replaced by one page per letter +https://markbac.github.io/glossary/l-n/ glossary/l-n.md glossary/index.md letter-group page, replaced by one page per letter +https://markbac.github.io/glossary/o-r/ glossary/o-r.md glossary/index.md letter-group page, replaced by one page per letter +https://markbac.github.io/glossary/s-z/ glossary/s-z.md glossary/index.md letter-group page, replaced by one page per letter \ No newline at end of file diff --git a/content/glossary/s-z.md b/content/glossary/s-z.md deleted file mode 100644 index 3c40a12..0000000 --- a/content/glossary/s-z.md +++ /dev/null @@ -1,1379 +0,0 @@ ---- -title: Glossary, S to Z and numbers ---- - -# S to Z and numbers - -## S - -### S-Tag -**Service VLAN tag.** In IEEE 802.1ad double tagging (Q-in-Q), the outer VLAN tag added by the service provider, with TPID 0x88A8. It sits outside the customer's own C-Tag. - -See also: [C-Tag](a-c.md#c-tag), [VLAN](#vlan), [Dot1Q](d-f.md#dot1q). - -### S-GW -**Serving Gateway.** The LTE core gateway that forwards users' packets between the eNodeBs and the PGW. It anchors the user plane during handovers and buffers data for idle devices while the MME pages them. - -See also: [PGW](o-r.md#pgw), [MME](l-n.md#mme), [LTE (4G)](l-n.md#lte-4g). - -### SAAL -**Signalling ATM Adaptation Layer.** The ATM adaptation layer that carries signalling messages reliably between ATM switches. - -See also: [ATM](a-c.md#atm), [AAL5](a-c.md#aal5). - -### SaaS -**Software as a Service.** Software that a provider hosts and runs, and customers use over the internet, usually in a browser, on a subscription. - -See also: [PaaS](o-r.md#paas), [IaaS](g-k.md#iaas). - -### SAE -**System Architecture Evolution.** The 3GPP work that defined the LTE core network, the Evolved Packet Core: a simpler, all-IP core that also supports other access networks, such as Wi-Fi. - -See also: [LTE (4G)](l-n.md#lte-4g), [MME](l-n.md#mme), [S-GW](#s-gw), [PGW](o-r.md#pgw). - -### SAFe -**Scaled Agile Framework.** A framework for running agile development across many teams in a large organisation, with set roles, planning events and portfolio management. - -See also: [Scaled Agile](#scaled-agile), [LeSS](l-n.md#less), [DAD](d-f.md#dad). - -### SBC -**Session Border Controller.** A device at the edge of a VoIP network that controls the signalling, and usually the media, of calls crossing it. It provides security, NAT traversal, interworking and policing. - -See also: [SIP](#sip), [SIP Trunk](#sip-trunk), [NAT](l-n.md#nat). - -### SC-FDMA -**Single-Carrier FDMA.** A multiple access scheme, also called DFT-spread OFDMA, with a lower peak-to-average power ratio than OFDMA. That saves battery power, so LTE uses it for the uplink. - -See also: [OFDMA](o-r.md#ofdma), [LTE (4G)](l-n.md#lte-4g). - -### Scala -A statically typed language for the Java virtual machine that combines object-oriented and functional programming, and works with Java libraries. - -See also: [Java](g-k.md#java). - -### Scaled Agile -Agile methods applied to large projects and programmes, with many teams working on one product. - -See also: [SAFe](#safe), [LeSS](l-n.md#less), [Scrum of Scrums](#scrum-of-scrums). - -### SCCP -**Signalling Connection Control Part.** The SS7 layer above MTP that adds richer addressing (global title translation), connection-oriented services and segmentation. MAP and other application parts use it. - -See also: [MTP](l-n.md#mtp), [SS7](#ss7), [TCAP](#tcap). - -### Scenario -A narrative of an activity, told as a story, often a concrete path through a use case. - -See also: [Use Case](#use-case), [Storyboard](#storyboard). - -### Schedule -A plan that sets the timing, resources and dependencies of the tasks needed to finish a project or part of one. - -See also: [Business Schedule](a-c.md#business-schedule), [Detailed Schedule](d-f.md#detailed-schedule). - -### Scheduler -The part of an operating system, or a network device, that decides which task or packet gets a resource next, such as a processor or a link. Its aims include keeping resources busy, sharing them fairly and meeting deadlines or quality of service targets. - -See also: [Round Robin](o-r.md#round-robin), [Preemption](o-r.md#preemption), [RTOS](o-r.md#rtos). - -### SCM -**Software Configuration Management.** Tracking and controlling changes to software: version control, baselines, builds and releases. It is the software part of configuration management. - -See also: [Configuration Management](a-c.md#configuration-management), [Version Control](#version-control). - -### Scratch -A visual, block-based programming language and online community from the MIT Media Lab, designed to teach children to code. - -### Scribe -The person who records issues in an inspection or other formal meeting. - -See also: [Inspection](g-k.md#inspection). - -### Scrum -An agile framework in which a small team works in fixed-length sprints towards a sprint goal. It has three roles: product owner, Scrum Master and developers. Its events are sprint planning, the daily scrum, the sprint review and the retrospective. - -See also: [Sprints](#sprints), [Product Owner](o-r.md#product-owner), [Scrum Master](#scrum-master). - -### Scrum Master -The Scrum role that helps the team follow Scrum and work well, by coaching, removing obstacles and protecting the team's focus. - -See also: [Scrum](#scrum). - -### Scrum of Scrums -A way to coordinate several Scrum teams: each team sends a representative to a regular meeting to raise dependencies and problems between teams. - -See also: [Scaled Agile](#scaled-agile). - -### SCTP -**Stream Control Transmission Protocol.** A transport protocol (RFC 9260) that delivers messages reliably like TCP, keeps message boundaries like UDP, and supports multiple streams and multi-homing. It carries SS7 signalling over IP (SIGTRAN) and LTE control traffic. - -See also: [TCP](#tcp), [UDP](#udp), [SGW](#sgw). - -### SDES -**SDP Security Descriptions.** A way to exchange the keys for SRTP (secure RTP) inside the SDP of a SIP call. It relies on the signalling itself being encrypted. - -See also: [SDP](#sdp), [RTP](o-r.md#rtp), [SIP](#sip). - -### SDH -**Synchronous Digital Hierarchy.** The ITU-T standard for synchronous multiplexing of digital signals over fibre, from STM-1 (155 Mbit/s) upwards, with strong management and fast protection switching. It replaced PDH; SONET is the North American equivalent. - -See also: [SONET](#sonet), [PDH](o-r.md#pdh), [EoS](d-f.md#eos). - -### SDK -**Software Development Kit.** The libraries, tools, documentation and examples needed to build software for a particular platform or service. - -See also: [API](a-c.md#api), [DDK](d-f.md#ddk). - -### SDLC -**Systems Development Life Cycle.** A structured process for developing a system, from feasibility study through to maintenance. Waterfall, spiral and iterative models are all versions of it. - -See also: [Lifecycle](l-n.md#lifecycle), [Waterfall](#waterfall). - -### SDMA -**Space-Division Multiple Access.** Using smart antennas to point beams at different users, so that users in different places can share the same frequency at the same time. - -See also: [MIMO](l-n.md#mimo), [CDMA](a-c.md#cdma). - -### SDN -**Software-Defined Networking.** Separating a network's control plane from its data plane, and putting control in software controllers that configure the forwarding devices through open interfaces. The network is then managed centrally and programmatically. - -See also: [Control Plane](a-c.md#control-plane), [Data Plane](d-f.md#data-plane), [NaaS](l-n.md#naas). - -### SDP -**Session Description Protocol.** A text format (RFC 8866) for describing multimedia sessions: media types, codecs, addresses and ports. SIP uses it in an offer and answer exchange to agree the media of a call. - -See also: [SIP](#sip), [SDES](#sdes), [RTP](o-r.md#rtp). - -### SDS -**Software Design Specification.** A document that sets out a system's detailed design: its modules, interfaces, data models, state machines, timing and memory layout. Ideally it is kept up to date alongside the code. - -See also: [Software Requirements Specification](#software-requirements-specification), [Low Level Design](l-n.md#low-level-design). - -### SDSL -**Symmetric Digital Subscriber Line.** DSL with the same speed in both directions, aimed at businesses. HDSL and G.SHDSL are examples. - -See also: [ADSL](a-c.md#adsl), [HDSL](g-k.md#hdsl), [DSL](d-f.md#dsl). - -### Sed -**Stream editor.** A Unix tool, written at Bell Labs in the 1970s, that transforms text line by line using a small command language, most often for substitutions with regular expressions. - -See also: [Awk](a-c.md#awk), [Regular expression](o-r.md#regular-expression). - -### Secure Element -**SE.** A tamper-resistant chip that stores keys and runs sensitive code in isolation, with protection against physical and side-channel attacks. SIM cards, payment cards and many IoT devices use one. - -See also: [Root of Trust](o-r.md#root-of-trust), [TrustZone](#trustzone), [SIM](#sim). - -### SE -See [Secure Element](#secure-element). - -### Semaphore -A synchronisation primitive: a counter that threads decrement to take one unit of a resource, waiting if none is left, and increment to release it. A binary semaphore works like a lock. - -See also: [Mutex](l-n.md#mutex), [Mutual Exclusion](l-n.md#mutual-exclusion). - -### Semantic Versioning -**SemVer.** A version numbering scheme, `MAJOR.MINOR.PATCH`. Increase the major number for incompatible API changes, the minor number for compatible new features and the patch number for compatible bug fixes. Pre-release and build labels can follow, as in `2.1.0-rc.1+build.5`. - -See also: [Release Management](o-r.md#release-management), [Version](#version). - -### Sequence Model -A model, such as a UML sequence diagram, that shows the messages exchanged between objects or roles in time order. - -See also: [Collaboration Model](a-c.md#collaboration-model), [UML](#uml). - -### SF -**Superframe.** The T1 framing format that groups 12 frames and uses their framing bits for alignment. Extended Superframe (ESF), with 24 frames, replaced it. - -See also: [PDH](o-r.md#pdh), [TDM](#tdm). - -### SFTP -**SSH File Transfer Protocol.** A protocol for transferring and managing files over an SSH connection. It is unrelated to FTP and FTPS, apart from its purpose. - -See also: [SSH](#ssh), [FTPS](d-f.md#ftps), [FTP](d-f.md#ftp). - -### SGML -**Standard Generalized Markup Language.** The ISO standard (ISO 8879) for defining markup languages that describe a document's structure rather than its appearance. HTML and XML derive from it. - -See also: [XML](#xml). - -### SGSN -**Serving GPRS Support Node.** The GPRS and UMTS core node that serves devices in its area. It tracks their location, authenticates them and routes their packets to and from the GGSN. - -See also: [GGSN](g-k.md#ggsn), [GPRS](g-k.md#gprs), [MME](l-n.md#mme). - -### SGW -**Signalling Gateway.** A gateway that carries SS7 signalling between circuit-switched networks and IP networks, for example passing ISUP from an MGCF over SCTP (SIGTRAN) to the SS7 network over MTP. - -See also: [MGCF](l-n.md#mgcf), [SCTP](#sctp), [SS7](#ss7). - -### Sh -The Bourne shell, the standard shell of Version 7 Unix, written by Stephen Bourne. `/bin/sh` on Unix-like systems today is a shell compatible with it, as POSIX defines. - -See also: [Bash](a-c.md#bash), [Ksh](g-k.md#ksh), [Shell Scripting](#shell-scripting), [POSIX](o-r.md#posix). - -### SHA -**Secure Hash Algorithm.** A family of cryptographic hash functions published by NIST. SHA-1 (160-bit) is broken and should not be used for security. SHA-2 (SHA-256, SHA-512) and SHA-3 are current. - -See also: [Hash Function](g-k.md#hash-function), [MD5](l-n.md#md5). - -### Shell Scripting -Writing scripts for a command-line shell, such as Bash, to automate tasks like handling files and running other tools. - -See also: [Bash](a-c.md#bash), [Sh](#sh). - -### Sidetone -The small part of a speaker's own voice fed back into their earpiece, so that a phone line sounds alive and people do not shout. - -See also: [Echo](d-f.md#echo). - -### Sign Up -**Signing up for tasks.** In agile teams, members choose the tasks they will work on, rather than a manager assigning them. - -See also: [Task Board](#task-board). - -### SIM -**Subscriber Identity Module.** The smart card, or embedded chip (eSIM), that holds a mobile subscriber's IMSI and secret key. The network uses them to identify and authenticate the subscriber. - -See also: [IMSI](g-k.md#imsi), [Secure Element](#secure-element), [HLR](g-k.md#hlr). - -### SIMPLE -**SIP for Instant Messaging and Presence Leveraging Extensions.** The IETF's SIP-based protocols for instant messaging and presence. XMPP is the main alternative. - -See also: [SIP](#sip), [XMPP](#xmpp). - -### Simple Design -The agile practice of keeping the design as simple as the current needs allow, guided by rules such as Kent Beck's. - -See also: [Rule of Simplicity](o-r.md#rule-of-simplicity), [KISS Principle](g-k.md#kiss-principle), [Quick Design Session](o-r.md#quick-design-session). - -### Simplex -Communication in one direction only, such as broadcast radio. ITU usage also applies the term to links that work one way at a time, which most people call half duplex. - -See also: [Duplex](d-f.md#duplex), [Half Duplex](g-k.md#half-duplex). - -### Single Step Test -A test in a debugger, stepping through the source code line by line as it runs. - -### SIP -**Session Initiation Protocol.** The IETF's text-based signalling protocol (RFC 3261) for setting up, changing and ending voice, video and messaging sessions. It is modelled on HTTP and carries SDP to agree the media. SIP is the basis of most VoIP and of IMS. - -See also: [SDP](#sdp), [RTP](o-r.md#rtp), [IMS](g-k.md#ims), [SIP Trunk](#sip-trunk). - -### SIP Trunk -A VoIP connection, using SIP, between a business's phone system and a service provider, replacing ISDN lines for external calls. - -See also: [SIP](#sip), [SBC](#sbc), [ISDN](g-k.md#isdn). - -### SISO -**Single-Input, Single-Output.** A system with one input and one output; in radio, a link with one antenna at each end. - -See also: [MIMO](l-n.md#mimo). - -### Skew -**Clock skew.** The difference in the arrival time of the same clock signal at different parts of a circuit, or between the readings of two clocks. - -See also: [Jitter](g-k.md#jitter). - -### SLA -**Service Level Agreement.** An agreement between a provider and a customer on the service's measurable levels, such as availability, response and repair times, with remedies if they are missed. - -See also: [KPI](g-k.md#kpi), [MTR](l-n.md#mtr). - -### SLIP -**Serial Line Internet Protocol.** A very simple way of sending IP packets over a serial line (RFC 1055), with no negotiation or error checking. PPP replaced it, though it is still used on small microcontrollers. - -See also: [PPP](o-r.md#ppp). - -### SME -**Subject Matter Expert.** A person with deep knowledge of a particular area, often a business domain that the software serves. SME also commonly means small and medium-sized enterprise. - -### Smoke Test -A quick test that a new build works at all, run before deeper testing. - -See also: [Smoke testing](#smoke-testing), [Build](a-c.md#build). - -### Smoke testing -Quick, shallow testing of a system's most important functions, to decide whether it is stable enough for further testing. - -See also: [Smoke Test](#smoke-test), [Regression testing](o-r.md#regression-testing). - -### SMP -**Symmetric Multiprocessing.** A design in which two or more identical processors, or cores, share one memory and are run by one operating system that treats them all equally. Most multi-core computers work this way. - -See also: [AMP](a-c.md#amp). - -### SMPF -**Shared Metallic Path Facility.** An Openreach product that shares one copper line between two providers. Typically one provides voice over the low frequencies and the other broadband over the high frequencies. - -See also: [MPF](l-n.md#mpf), [WLR](#wlr). - -### SMSC -**Short Message Service Centre.** The network element that stores SMS messages and forwards them to their destination, retrying until the recipient is reachable. - -See also: [MAP](l-n.md#map), [GSM](g-k.md#gsm). - -### Snake Case -Writing compound names in lower case with underscores between the words, as in `snake_case`. Python uses it for functions and variables. - -See also: [Camel Case](a-c.md#camel-case). - -### SNMP -**Simple Network Management Protocol.** The IETF protocol for monitoring and managing network devices. Managers read and set variables in each device's MIB, and devices send traps when events happen. SNMPv3 adds proper security. - -See also: [MIB](l-n.md#mib), [Management Plane](l-n.md#management-plane), [Cacti](a-c.md#cacti). - -### SNTP -**Simple Network Time Protocol.** A simpler form of NTP, without NTP's long-term state and filtering, for devices that need only modest clock accuracy. - -See also: [NTP](l-n.md#ntp). - -### SOA -**Service-Oriented Architecture.** An architecture style in which applications are built from services. Each service provides a business function through a network interface, can be used without knowing how it works, and can be combined with others. - -See also: [ESB](d-f.md#esb), [Microservice](l-n.md#microservice), [SOAP](#soap). - -### SOAP -An XML-based messaging protocol for web services, usually over HTTP. A SOAP envelope holds a header and a body, and WSDL describes the services. REST has largely replaced it for new APIs. - -See also: [WSDL](#wsdl), [REST](o-r.md#rest), [XML](#xml). - -### Softswitch -A call control system implemented in software on general-purpose servers, which routes calls in VoIP and next-generation networks. It controls media gateways rather than switching media itself. - -See also: [Class 4 Softswitch](a-c.md#class-4-softswitch), [Class 5 Softswitch](a-c.md#class-5-softswitch), [MGCP](l-n.md#mgcp), [H.248](g-k.md#h248). - -### Software Change Request -See [Change Request](a-c.md#change-request). - -### Software Configuration Management -See [SCM](#scm). - -### Software Development Plan -**SDP.** On software projects, another name for the [Project Plan](o-r.md#project-plan). - -### Software Engineering Body of Knowledge -**SWEBOK.** The IEEE Computer Society's guide to the generally accepted knowledge of software engineering, organised into knowledge areas. - -See also: [IEEE](g-k.md#ieee). - -### Software Engineering Lab -**SEL.** The Software Engineering Laboratory at NASA Goddard, which studied and improved software processes from 1976 into the 2000s. - -### Software Engineering Process Group -**SEPG.** A group that leads software process improvement in an organisation, and gives projects independent technical oversight. - -See also: [Capability Maturity Model](a-c.md#capability-maturity-model). - -### Software Requirements Specification -**SRS.** A document that captures the requirements for a software system, often alongside models and prototypes. - -See also: [Requirements](o-r.md#requirements), [SDS](#sds). - -### SOLID Principles -Five object-oriented design principles promoted by Robert C. Martin: single responsibility, open-closed, Liskov substitution, interface segregation and dependency inversion. Together they aim for code that is modular and easy to change. - -See also: [OO](o-r.md#oo), [Design Pattern](d-f.md#design-pattern). - -### SoGEA -**Single Order Generic Ethernet Access.** An Openreach broadband product over fibre to the cabinet with no phone service on the line, ordered as one product. It is the standalone broadband line used as the copper phone network closes. - -See also: [VDSL](#vdsl), [EoFTTC](d-f.md#eofttc), [PSTN](o-r.md#pstn). - -### SONET -**Synchronous Optical Network.** The North American standard for synchronous multiplexing over fibre, equivalent to SDH, with rates starting at OC-1 (51.84 Mbit/s). - -See also: [SDH](#sdh). - -### Source Code -The human-readable instructions of software, written in a programming language, that a compiler or interpreter turns into something a computer can run. - -See also: [Machine code](l-n.md#machine-code), [Version Control](#version-control). - -### SPEEX -**Speex.** An open, patent-free speech codec from the Xiph.Org Foundation, based on CELP. Opus has replaced it. - -See also: [OPUS](o-r.md#opus), [Vorbis](#vorbis). - -### SPI -**Serial Peripheral Interface.** A synchronous, full-duplex serial bus for short-distance links between a microcontroller and peripherals such as sensors, flash memory and displays. It uses a clock, two data lines and a chip select for each device. - -See also: [MCU](l-n.md#mcu). - -### Spiral Lifecycle -A lifecycle made of a series of small projects, each tackling the biggest remaining risks, until the system is delivered. - -See also: [Spiral Model](#spiral-model), [Lifecycle](l-n.md#lifecycle). - -### Sponsor -See [Project Sponsor](o-r.md#project-sponsor). - -### Spiral Model -Barry Boehm's risk-driven process model. Each loop of the spiral sets objectives, identifies and resolves risks, develops and tests, and plans the next loop. It can borrow from other models, such as waterfall or prototyping, as the risks suggest. - -See also: [Spiral Lifecycle](#spiral-lifecycle), [Waterfall](#waterfall). - -### Sprint Planning -The Scrum event at the start of a sprint, in which the team agrees the sprint goal and chooses the backlog items to work on. - -See also: [Sprints](#sprints), [Scrum](#scrum). - -### Sprint retrospective -The Scrum event at the end of a sprint, in which the team reviews how it worked and agrees improvements. - -See also: [Heartbeat Retrospective](g-k.md#heartbeat-retrospective), [Scrum](#scrum). - -### Sprints -Fixed-length iterations, usually two to four weeks, in which a Scrum team produces a usable increment. - -See also: [Iteration](g-k.md#iteration), [Timebox](#timebox), [Scrum](#scrum). - -### SQL -**Structured Query Language.** The standard language for defining, querying and changing data in relational databases. - -See also: [RDBMS](o-r.md#rdbms), [NoSQL](l-n.md#nosql). - -### SS7 -**Signalling System No. 7.** The ITU-T set of protocols that telephone networks use to set up and clear calls and to provide services such as number translation, number portability, prepaid and SMS. In the UK it is also called C7. Its layers include MTP, SCCP, TCAP, ISUP and MAP. - -See also: [MTP](l-n.md#mtp), [SCCP](#sccp), [ISUP](g-k.md#isup), [TCAP](#tcap), [MAP](l-n.md#map), [CCS](a-c.md#ccs). - -### SSADM -**Structured Systems Analysis and Design Method.** A waterfall method for analysing and designing information systems, developed for and widely used in UK government from the 1980s. - -See also: [Waterfall](#waterfall), [CCTA](a-c.md#ccta). - -### SSH -**Secure Shell.** A cryptographic protocol for logging in to and running commands on remote machines securely over an untrusted network. It also tunnels other traffic and carries SFTP. - -See also: [SFTP](#sftp). - -### SSID -**Service Set Identifier.** The name of a Wi-Fi network. - -See also: [Wi-Fi](#wi-fi). - -### SSIS -**SQL Server Integration Services.** The Microsoft SQL Server component for data integration and ETL workflows. - -See also: [ETL](d-f.md#etl). - -### SSL -**Secure Sockets Layer.** The predecessor of TLS. All SSL versions are broken and must not be used, though "SSL" is still often said when TLS is meant. See [TLS](#tls). - -### Stack -A protocol stack: the layered set of protocols that together provide communication, such as Ethernet, IP, TCP and HTTP, or the software that implements them. - -```mermaid -flowchart TB - A["HTTP (application)"] --> B["TCP (transport)"] --> C["IP (network)"] --> D["Ethernet (link)"] --> E["Cable (physical)"] -``` - -[[CAPTION:Figure]] A typical protocol stack, from application to physical layer. - -See also: [Internet Protocol Suite](g-k.md#internet-protocol-suite), [L1](l-n.md#l1), [L2](l-n.md#l2). - -### Staff Days -A measure of effort: how much work an activity needs, in days of one person's time, as distinct from calendar days. - -See also: [Calendar Days](a-c.md#calendar-days). - -### Stage -Often a synonym for phase. More precisely, a phase can be divided into stages. - -See also: [Phase](o-r.md#phase). - -### Staged Delivery Lifecycle -A lifecycle that delivers a system in stages, with the most important functions first. - -See also: [Evolutionary Delivery Lifecycle](d-f.md#evolutionary-delivery-lifecycle), [Incremental Development](g-k.md#incremental-development). - -### Stakeholder -A person or organisation with an interest in a project's outcome, such as its customers, users and sponsors, and any group it affects. - -### Standard -An agreed, documented set of rules, conventions or requirements that work must follow. - -See also: [Construction Standard](a-c.md#construction-standard). - -### Statistical Estimation -Estimating with a statistical model built from historical data, usually predicting effort and time from size. It works best when calibrated on similar past projects. - -See also: [Parametric Estimation](o-r.md#parametric-estimation). - -### STB -**Set-Top Box.** A device that receives television signals, by aerial, satellite, cable or IP, and turns them into pictures and sound on a TV. - -See also: [IPTV](g-k.md#iptv), [DTT](d-f.md#dtt). - -### Story Mapping -Arranging user stories in two dimensions: along the user's journey from left to right, and by priority from top to bottom. The map helps plan releases. - -See also: [User Stories](#user-stories). - -### Story Splitting -Breaking a user story into smaller stories, each of which still has value of its own. - -See also: [User Stories](#user-stories), [Epic](d-f.md#epic). - -### Storyboard -A sequence of pictures showing a scenario or use case. - -See also: [Scenario](#scenario), [Low Fidelity Prototype](l-n.md#low-fidelity-prototype). - -### STRIDE Threat Model -Microsoft's mnemonic for six kinds of security threat: Spoofing, Tampering, Repudiation, Information disclosure, Denial of service and Elevation of privilege. Each one threatens a security property: authenticity, integrity, non-repudiation, confidentiality, availability and authorisation. - -See also: [Pen testing](o-r.md#pen-testing). - -### STP -**Spanning Tree Protocol.** The IEEE 802.1D protocol that stops loops in Ethernet networks by blocking redundant links so that one loop-free path remains, and unblocking them if the active path fails. Radia Perlman invented the algorithm. - -See also: [RSTP](o-r.md#rstp), [Switch](#switch). - -### Strategy -The overall approach taken to a problem or goal. - -### STUN -**Session Traversal Utilities for NAT.** A protocol (RFC 8489) with which a device behind a NAT discovers the public address and port the NAT has given it. ICE, SIP and WebRTC use it. - -See also: [ICE](g-k.md#ice), [NAT](l-n.md#nat). - -### Subversion -**SVN.** Apache Subversion, a centralised version control system designed as a successor to CVS. - -See also: [Git](g-k.md#git), [Version Control](#version-control). - -### Sustainable Pace -Working at a pace the team could keep up indefinitely, without overtime and burnout. - -See also: [XP](#xp). - -### SVC -**Switched Virtual Circuit.** A virtual circuit set up on demand for a call or transfer and cleared when it ends. - -See also: [PVC](o-r.md#pvc), [ATM](a-c.md#atm), [X.25](#x25). - -### Swift -Apple's compiled, multi-paradigm language for its platforms, released in 2014 as the successor to Objective-C. It is open source and also runs on Linux. - -See also: [Objective C](o-r.md#objective-c). - -### Switch -A network device that connects devices on a LAN and forwards each frame only to the port where its destination MAC address is. It learns those addresses from the traffic it sees. A layer-3 switch can also route. - -See also: [Hub](g-k.md#hub), [Router](o-r.md#router), [VLAN](#vlan). - -### System -A set of hardware, software and processes that work together. In a project, the system is what the project was set up to build. - -### System Test -A test of the whole system's behaviour, using a project build in the system test environment. - -See also: [Integration Test](g-k.md#integration-test), [System Test Environment](#system-test-environment). - -### System Test Environment -An environment that copies the live environment and its conditions as closely as possible, for system testing. - -See also: [System Test](#system-test). - -### SystemX -**System X.** The UK's national digital telephone exchange system, developed by the Post Office (later BT) with GEC, Plessey and STC, and introduced in the 1980s. It includes concentrators, digital local exchanges (DLEs) and trunk exchanges (DMSUs). - -See also: [DLE](d-f.md#dle), [PSTN](o-r.md#pstn). - -## T - -### T-Shirt Sizing -Estimating by sorting items into rough relative sizes, such as extra small, small, medium, large and extra large, rather than giving false precision. - -See also: [Relative Estimation](o-r.md#relative-estimation), [Points](o-r.md#points). - -### Task -A small, distinct unit of work in a project. - -See also: [Activity](a-c.md#activity), [Work Item](#work-item). - -### Task Board -A board with columns such as "To do", "In progress" and "Done", on which cards for tasks move as work progresses. - -See also: [Kanban Board](g-k.md#kanban-board), [Information Radiators](g-k.md#information-radiators). - -### Task Estimate -An estimate for a single task, usually made during planning and tracking and fed into work plans and detailed schedules. - -See also: [Project Estimate](o-r.md#project-estimate), [Estimate](d-f.md#estimate). - -### TCAP -**Transaction Capabilities Application Part.** The SS7 layer that manages dialogues between applications, such as MAP, INAP and CAP, using transaction IDs to run many at once. It encodes messages with ASN.1 BER. - -See also: [SS7](#ss7), [MAP](l-n.md#map), [INAP](g-k.md#inap), [CAP](a-c.md#cap). - -### Tcl/tk -Tcl is a dynamic scripting language, and Tk is the cross-platform GUI toolkit that grew up with it. Together they offer a quick way to build desktop interfaces, and other languages, such as Python's Tkinter, also use Tk. - -### TCP -**Transmission Control Protocol.** The internet's main transport protocol (RFC 9293). It gives applications a reliable, ordered, error-checked byte stream over IP, with flow and congestion control. The web, email and file transfer use it. - -See also: [UDP](#udp), [Internet Protocol Suite](g-k.md#internet-protocol-suite), [Out-of-order delivery](o-r.md#out-of-order-delivery). - -### Tcsh -An enhanced C shell that adds command-line editing and completion. It was long the default shell for the root user on FreeBSD. - -See also: [Csh](a-c.md#csh). - -### TDD -**Test-Driven Development.** Writing a failing unit test first, then the simplest code that passes it, then refactoring, in short repeated cycles. - -See also: [Unit Testing](#unit-testing), [Refactoring](o-r.md#refactoring), [BDD](a-c.md#bdd). - -### TDM -**Time-Division Multiplexing.** Sharing a channel by giving each signal its own repeating time slot. Digital telephony carries each call in a 64 kbit/s slot, for example 30 calls on an E1. - -See also: [Multiplex](l-n.md#multiplex), [PDH](o-r.md#pdh), [SDH](#sdh). - -### Team -In agile terms, a small group of people working on the same product or goal, nearly all of them full time. - -### Team Room -A space set aside for a team for the length of a project, so that its members, including the product owner, can work together closely. - -See also: [Information Radiators](g-k.md#information-radiators). - -### Technical Debt -The future cost of rework caused by choosing a quick solution now over a better one that would take longer. Like money, it gathers interest: left unpaid, it makes every later change harder. - -See also: [Refactoring](o-r.md#refactoring). - -### Template -An outline or framework for creating an artefact. - -### Test -A single test case, or testing in general. - -See also: [Testing](#testing), [Test Case](#test-case). - -### Test Activity -A specific test procedure, combining one or more test techniques and test types. - -### Test Case -**TC.** The inputs, steps and expected results that check whether a feature works or a requirement is met. - -See also: [Test Case Specification](#test-case-specification), [Test Plan](#test-plan). - -### Test Case Specification -**TCS.** A document listing the test cases that verify one or more features. - -See also: [Test Case](#test-case). - -### Test Design -Working out how to test a product: linking test cases to its requirements and design, when testing needs real analysis and structure. - -See also: [Test Design Specification](#test-design-specification). - -### Test Design Specification -**TDS.** A document recording the test design for a group of test cases. - -See also: [Test Design](#test-design). - -### Test Level -See [Test Type](#test-type). - -### Test Plan -**TP.** A document setting out the scope, approach, resources, test items and schedule of testing. - -See also: [Quality Plan](o-r.md#quality-plan), [Test Case](#test-case). - -### Test Technique -The method used in a test activity, such as boundary value analysis or equivalence partitioning. - -### Test Type -A category of testing, such as unit, integration, system or acceptance testing, defined by what it tests, how, why and when. - -### Testing -Running software to find defects and to show that it meets its requirements. - -See also: [Quality Control](o-r.md#quality-control), [Test Plan](#test-plan). - -### Three Amigos -A meeting of the three main perspectives on a piece of work, business, development and testing, to agree what it should do before, during and after development. - -See also: [ATDD](a-c.md#atdd), [BDD](a-c.md#bdd). - -### Three C's -**Card, Conversation, Confirmation.** The three parts of a user story: a short note on a card, the conversation that fills in the detail, and the acceptance tests that confirm it is done. - -See also: [User Stories](#user-stories). - -### Three Questions -The traditional structure of the daily stand-up: what did I complete, what will I do next, and what is in my way? - -See also: [Daily Scrum](d-f.md#daily-scrum). - -### TIA -**Telecommunications Industry Association.** The US trade body, accredited by ANSI, that develops standards such as TIA-568 for structured cabling. - -See also: [EIA](d-f.md#eia), [ANSI](a-c.md#ansi). - -### Timebox -A fixed, agreed period in which a person or team works towards a goal, and which ends on time whether or not everything is done. - -See also: [Sprints](#sprints), [Iteration](g-k.md#iteration). - -### TKIP -**Temporal Key Integrity Protocol.** The stop-gap Wi-Fi security protocol, from 802.11i, that replaced WEP on existing hardware. It is no longer secure and was deprecated in 2012. - -See also: [WEP](#wep), [WPA2](#wpa2). - -### TLS -**Transport Layer Security.** The protocol that encrypts and authenticates connections between applications, such as HTTPS. A handshake authenticates the server, usually with an X.509 certificate, and agrees keys. Symmetric encryption then protects the data. TLS 1.3 (RFC 8446) is current, and it replaced SSL. - -```mermaid -sequenceDiagram - participant C as Client - participant S as Server - C->>S: ClientHello (versions, ciphers, key share) - S->>C: ServerHello, certificate, Finished - C->>S: Finished - C-->>S: Encrypted application data -``` - -[[CAPTION:Figure]] A TLS 1.3 handshake, simplified. - -See also: [HTTPS](g-k.md#https), [DTLS](d-f.md#dtls), [X.509](#x509), [SSL](#ssl). - -### TLV -**Type-Length-Value.** An encoding in which each element has a type code, a length and then the value. Receivers can skip elements they do not understand, so formats can grow. BER, IS-IS and many other protocols use it. - -See also: [BER](a-c.md#ber). - -### TOGAF -**The Open Group Architecture Framework.** A widely used enterprise architecture framework. Its Architecture Development Method (ADM) guides architecture work across the business, data, application and technology domains. - -See also: [Architecture](a-c.md#architecture), [ADR](a-c.md#adr). - -### Token Ring -A LAN technology from IBM, standardised as IEEE 802.5. A token passes round a ring of stations, and only the station holding it may transmit. There are therefore no collisions. Ethernet replaced it. - -See also: [FDDI](d-f.md#fddi). - -### TOML -**Tom's Obvious, Minimal Language.** A configuration file format, created by Tom Preston-Werner, designed to be easy to read and to map clearly to a dictionary. Python's `pyproject.toml` and Rust's `Cargo.toml` use it. - -See also: [YAML](#yaml), [JSON](g-k.md#json). - -### Top Level Schedule -See [Business Schedule](a-c.md#business-schedule). - -### Top-Down Estimation -Estimating a whole system first, then dividing the total among its parts. - -See also: [Bottom-Up Estimation](a-c.md#bottom-up-estimation), [Macro Estimation](l-n.md#macro-estimation). - -### Triple DES -**3DES, or TDEA.** DES applied three times to each block with two or three keys, to extend its life after DES's 56-bit key became too short. It is slow and now deprecated in favour of AES. - -See also: [DES](d-f.md#des), [AES](a-c.md#aes). - -### TTY -**Teletypewriter.** Originally an electromechanical typewriter used as a terminal. On Unix the name now means any terminal device, including a terminal emulator window. - -See also: [Unix](#unix). - -### TrustZone -Arm's hardware security extension that splits a processor into a secure world and a normal world. The secure world holds keys, secure boot and trusted applications, isolated from the main operating system. It exists on Cortex-A and, as Armv8-M, on Cortex-M. - -See also: [Root of Trust](o-r.md#root-of-trust), [Secure Element](#secure-element). - -### TUP -**Telephone User Part.** The first SS7 call control protocol, for ordinary telephone calls without ISDN features. ISUP has largely replaced it. - -See also: [ISUP](g-k.md#isup), [SS7](#ss7). - -### Twisted pair -Cable in which the two wires of each circuit are twisted together, which cuts interference and crosstalk. It is used for telephone lines and Ethernet. - -See also: [Cat 5](a-c.md#cat-5), [Ethernet over twisted pair](d-f.md#ethernet-over-twisted-pair). - -## U - -### UA -**User Agent.** Software acting for a user. In HTTP, the browser or client, which names itself in the `User-Agent` header; in SIP, an endpoint that makes or receives calls. - -See also: [HTTP](g-k.md#http), [SIP](#sip). - -### Ubiquitous Language -In domain-driven design, a shared vocabulary drawn from the business domain, used consistently by everyone, from conversations and requirements through to the code. - -### UDP -**User Datagram Protocol.** A simple, connectionless transport protocol (RFC 768) that sends datagrams with port numbers and a checksum, but no guarantee of delivery or order. DNS, VoIP media and streaming use it, along with protocols that add their own reliability. - -[[CAPTION:Table]] The UDP header: four 16-bit fields, then the data. - -| Bits 0 to 15 | Bits 16 to 31 | -| --- | --- | -| Source port | Destination port | -| Length | Checksum | -| Data | Data | - -See also: [TCP](#tcp), [RTP](o-r.md#rtp), [DTLS](d-f.md#dtls). - -### UER -**Universal Edge Router.** Juniper's name for its broadband edge routers, which do the job of a BRAS. - -See also: [BRAS](a-c.md#bras), [BNG](a-c.md#bng). - -### UML - -**Unified Modeling Language.** - -The standard graphical language for modelling software systems, maintained by the Object Management Group. It was created by Grady Booch, Ivar Jacobson and James Rumbaugh in the mid-1990s. Its diagrams include class, sequence, activity, state and use case diagrams. - -See also: [Class Model](a-c.md#class-model), [Sequence Model](#sequence-model), [Use Case](#use-case). - -### UMTS -**Universal Mobile Telecommunications System.** The 3GPP third-generation (3G) mobile system, which evolved from GSM. It uses wideband CDMA radio, and HSPA later speeded up its data. - -See also: [W-CDMA](#w-cdma), [UTRAN](#utran), [GSM](g-k.md#gsm), [LTE (4G)](l-n.md#lte-4g). - -### UNI -**User-Network Interface.** The demarcation between a service provider's network and a customer's equipment, where the provider's responsibility ends. - -See also: [NNI](l-n.md#nni), [EVC](d-f.md#evc). - -### Unicast -Sending data from one sender to one receiver, each identified by its own address. Most internet traffic is unicast. - -See also: [Multicast](l-n.md#multicast), [Broadcast](a-c.md#broadcast), [Anycast](a-c.md#anycast). - -### Unicode -The standard that gives a unique number, or code point, to every character in the world's writing systems, plus symbols and emoji. It is kept in step with ISO/IEC 10646. UTF-8 and UTF-16 are ways of encoding it. - -See also: [UTF-8](#utf-8), [UTF-16](#utf-16), [ASCII](a-c.md#ascii). - -### Unified Communications -**UC.** Bringing an organisation's communication services, such as voice, video, messaging, presence, conferencing and voicemail, together behind one consistent user experience across devices. - -See also: [VoIP](#voip), [SIP](#sip). - -### Unit Test -The lowest level of component test: a test of one small piece of code on its own. - -See also: [Unit Testing](#unit-testing). - -### Unit Testing -Writing and running small automated tests, written by the developers, that each check a narrow piece of the code. - -See also: [Unit Test](#unit-test), [TDD](#tdd), [Mock Objects](l-n.md#mock-objects). - -### Unix -A family of multi-user, multitasking operating systems descended from the original developed at Bell Labs by Ken Thompson, Dennis Ritchie and others from 1969. BSD, Solaris, AIX and macOS are Unix systems; Linux is Unix-like. - -See also: [Linux](l-n.md#linux), [POSIX](o-r.md#posix), [Sh](#sh). - -### Update -Planned work to bring an artefact up to date with the project's current needs. - -See also: [Rework](o-r.md#rework). - -### Upstream -In a project, the early activities and artefacts, such as chartering, planning, requirements and architecture. In networking, the direction from the customer towards the network. - -See also: [Downstream](d-f.md#downstream). - -### URI -**Uniform Resource Identifier.** A string that identifies a resource (RFC 3986). URLs, which say where to find it, and URNs, which name it, are both kinds of URI. - -See also: [URL](#url), [URN](#urn). - -### URL -**Uniform Resource Locator.** A URI that says how and where to get a resource: a scheme, such as `https`, a host, a path and optional query and fragment, as in `https://example.com/page?id=1#top`. - -See also: [URI](#uri), [URN](#urn), [HTTP](g-k.md#http). - -### URN -**Uniform Resource Name.** A URI that names a resource permanently, without saying where it is, as in `urn:isbn:0451450523`. - -See also: [URI](#uri), [URL](#url). - -### Usability Testing -Watching real users try to do realistic tasks with a product, to find where it is hard to use. - -See also: [User Experience](#user-experience). - -### Use Case -A description of how a user (actor) interacts with a system to achieve a goal, including the main path and alternatives. - -See also: [Use Case Model](#use-case-model), [Use Case Specification](#use-case-specification), [Nominal Path](l-n.md#nominal-path). - -### Use Case Model -The set of use cases and actors that describes a system's behaviour from the outside. - -See also: [Use Case](#use-case), [UML](#uml). - -### Use Case Specification -The document that describes one use case in full. - -See also: [Use Case](#use-case). - -### User Experience -**UX.** The whole of a person's experience of using a product, including but going beyond its user interface: how useful, easy and pleasant it is. - -See also: [User Interface](#user-interface), [Usability Testing](#usability-testing), [Personas](o-r.md#personas). - -### User Interface -**UI.** The part of a system through which people interact with it, such as screens, controls and messages. - -See also: [User Interface Design](#user-interface-design), [User Experience](#user-experience). - -### User Interface Design -Designing how people interact with a system: layout, controls, flows and feedback. - -See also: [User Interface](#user-interface), [User Interface Prototype](#user-interface-prototype). - -### User Interface Prototype -A prototype built to explore a user interface's behaviour or look and feel. - -See also: [High Fidelity Prototype](g-k.md#high-fidelity-prototype), [Low Fidelity Prototype](l-n.md#low-fidelity-prototype). - -### User Plane -The part of a network that carries users' traffic, as opposed to the control plane, which carries signalling. In routers it is also called the data plane or forwarding plane. In mobile networks, separating the two planes lets each scale on its own. - -See also: [Data Plane](d-f.md#data-plane), [Control Plane](a-c.md#control-plane), [Management Plane](l-n.md#management-plane). - -### User Stories -Short descriptions of functionality from the user's point of view, agreed with the customer or product owner, that the team uses to divide up the work. - -See also: [Role-feature-reason](o-r.md#role-feature-reason), [INVEST](g-k.md#invest), [Three C's](#three-cs), [Epic](d-f.md#epic). - -### UTC -**Coordinated Universal Time.** The world's primary time standard, kept by atomic clocks and adjusted with leap seconds to stay close to the Earth's rotation. It has no daylight saving time, and for everyday purposes it is the same as GMT. - -See also: [NTP](l-n.md#ntp). - -### UTF-16 -A Unicode encoding that uses one 16-bit unit for characters in the Basic Multilingual Plane and two units, a surrogate pair, for the rest. Windows, Java and JavaScript use it internally. UTF-32, by contrast, uses a fixed 32 bits for every code point. - -See also: [Unicode](#unicode), [UTF-8](#utf-8). - -### UTF-8 -The most widely used Unicode encoding, designed by Ken Thompson and Rob Pike. It uses one to four bytes per character, and the first 128 characters are the same as ASCII, so ASCII text is valid UTF-8. It is the default encoding of the web. - -See also: [Unicode](#unicode), [UTF-16](#utf-16), [ASCII](a-c.md#ascii). - -### UTRAN -**UMTS Terrestrial Radio Access Network.** The 3G radio access network: the Node Bs and the RNCs that control them. - -See also: [Node B](l-n.md#node-b), [RNC](o-r.md#rnc), [UMTS](#umts), [E-UTRAN](d-f.md#e-utran). - -## V - -### V Model -A development model that extends waterfall by pairing each specification and design stage with the test stage that checks it. Requirements pair with acceptance testing, and so on down to unit tests. Drawn out, the stages form a V. - -```mermaid -flowchart LR - R[Requirements] --> SD[System design] --> AD[Architecture] --> MD[Module design] --> C[Coding] - C --> UT[Unit testing] --> IT[Integration testing] --> ST[System testing] --> AT[Acceptance testing] - R -. verified by .- AT - SD -. verified by .- ST - AD -. verified by .- IT - MD -. verified by .- UT -``` - -[[CAPTION:Figure]] The V model: each design stage is verified by a matching test stage. - -See also: [Waterfall](#waterfall), [Testing](#testing). - -### V.90 {#v90} -The ITU-T standard, from 1998, for 56 kbit/s dial-up modems. It achieves 56 kbit/s downstream and 33.6 kbit/s upstream by using the digital connection at the network end. - -See also: [V.92](#v92). - -### V.92 {#v92} -The ITU-T successor to V.90, which raised upstream speed to 48 kbit/s and added quicker connection and modem-on-hold. - -See also: [V.90](#v90). - -### VAD -**Voice Activity Detection.** Detecting whether speech is present in an audio signal. VoIP and mobile systems use it to stop sending during silence, which saves bandwidth, and generate comfort noise at the other end. - -See also: [CNG](a-c.md#cng), [VoIP](#voip). - -### VB -**Visual Basic.** Microsoft's event-driven language and IDE, first released in 1991, for building Windows applications quickly from forms and components. Classic VB became legacy in 2008; VB.NET is its .NET successor. - -See also: [BASIC](a-c.md#basic), [VBS](#vbs), [RAD](o-r.md#rad). - -### VBS -**VBScript.** Microsoft's scripting language based on Visual Basic, used for Windows administration and old web pages. Microsoft is removing it from Windows. - -See also: [VB](#vb). - -### VC -**Virtual Circuit.** A connection across a packet network that behaves like a dedicated circuit between two endpoints, set up before data flows. X.25, Frame Relay and ATM use virtual circuits. - -See also: [PVC](o-r.md#pvc), [SVC](#svc), [VCI](#vci). - -### VCI -**Virtual Channel Identifier.** The 16-bit field in an ATM cell header that, with the VPI, identifies the virtual channel the cell belongs to. - -See also: [VPI](#vpi), [ATM](a-c.md#atm). - -### VCID -**Virtual Circuit Identifier.** The identifier that both ends of a pseudowire use to match up the same circuit, for example in an MPLS VPWS. - -See also: [PW](o-r.md#pw), [VPWS](#vpws). - -### VDSL -**Very-high-bit-rate DSL.** DSL using frequencies up to 30 MHz on short lines. VDSL2 (G.993.2) gives tens of megabits per second, and up to around 80 Mbit/s downstream from a street cabinet. It is the technology of fibre-to-the-cabinet broadband. - -See also: [ADSL](a-c.md#adsl), [G.Fast](g-k.md#gfast), [PCP](o-r.md#pcp). - -### Velocity -The amount of work, usually in story points, that a team completes in an iteration. Teams use it to forecast what they can take on next. - -See also: [Points](o-r.md#points), [Burndown Chart](a-c.md#burndown-chart). - -### Verifier -In an inspection, the person who checks that the rework has been done properly. - -See also: [Inspection](g-k.md#inspection). - -### Version -A formally released revision of an artefact, recognised by configuration management. - -See also: [Revision](o-r.md#revision), [Semantic Versioning](#semantic-versioning). - -### Version Control -Recording every change to files, such as source code, so that any earlier version can be recovered and several people can work in parallel and merge their work. Git is the most widely used tool. - -See also: [Git](g-k.md#git), [Subversion](#subversion), [SCM](#scm). - -### VHDL -**VHSIC Hardware Description Language.** A language, standardised as IEEE 1076, for describing and simulating digital hardware, from systems down to logic gates. It came out of the US Very High-Speed Integrated Circuits programme in the 1980s. Verilog is its main alternative. - -### VISP -**Virtual ISP.** An internet service provider that sells another ISP's service under its own brand. - -See also: [ISP](g-k.md#isp), [MVNO](l-n.md#mvno). - -### VLAN -**Virtual LAN.** A separate broadcast domain created on shared switches, usually by tagging frames with an 802.1Q VLAN ID, so that one physical network carries several isolated LANs. - -See also: [Dot1Q](d-f.md#dot1q), [Dot1Q trunking](d-f.md#dot1q-trunking), [Switch](#switch). - -### VLR -**Visitor Location Register.** The database, usually built into each MSC, of the subscribers currently in its area. It copies their details from their HLR. - -See also: [HLR](g-k.md#hlr), [MSC](l-n.md#msc). - -### VM -**Virtual Machine.** A software emulation of a computer, on which an operating system and applications run as if on real hardware. A hypervisor shares the physical machine between several VMs. - -See also: [Docker](d-f.md#docker), [IaaS](g-k.md#iaas). - -### VOD -**Video on Demand.** Letting viewers choose and watch video when they want, rather than at a broadcast time, by streaming or downloading. - -See also: [IPTV](g-k.md#iptv), [STB](#stb). - -### VoIP -**Voice over IP.** Carrying voice calls over IP networks, with signalling protocols such as SIP to set up calls and RTP to carry the media. - -See also: [SIP](#sip), [RTP](o-r.md#rtp), [SBC](#sbc), [VoLTE](#volte). - -### VoLTE -**Voice over LTE.** Voice calls over an LTE network as IP packets, using IMS, following the GSMA IR.92 profile. It needs no circuit-switched network, connects calls faster and supports HD voice. - -See also: [IMS](g-k.md#ims), [LTE (4G)](l-n.md#lte-4g), [AMR-WB](a-c.md#amr-wb). - -### Vorbis -An open, royalty-free lossy audio codec from the Xiph.Org Foundation, usually used in the Ogg container (Ogg Vorbis). Opus has largely replaced it. - -See also: [OPUS](o-r.md#opus), [MP3](l-n.md#mp3). - -### VPI -**Virtual Path Identifier.** The field in an ATM cell header that identifies a virtual path, a bundle of virtual channels. See [VCI](#vci). - -### VPLS -**Virtual Private LAN Service.** A provider service that joins several customer sites into one Ethernet broadcast domain over an MPLS network, using a mesh of pseudowires. It is defined in RFCs 4761 and 4762. - -See also: [L2VPN](l-n.md#l2vpn), [PW](o-r.md#pw), [VPWS](#vpws). - -### VPN -**Virtual Private Network.** A private network built over a shared or public one, such as the internet or a provider's MPLS core. It uses tunnelling and often encryption, so that remote sites or users act as if directly connected. - -See also: [IPSec](g-k.md#ipsec), [L3VPN](l-n.md#l3vpn), [L2VPN](l-n.md#l2vpn). - -### VPWS -**Virtual Private Wire Service.** A point-to-point layer-2 service, such as Ethernet, carried over an MPLS network by a pseudowire. It is also called a virtual leased line or Ethernet over MPLS (EoMPLS). - -See also: [PW](o-r.md#pw), [VPLS](#vpls), [EPWS](d-f.md#epws). - -### VRF -**Virtual Routing and Forwarding.** Several separate routing tables in one router, so that different customers or networks, even with overlapping addresses, stay apart. It is the building block of MPLS layer-3 VPNs. - -See also: [L3VPN](l-n.md#l3vpn), [CE](a-c.md#ce). - -## W - -### W-CDMA -**Wideband CDMA.** The UMTS air interface, which uses direct-sequence CDMA over pairs of 5 MHz channels. - -See also: [UMTS](#umts), [CDMA](a-c.md#cdma). - -### W3C -**World Wide Web Consortium.** The international body, founded by Tim Berners-Lee, that develops standards for the web, such as HTML, CSS and XML. - -See also: [XML](#xml), [CSS](a-c.md#css). - -### Walkthrough -An informal review in which the author takes one or more people through an artefact to find defects. - -See also: [Review](o-r.md#review), [Desk Check](d-f.md#desk-check). - -### WAN -**Wide Area Network.** A network that spans a large area, linking sites in different cities or countries, usually over carriers' circuits. The internet is the largest WAN. - -See also: [LAN](l-n.md#lan), [MAN](l-n.md#man). - -### WANET -**Wireless Ad hoc Network.** A decentralised wireless network with no routers or access points, in which every node forwards traffic for the others. A MANET is the mobile kind. - -See also: [MANET](l-n.md#manet), [Wireless Mesh Network](#wireless-mesh-network). - -### WAP -**Wireless Access Point.** A device that lets Wi-Fi devices connect to a wired network. WAP also stood for Wireless Application Protocol, an early way of browsing on mobile phones. - -See also: [Wi-Fi](#wi-fi), [Wireless Repeater](#wireless-repeater). - -### Waterfall -A sequential development model in which work flows through requirements, design, construction, testing and deployment, each finished before the next starts. Winston Royce's 1970 paper, often cited as its origin, described it as risky. - -See also: [Waterfall Lifecycle](#waterfall-lifecycle), [V Model](#v-model), [Spiral Model](#spiral-model). - -### Waterfall Lifecycle -A lifecycle that moves through ordered phases, from concept to delivery, with a review at the end of each. - -See also: [Waterfall](#waterfall). - -### WBC -**Wholesale Broadband Connect.** BT Wholesale's broadband access product on the 21CN network. ISPs connect at regional aggregation points to collect their customers' traffic. - -See also: [WBMC](#wbmc), [BT 21CN](a-c.md#bt-21cn). - -### WBCC -**Wholesale Broadband Connect Converge.** A BT Wholesale package, based on WBC, that combined voice and broadband on one bill for the ISP. - -See also: [WBC](#wbc). - -### WBMC -**Wholesale Broadband Managed Connect.** A BT Wholesale product, built on WBC, in which BT carries the traffic end to end to a single handover point. It replaced IPStream. - -See also: [WBC](#wbc), [IPSC](g-k.md#ipsc). - -### Weekly Individual Plan -A plan in which each person sets their miniature milestones for the week. - -See also: [Miniature Milestone](l-n.md#miniature-milestone). - -### WEP -**Wired Equivalent Privacy.** The original Wi-Fi security from 1997, which is easily broken. WPA and WPA2 replaced it, and it was deprecated in 2004. - -See also: [WPA2](#wpa2), [TKIP](#tkip). - -### What Requirement -See [Functional Requirement](d-f.md#functional-requirement). - -### Why Requirement -See [Business Requirement](a-c.md#business-requirement). - -### Wi-Fi -Wireless LAN technology based on the IEEE 802.11 standards. Wi-Fi is the Wi-Fi Alliance's trademark for certified products. Recent generations are numbered, such as Wi-Fi 6 (802.11ax) and Wi-Fi 7 (802.11be). - -See also: [802.11](#80211), [WAP](#wap), [WPA2](#wpa2), [SSID](#ssid). - -### Wiki -A website that its users edit together in the browser, usually with simple markup. - -See also: [Document Collaboration](d-f.md#document-collaboration), [Knowledge Management](g-k.md#knowledge-management). - -### WiMAX -**Worldwide Interoperability for Microwave Access.** Wireless broadband based on the IEEE 802.16 standards, promoted as an alternative to DSL and as a 4G candidate. LTE won out. - -See also: [LTE (4G)](l-n.md#lte-4g). - -### WIN-T -**Warfighter Information Network-Tactical.** The US Army's mobile tactical communications network, giving units secure voice, data and video on the move without fixed infrastructure. - -See also: [MANET](l-n.md#manet). - -### Wireless Mesh Network -A wireless network in which nodes connect in a mesh and relay traffic for each other, with some nodes acting as gateways to a wired network. The nodes rarely move, so routes stay stable. - -See also: [WANET](#wanet), [MANET](l-n.md#manet), [Network Topologies](l-n.md#network-topologies). - -### Wireless Repeater -A device that receives a Wi-Fi signal and rebroadcasts it to extend coverage. Also called a range extender. - -See also: [WAP](#wap). - -### Wireshark -A free, open-source packet analyser, formerly called Ethereal, used for troubleshooting networks and developing protocols. - -See also: [Packet sniffing](o-r.md#packet-sniffing). - -### WISP -**Wireless ISP.** An internet service provider whose access network is wireless, such as fixed wireless or Wi-Fi. - -See also: [ISP](g-k.md#isp). - -### WLR -**Wholesale Line Rental.** A UK product that let a provider rent a phone line from Openreach and bill the customer for line and calls. It is being withdrawn as the copper phone network closes. - -See also: [CPS](a-c.md#cps), [SMPF](#smpf), [SoGEA](#sogea). - -### WMA -**Windows Media Audio.** Microsoft's family of audio codecs, including lossy, professional, lossless and voice versions. - -See also: [MP3](l-n.md#mp3), [AAC](a-c.md#aac). - -### Work Breakdown Structure -**WBS.** A hierarchical breakdown of all a project's work into work packages. Larger projects keep it as a separate document; smaller ones hold it in the work plan or schedule. - -See also: [Work Package](#work-package), [Work Plan](#work-plan). - -### Work Item -A unit of tracked work, such as a task, story or defect. Also used for the artefact the work produces. - -See also: [Artefact](a-c.md#artefact), [Item](g-k.md#item). - -### Work Package -The lowest level of a work breakdown structure, often a tangible deliverable, which is broken down further into activities and tasks for estimating and scheduling. - -See also: [Work Breakdown Structure](#work-breakdown-structure). - -### Work Plan -A plan of the work packages and their task estimates, used to plan, track and control detailed work. - -See also: [Work Package](#work-package), [Detailed Schedule](d-f.md#detailed-schedule), [Earned Value](d-f.md#earned-value). - -### Work Product -See [Artefact](a-c.md#artefact). - -### WPA2 -**Wi-Fi Protected Access 2.** The Wi-Fi security certification, from 2004, based on the full IEEE 802.11i standard with AES encryption. It replaced WEP and WPA, and WPA3 now succeeds it. - -See also: [WEP](#wep), [TKIP](#tkip), [EAP](d-f.md#eap). - -### WPS -**Wi-Fi Protected Setup.** A Wi-Fi Alliance scheme, from 2006, for adding devices to a network with a button or PIN rather than a passphrase. The PIN method is insecure and should be disabled. - -See also: [WPA2](#wpa2). - -### Write-only code -Code so hard to read that nobody, often not even its author, can understand it later. - -See also: [Construction Standard](a-c.md#construction-standard). - -### WS-BPEL -**Web Services Business Process Execution Language.** The full name of [BPEL](a-c.md#bpel). - -### WSDL -**Web Services Description Language.** An XML language for describing a web service's interface: its operations, messages, data types and endpoints. It is mostly used with SOAP. - -See also: [SOAP](#soap), [XML Schema](#xml-schema). - -## X - -### X.25 {#x25} -An ITU-T packet-switching protocol suite for WANs, from 1976, using virtual circuits with error correction at every hop. It was widely used in the 1980s, for example for cash machines, and IP has replaced it. - -See also: [Frame Relay](d-f.md#frame-relay), [HDLC](g-k.md#hdlc). - -### X.509 {#x509} -The ITU-T standard format for public key certificates, which bind an identity, such as a host name, to a public key under a CA's signature. TLS and many other protocols use them. - -See also: [CA](a-c.md#ca), [TLS](#tls), [DER](d-f.md#der). - -### XML -**Extensible Markup Language.** A W3C text format for structured documents and data, with user-defined, nested elements. It is the basis of many formats and protocols, such as SOAP and SVG. - -See also: [XML Schema](#xml-schema), [JSON](g-k.md#json), [SGML](#sgml). - -### XML Schema -A description of the allowed structure and content of a type of XML document, used to validate documents. DTD, XSD and RELAX NG are schema languages. - -See also: [Xsd](#xsd), [XML](#xml). - -### XML-RPC -A simple remote procedure call protocol that encodes calls in XML and sends them over HTTP. SOAP grew out of it. - -See also: [JSON-RPC](g-k.md#json-rpc), [SOAP](#soap), [RPC](o-r.md#rpc). - -### XMPP -**Extensible Messaging and Presence Protocol.** An open, XML-based protocol for instant messaging and presence, originally called Jabber. It has been extended for many other real-time uses. - -See also: [SIMPLE](#simple). - -### XP -**Extreme Programming.** An agile method that focuses on engineering practices, such as pair programming, test-driven development, continuous integration, refactoring and small releases. - -See also: [Pair Programming](o-r.md#pair-programming), [TDD](#tdd), [Agile development](a-c.md#agile-development). - -### Xsd -**XML Schema Definition.** The W3C's XML schema language. It describes the elements and attributes an XML document may contain, with data types, so documents can be validated. - -See also: [XML Schema](#xml-schema). - -### XSS -**Cross-Site Scripting.** A web vulnerability in which an attacker gets a site to include their script in pages other users view. The script then runs with the site's privileges in the victims' browsers. Encoding output and a content security policy prevent it. - -See also: [Pen testing](o-r.md#pen-testing). - -## Y - -### YAML -**YAML Ain't Markup Language.** A human-readable data format that uses indentation for nesting, widely used for configuration files. YAML 1.2 is a superset of JSON. - -See also: [JSON](g-k.md#json), [TOML](#toml). - -## Z - -### ZigBee -A low-power, low-rate wireless mesh protocol based on IEEE 802.15.4, used in home automation and sensors. It is simpler and uses less power than Bluetooth or Wi-Fi. - -See also: [Bluetooth](a-c.md#bluetooth), [Wireless Mesh Network](#wireless-mesh-network). - -## Numbers - -### .NET -Microsoft's software platform for building and running applications. Code compiles to an intermediate language that the Common Language Runtime runs, with a large class library. The original .NET Framework was Windows only; modern .NET is open source and cross-platform. - -See also: [C#](a-c.md#c_2), [VB](#vb). - -### 100 Mbps (Fast Ethernet) {#100-mbps-fast-ethernet} -**Fast Ethernet.** Ethernet at 100 Mbit/s, introduced in 1995 as IEEE 802.3u. 100BASE-TX over Cat 5 cable is by far the most common form. - -See also: [Gigabit Ethernet](g-k.md#gigabit-ethernet), [Ethernet over twisted pair](d-f.md#ethernet-over-twisted-pair). - -### 5ESS {#5ess} -A digital telephone switch developed by Western Electric for AT&T, first in service in 1982. It became one of the most widely used local exchanges in the United States. - -See also: [SystemX](#systemx), [Class 5 Softswitch](a-c.md#class-5-softswitch). - -### 5G {#5g} -The fifth generation of mobile networks, standardised by 3GPP from Release 15 (2018) to meet the ITU's IMT-2020 targets. It uses the New Radio (NR) air interface and a service-based core. It offers faster broadband, low-latency services and very large numbers of IoT devices. Speeds depend heavily on the frequency band used. - -See also: [LTE (4G)](l-n.md#lte-4g), [OFDMA](o-r.md#ofdma), [MIMO](l-n.md#mimo). - -### 802.11 {#80211} -The IEEE family of standards for wireless LANs, first published in 1997, which defines the physical and MAC layers that Wi-Fi products use. Amendments add speed and features, such as 802.11n, ac, ax and be. - -See also: [Wi-Fi](#wi-fi), [IEEE](g-k.md#ieee). - -### 802.3 Physical Layer {#8023-physical-layer} -**IEEE 802.3.** The IEEE working group and family of standards that define wired Ethernet: the physical layers and the MAC. They cover speeds from 10 Mbit/s to hundreds of gigabits per second, over copper and fibre. - -See also: [Ethernet over twisted pair](d-f.md#ethernet-over-twisted-pair), [MAC](l-n.md#mac), [CSMA](a-c.md#csma). diff --git a/content/glossary/s.md b/content/glossary/s.md new file mode 100644 index 0000000..1933cab --- /dev/null +++ b/content/glossary/s.md @@ -0,0 +1,669 @@ +--- +title: S +--- + +# S + +## S-Tag +**Service VLAN tag.** In IEEE 802.1ad double tagging (Q-in-Q), the outer VLAN tag added by the service provider, with TPID 0x88A8. It sits outside the customer's own C-Tag. + +![S-Tag](images/15008934.png) + +[[CAPTION:Figure]] S-Tag. + +![S-Tag](images/14123086.png) + +[[CAPTION:Figure]] S-Tag. + +See also: [C-Tag](c.md#c-tag), [VLAN](v.md#vlan), [Dot1Q](d.md#dot1q). + +## S-GW +**Serving Gateway.** The LTE core gateway that forwards users' packets between the eNodeBs and the PGW. It anchors the user plane during handovers and buffers data for idle devices while the MME pages them. + +![S-GW](images/15237222.png) + +[[CAPTION:Figure]] S-GW. + +See also: [PGW](p.md#pgw), [MME](m.md#mme), [LTE (4G)](l.md#lte-4g). + +## SAAL +**Signalling ATM Adaptation Layer.** The ATM adaptation layer that carries signalling messages reliably between ATM switches. + +See also: [ATM](a.md#atm), [AAL5](a.md#aal5). + +## SaaS +**Software as a Service.** Software that a provider hosts and runs, and customers use over the internet, usually in a browser, on a subscription. + +![SaaS](images/15008942.png) + +[[CAPTION:Figure]] SaaS. + +![SaaS](images/15008948.png) + +[[CAPTION:Figure]] SaaS. + +See also: [PaaS](p.md#paas), [IaaS](i.md#iaas). + +## SAE +**System Architecture Evolution.** The 3GPP work that defined the LTE core network, the Evolved Packet Core: a simpler, all-IP core that also supports other access networks, such as Wi-Fi. + +See also: [LTE (4G)](l.md#lte-4g), [MME](m.md#mme), [S-GW](#s-gw), [PGW](p.md#pgw). + +## SAFe +**Scaled Agile Framework.** A framework for running agile development across many teams in a large organisation, with set roles, planning events and portfolio management. + +See also: [Scaled Agile](#scaled-agile), [LeSS](l.md#less), [DAD](d.md#dad). + +## SBC +**Session Border Controller.** A device at the edge of a VoIP network that controls the signalling, and usually the media, of calls crossing it. It provides security, NAT traversal, interworking and policing. + +![SBC](images/15008954.png) + +[[CAPTION:Figure]] SBC. + +See also: [SIP](#sip), [SIP Trunk](#sip-trunk), [NAT](n.md#nat). + +## SC-FDMA +**Single-Carrier FDMA.** A multiple access scheme, also called DFT-spread OFDMA, with a lower peak-to-average power ratio than OFDMA. That saves battery power, so LTE uses it for the uplink. + +See also: [OFDMA](o.md#ofdma), [LTE (4G)](l.md#lte-4g). + +## Scala +A statically typed language for the Java virtual machine that combines object-oriented and functional programming, and works with Java libraries. + +See also: [Java](j.md#java). + +## Scaled Agile +Agile methods applied to large projects and programmes, with many teams working on one product. + +See also: [SAFe](#safe), [LeSS](l.md#less), [Scrum of Scrums](#scrum-of-scrums). + +## SCCP +**Signalling Connection Control Part.** The SS7 layer above MTP that adds richer addressing (global title translation), connection-oriented services and segmentation. MAP and other application parts use it. + +See also: [MTP](m.md#mtp), [SS7](#ss7), [TCAP](t.md#tcap). + +## Scenario +A narrative of an activity, told as a story, often a concrete path through a use case. + +See also: [Use Case](u.md#use-case), [Storyboard](#storyboard). + +## Schedule +A plan that sets the timing, resources and dependencies of the tasks needed to finish a project or part of one. + +See also: [Business Schedule](b.md#business-schedule), [Detailed Schedule](d.md#detailed-schedule). + +## Scheduler +The part of an operating system, or a network device, that decides which task or packet gets a resource next, such as a processor or a link. Its aims include keeping resources busy, sharing them fairly and meeting deadlines or quality of service targets. + +See also: [Round Robin](r.md#round-robin), [Preemption](p.md#preemption), [RTOS](r.md#rtos). + +## SCM +**Software Configuration Management.** Tracking and controlling changes to software: version control, baselines, builds and releases. It is the software part of configuration management. + +See also: [Configuration Management](c.md#configuration-management), [Version Control](v.md#version-control). + +## Scratch +A visual, block-based programming language and online community from the MIT Media Lab, designed to teach children to code. + +## Scribe +The person who records issues in an inspection or other formal meeting. + +See also: [Inspection](i.md#inspection). + +## Scrum +An agile framework in which a small team works in fixed-length sprints towards a sprint goal. It has three roles: product owner, Scrum Master and developers. Its events are sprint planning, the daily scrum, the sprint review and the retrospective. + +See also: [Sprints](#sprints), [Product Owner](p.md#product-owner), [Scrum Master](#scrum-master). + +## Scrum Master +The Scrum role that helps the team follow Scrum and work well, by coaching, removing obstacles and protecting the team's focus. + +See also: [Scrum](#scrum). + +## Scrum of Scrums +A way to coordinate several Scrum teams: each team sends a representative to a regular meeting to raise dependencies and problems between teams. + +See also: [Scaled Agile](#scaled-agile). + +## SCTP +**Stream Control Transmission Protocol.** A transport protocol (RFC 9260) that delivers messages reliably like TCP, keeps message boundaries like UDP, and supports multiple streams and multi-homing. It carries SS7 signalling over IP (SIGTRAN) and LTE control traffic. + +See also: [TCP](t.md#tcp), [UDP](u.md#udp), [SGW](#sgw). + +## SDES +**SDP Security Descriptions.** A way to exchange the keys for SRTP (secure RTP) inside the SDP of a SIP call. It relies on the signalling itself being encrypted. + +See also: [SDP](#sdp), [RTP](r.md#rtp), [SIP](#sip). + +## SDH +**Synchronous Digital Hierarchy.** The ITU-T standard for synchronous multiplexing of digital signals over fibre, from STM-1 (155 Mbit/s) upwards, with strong management and fast protection switching. It replaced PDH; SONET is the North American equivalent. + +See also: [SONET](#sonet), [PDH](p.md#pdh), [EoS](e.md#eos). + +## SDK +**Software Development Kit.** The libraries, tools, documentation and examples needed to build software for a particular platform or service. + +See also: [API](a.md#api), [DDK](d.md#ddk). + +## SDLC +**Systems Development Life Cycle.** A structured process for developing a system, from feasibility study through to maintenance. Waterfall, spiral and iterative models are all versions of it. + +See also: [Lifecycle](l.md#lifecycle), [Waterfall](w.md#waterfall). + +## SDMA +**Space-Division Multiple Access.** Using smart antennas to point beams at different users, so that users in different places can share the same frequency at the same time. + +See also: [MIMO](m.md#mimo), [CDMA](c.md#cdma). + +## SDN +**Software-Defined Networking.** Separating a network's control plane from its data plane, and putting control in software controllers that configure the forwarding devices through open interfaces. The network is then managed centrally and programmatically. + +See also: [Control Plane](c.md#control-plane), [Data Plane](d.md#data-plane), [NaaS](n.md#naas). + +## SDP +**Session Description Protocol.** A text format (RFC 8866) for describing multimedia sessions: media types, codecs, addresses and ports. SIP uses it in an offer and answer exchange to agree the media of a call. + +See also: [SIP](#sip), [SDES](#sdes), [RTP](r.md#rtp). + +## SDS +**Software Design Specification.** A document that sets out a system's detailed design: its modules, interfaces, data models, state machines, timing and memory layout. Ideally it is kept up to date alongside the code. + +See also: [Software Requirements Specification](#software-requirements-specification), [Low Level Design](l.md#low-level-design). + +## SDSL +**Symmetric Digital Subscriber Line.** DSL with the same speed in both directions, aimed at businesses. HDSL and G.SHDSL are examples. + +See also: [ADSL](a.md#adsl), [HDSL](h.md#hdsl), [DSL](d.md#dsl). + +## Sed +**Stream editor.** A Unix tool, written at Bell Labs in the 1970s, that transforms text line by line using a small command language, most often for substitutions with regular expressions. + +See also: [Awk](a.md#awk), [Regular expression](r.md#regular-expression). + +## Secure Element +**SE.** A tamper-resistant chip that stores keys and runs sensitive code in isolation, with protection against physical and side-channel attacks. SIM cards, payment cards and many IoT devices use one. + +See also: [Root of Trust](r.md#root-of-trust), [TrustZone](t.md#trustzone), [SIM](#sim). + +## SE +See [Secure Element](#secure-element). + +## Semaphore +A synchronisation primitive: a counter that threads decrement to take one unit of a resource, waiting if none is left, and increment to release it. A binary semaphore works like a lock. + +See also: [Mutex](m.md#mutex), [Mutual Exclusion](m.md#mutual-exclusion). + +## Semantic Versioning +**SemVer.** A version numbering scheme, `MAJOR.MINOR.PATCH`. Increase the major number for incompatible API changes, the minor number for compatible new features and the patch number for compatible bug fixes. Pre-release and build labels can follow, as in `2.1.0-rc.1+build.5`. + +![Semantic Versioning](images/56033307.png) + +[[CAPTION:Figure]] Semantic Versioning. + +See also: [Release Management](r.md#release-management), [Version](v.md#version). + +## Sequence Model +A model, such as a UML sequence diagram, that shows the messages exchanged between objects or roles in time order. + +See also: [Collaboration Model](c.md#collaboration-model), [UML](u.md#uml). + +## SF +**Superframe.** The T1 framing format that groups 12 frames and uses their framing bits for alignment. Extended Superframe (ESF), with 24 frames, replaced it. + +See also: [PDH](p.md#pdh), [TDM](t.md#tdm). + +## SFTP +**SSH File Transfer Protocol.** A protocol for transferring and managing files over an SSH connection. It is unrelated to FTP and FTPS, apart from its purpose. + +See also: [SSH](#ssh), [FTPS](f.md#ftps), [FTP](f.md#ftp). + +## SGML +**Standard Generalized Markup Language.** The ISO standard (ISO 8879) for defining markup languages that describe a document's structure rather than its appearance. HTML and XML derive from it. + +See also: [XML](x.md#xml). + +## SGSN +**Serving GPRS Support Node.** The GPRS and UMTS core node that serves devices in its area. It tracks their location, authenticates them and routes their packets to and from the GGSN. + +![SGSN](images/15302771.png) + +[[CAPTION:Figure]] SGSN. + +See also: [GGSN](g.md#ggsn), [GPRS](g.md#gprs), [MME](m.md#mme). + +## SGW +**Signalling Gateway.** A gateway that carries SS7 signalling between circuit-switched networks and IP networks, for example passing ISUP from an MGCF over SCTP (SIGTRAN) to the SS7 network over MTP. + +![SGW](images/15138956.png) + +[[CAPTION:Figure]] SGW. + +See also: [MGCF](m.md#mgcf), [SCTP](#sctp), [SS7](#ss7). + +## Sh +The Bourne shell, the standard shell of Version 7 Unix, written by Stephen Bourne. `/bin/sh` on Unix-like systems today is a shell compatible with it, as POSIX defines. + +See also: [Bash](b.md#bash), [Ksh](k.md#ksh), [Shell Scripting](#shell-scripting), [POSIX](p.md#posix). + +## SHA +**Secure Hash Algorithm.** A family of cryptographic hash functions published by NIST. SHA-1 (160-bit) is broken and should not be used for security. SHA-2 (SHA-256, SHA-512) and SHA-3 are current. + +See also: [Hash Function](h.md#hash-function), [MD5](m.md#md5). + +## Shell Scripting +Writing scripts for a command-line shell, such as Bash, to automate tasks like handling files and running other tools. + +See also: [Bash](b.md#bash), [Sh](#sh). + +## Sidetone +The small part of a speaker's own voice fed back into their earpiece, so that a phone line sounds alive and people do not shout. + +See also: [Echo](e.md#echo). + +## Sign Up +**Signing up for tasks.** In agile teams, members choose the tasks they will work on, rather than a manager assigning them. + +See also: [Task Board](t.md#task-board). + +## SIM +**Subscriber Identity Module.** The smart card, or embedded chip (eSIM), that holds a mobile subscriber's IMSI and secret key. The network uses them to identify and authenticate the subscriber. + +![SIM](images/15008960.png) + +[[CAPTION:Figure]] SIM. + +![SIM](images/15008966.png) + +[[CAPTION:Figure]] SIM. + +See also: [IMSI](i.md#imsi), [Secure Element](#secure-element), [HLR](h.md#hlr). + +## SIMPLE +**SIP for Instant Messaging and Presence Leveraging Extensions.** The IETF's SIP-based protocols for instant messaging and presence. XMPP is the main alternative. + +See also: [SIP](#sip), [XMPP](x.md#xmpp). + +## Simple Design +The agile practice of keeping the design as simple as the current needs allow, guided by rules such as Kent Beck's. + +See also: [Rule of Simplicity](r.md#rule-of-simplicity), [KISS Principle](k.md#kiss-principle), [Quick Design Session](q.md#quick-design-session). + +## Simplex +Communication in one direction only, such as broadcast radio. ITU usage also applies the term to links that work one way at a time, which most people call half duplex. + +![Simplex](images/15008972.png) + +[[CAPTION:Figure]] Simplex. + +See also: [Duplex](d.md#duplex), [Half Duplex](h.md#half-duplex). + +## Single Step Test +A test in a debugger, stepping through the source code line by line as it runs. + +## SIP +**Session Initiation Protocol.** The IETF's text-based signalling protocol (RFC 3261) for setting up, changing and ending voice, video and messaging sessions. It is modelled on HTTP and carries SDP to agree the media. SIP is the basis of most VoIP and of IMS. + +![SIP](images/15008978.png) + +[[CAPTION:Figure]] SIP. + +![SIP](images/15008984.png) + +[[CAPTION:Figure]] SIP. + +See also: [SDP](#sdp), [RTP](r.md#rtp), [IMS](i.md#ims), [SIP Trunk](#sip-trunk). + +## SIP Trunk +A VoIP connection, using SIP, between a business's phone system and a service provider, replacing ISDN lines for external calls. + +See also: [SIP](#sip), [SBC](#sbc), [ISDN](i.md#isdn). + +## SISO +**Single-Input, Single-Output.** A system with one input and one output; in radio, a link with one antenna at each end. + +See also: [MIMO](m.md#mimo). + +## Skew +**Clock skew.** The difference in the arrival time of the same clock signal at different parts of a circuit, or between the readings of two clocks. + +See also: [Jitter](j.md#jitter). + +## SLA +**Service Level Agreement.** An agreement between a provider and a customer on the service's measurable levels, such as availability, response and repair times, with remedies if they are missed. + +See also: [KPI](k.md#kpi), [MTR](m.md#mtr). + +## SLIP +**Serial Line Internet Protocol.** A very simple way of sending IP packets over a serial line (RFC 1055), with no negotiation or error checking. PPP replaced it, though it is still used on small microcontrollers. + +See also: [PPP](p.md#ppp). + +## SME +**Subject Matter Expert.** A person with deep knowledge of a particular area, often a business domain that the software serves. SME also commonly means small and medium-sized enterprise. + +## Smoke Test +A quick test that a new build works at all, run before deeper testing. + +See also: [Smoke testing](#smoke-testing), [Build](b.md#build). + +## Smoke testing +Quick, shallow testing of a system's most important functions, to decide whether it is stable enough for further testing. + +See also: [Smoke Test](#smoke-test), [Regression testing](r.md#regression-testing). + +## SMP +**Symmetric Multiprocessing.** A design in which two or more identical processors, or cores, share one memory and are run by one operating system that treats them all equally. Most multi-core computers work this way. + +![SMP](images/15008990.png) + +[[CAPTION:Figure]] SMP. + +See also: [AMP](a.md#amp). + +## SMPF +**Shared Metallic Path Facility.** An Openreach product that shares one copper line between two providers. Typically one provides voice over the low frequencies and the other broadband over the high frequencies. + +See also: [MPF](m.md#mpf), [WLR](w.md#wlr). + +## SMSC +**Short Message Service Centre.** The network element that stores SMS messages and forwards them to their destination, retrying until the recipient is reachable. + +See also: [MAP](m.md#map), [GSM](g.md#gsm). + +## Snake Case +Writing compound names in lower case with underscores between the words, as in `snake_case`. Python uses it for functions and variables. + +See also: [Camel Case](c.md#camel-case). + +## SNMP +**Simple Network Management Protocol.** The IETF protocol for monitoring and managing network devices. Managers read and set variables in each device's MIB, and devices send traps when events happen. SNMPv3 adds proper security. + +![SNMP](images/15008996.png) + +[[CAPTION:Figure]] SNMP. + +See also: [MIB](m.md#mib), [Management Plane](m.md#management-plane), [Cacti](c.md#cacti). + +## SNTP +**Simple Network Time Protocol.** A simpler form of NTP, without NTP's long-term state and filtering, for devices that need only modest clock accuracy. + +See also: [NTP](n.md#ntp). + +## SOA +**Service-Oriented Architecture.** An architecture style in which applications are built from services. Each service provides a business function through a network interface, can be used without knowing how it works, and can be combined with others. + +See also: [ESB](e.md#esb), [Microservice](m.md#microservice), [SOAP](#soap). + +## SOAP +An XML-based messaging protocol for web services, usually over HTTP. A SOAP envelope holds a header and a body, and WSDL describes the services. REST has largely replaced it for new APIs. + +![SOAP](images/15009002.png) + +[[CAPTION:Figure]] SOAP. + +See also: [WSDL](w.md#wsdl), [REST](r.md#rest), [XML](x.md#xml). + +## Softswitch +A call control system implemented in software on general-purpose servers, which routes calls in VoIP and next-generation networks. It controls media gateways rather than switching media itself. + +See also: [Class 4 Softswitch](c.md#class-4-softswitch), [Class 5 Softswitch](c.md#class-5-softswitch), [MGCP](m.md#mgcp), [H.248](h.md#h248). + +## Software Change Request +See [Change Request](c.md#change-request). + +## Software Configuration Management +See [SCM](#scm). + +## Software Development Plan +**SDP.** On software projects, another name for the [Project Plan](p.md#project-plan). + +## Software Engineering Body of Knowledge +**SWEBOK.** The IEEE Computer Society's guide to the generally accepted knowledge of software engineering, organised into knowledge areas. + +See also: [IEEE](i.md#ieee). + +## Software Engineering Lab +**SEL.** The Software Engineering Laboratory at NASA Goddard, which studied and improved software processes from 1976 into the 2000s. + +## Software Engineering Process Group +**SEPG.** A group that leads software process improvement in an organisation, and gives projects independent technical oversight. + +See also: [Capability Maturity Model](c.md#capability-maturity-model). + +## Software Requirements Specification +**SRS.** A document that captures the requirements for a software system, often alongside models and prototypes. + +See also: [Requirements](r.md#requirements), [SDS](#sds). + +## SOLID Principles +Five object-oriented design principles promoted by Robert C. Martin: single responsibility, open-closed, Liskov substitution, interface segregation and dependency inversion. Together they aim for code that is modular and easy to change. + +See also: [OO](o.md#oo), [Design Pattern](d.md#design-pattern). + +## SoGEA +**Single Order Generic Ethernet Access.** An Openreach broadband product over fibre to the cabinet with no phone service on the line, ordered as one product. It is the standalone broadband line used as the copper phone network closes. + +![SoGEA](images/15009008.png) + +[[CAPTION:Figure]] SoGEA. + +![SoGEA](images/15009014.png) + +[[CAPTION:Figure]] SoGEA. + +See also: [VDSL](v.md#vdsl), [EoFTTC](e.md#eofttc), [PSTN](p.md#pstn). + +## SONET +**Synchronous Optical Network.** The North American standard for synchronous multiplexing over fibre, equivalent to SDH, with rates starting at OC-1 (51.84 Mbit/s). + +See also: [SDH](#sdh). + +## Source Code +The human-readable instructions of software, written in a programming language, that a compiler or interpreter turns into something a computer can run. + +See also: [Machine code](m.md#machine-code), [Version Control](v.md#version-control). + +## SPEEX +**Speex.** An open, patent-free speech codec from the Xiph.Org Foundation, based on CELP. Opus has replaced it. + +See also: [OPUS](o.md#opus), [Vorbis](v.md#vorbis). + +## SPI +**Serial Peripheral Interface.** A synchronous, full-duplex serial bus for short-distance links between a microcontroller and peripherals such as sensors, flash memory and displays. It uses a clock, two data lines and a chip select for each device. + +See also: [MCU](m.md#mcu). + +## Spiral Lifecycle +A lifecycle made of a series of small projects, each tackling the biggest remaining risks, until the system is delivered. + +See also: [Spiral Model](#spiral-model), [Lifecycle](l.md#lifecycle). + +## Sponsor +See [Project Sponsor](p.md#project-sponsor). + +## Spiral Model +Barry Boehm's risk-driven process model. Each loop of the spiral sets objectives, identifies and resolves risks, develops and tests, and plans the next loop. It can borrow from other models, such as waterfall or prototyping, as the risks suggest. + +![Spiral Model](images/15009020.png) + +[[CAPTION:Figure]] Spiral Model. + +See also: [Spiral Lifecycle](#spiral-lifecycle), [Waterfall](w.md#waterfall). + +## Sprint Planning +The Scrum event at the start of a sprint, in which the team agrees the sprint goal and chooses the backlog items to work on. + +See also: [Sprints](#sprints), [Scrum](#scrum). + +## Sprint retrospective +The Scrum event at the end of a sprint, in which the team reviews how it worked and agrees improvements. + +See also: [Heartbeat Retrospective](h.md#heartbeat-retrospective), [Scrum](#scrum). + +## Sprints +Fixed-length iterations, usually two to four weeks, in which a Scrum team produces a usable increment. + +See also: [Iteration](i.md#iteration), [Timebox](t.md#timebox), [Scrum](#scrum). + +## SQL +**Structured Query Language.** The standard language for defining, querying and changing data in relational databases. + +See also: [RDBMS](r.md#rdbms), [NoSQL](n.md#nosql). + +## SS7 +**Signalling System No. 7.** The ITU-T set of protocols that telephone networks use to set up and clear calls and to provide services such as number translation, number portability, prepaid and SMS. In the UK it is also called C7. Its layers include MTP, SCCP, TCAP, ISUP and MAP. + +![SS7](images/15009026.png) + +[[CAPTION:Figure]] SS7. + +See also: [MTP](m.md#mtp), [SCCP](#sccp), [ISUP](i.md#isup), [TCAP](t.md#tcap), [MAP](m.md#map), [CCS](c.md#ccs). + +## SSADM +**Structured Systems Analysis and Design Method.** A waterfall method for analysing and designing information systems, developed for and widely used in UK government from the 1980s. + +See also: [Waterfall](w.md#waterfall), [CCTA](c.md#ccta). + +## SSH +**Secure Shell.** A cryptographic protocol for logging in to and running commands on remote machines securely over an untrusted network. It also tunnels other traffic and carries SFTP. + +See also: [SFTP](#sftp). + +## SSID +**Service Set Identifier.** The name of a Wi-Fi network. + +See also: [Wi-Fi](w.md#wi-fi). + +## SSIS +**SQL Server Integration Services.** The Microsoft SQL Server component for data integration and ETL workflows. + +See also: [ETL](e.md#etl). + +## SSL +**Secure Sockets Layer.** The predecessor of TLS. All SSL versions are broken and must not be used, though "SSL" is still often said when TLS is meant. See [TLS](t.md#tls). + +## Stack +A protocol stack: the layered set of protocols that together provide communication, such as Ethernet, IP, TCP and HTTP, or the software that implements them. + +![Stack](images/15009032.png) + +[[CAPTION:Figure]] Stack. + +```mermaid +flowchart TB + A["HTTP (application)"] --> B["TCP (transport)"] --> C["IP (network)"] --> D["Ethernet (link)"] --> E["Cable (physical)"] +``` + +[[CAPTION:Figure]] A typical protocol stack, from application to physical layer. + +See also: [Internet Protocol Suite](i.md#internet-protocol-suite), [L1](l.md#l1), [L2](l.md#l2). + +## Staff Days +A measure of effort: how much work an activity needs, in days of one person's time, as distinct from calendar days. + +See also: [Calendar Days](c.md#calendar-days). + +## Stage +Often a synonym for phase. More precisely, a phase can be divided into stages. + +See also: [Phase](p.md#phase). + +## Staged Delivery Lifecycle +A lifecycle that delivers a system in stages, with the most important functions first. + +See also: [Evolutionary Delivery Lifecycle](e.md#evolutionary-delivery-lifecycle), [Incremental Development](i.md#incremental-development). + +## Stakeholder +A person or organisation with an interest in a project's outcome, such as its customers, users and sponsors, and any group it affects. + +## Standard +An agreed, documented set of rules, conventions or requirements that work must follow. + +See also: [Construction Standard](c.md#construction-standard). + +## Statistical Estimation +Estimating with a statistical model built from historical data, usually predicting effort and time from size. It works best when calibrated on similar past projects. + +See also: [Parametric Estimation](p.md#parametric-estimation). + +## STB +**Set-Top Box.** A device that receives television signals, by aerial, satellite, cable or IP, and turns them into pictures and sound on a TV. + +See also: [IPTV](i.md#iptv), [DTT](d.md#dtt). + +## Story Mapping +Arranging user stories in two dimensions: along the user's journey from left to right, and by priority from top to bottom. The map helps plan releases. + +See also: [User Stories](u.md#user-stories). + +## Story Splitting +Breaking a user story into smaller stories, each of which still has value of its own. + +See also: [User Stories](u.md#user-stories), [Epic](e.md#epic). + +## Storyboard +A sequence of pictures showing a scenario or use case. + +See also: [Scenario](#scenario), [Low Fidelity Prototype](l.md#low-fidelity-prototype). + +## STRIDE Threat Model +Microsoft's mnemonic for six kinds of security threat: Spoofing, Tampering, Repudiation, Information disclosure, Denial of service and Elevation of privilege. Each one threatens a security property: authenticity, integrity, non-repudiation, confidentiality, availability and authorisation. + +See also: [Pen testing](p.md#pen-testing). + +## STP +**Spanning Tree Protocol.** The IEEE 802.1D protocol that stops loops in Ethernet networks by blocking redundant links so that one loop-free path remains, and unblocking them if the active path fails. Radia Perlman invented the algorithm. + +See also: [RSTP](r.md#rstp), [Switch](#switch). + +## Strategy +The overall approach taken to a problem or goal. + +## STUN +**Session Traversal Utilities for NAT.** A protocol (RFC 8489) with which a device behind a NAT discovers the public address and port the NAT has given it. ICE, SIP and WebRTC use it. + +See also: [ICE](i.md#ice), [NAT](n.md#nat). + +## Subversion +**SVN.** Apache Subversion, a centralised version control system designed as a successor to CVS. + +See also: [Git](g.md#git), [Version Control](v.md#version-control). + +## Sustainable Pace +Working at a pace the team could keep up indefinitely, without overtime and burnout. + +See also: [XP](x.md#xp). + +## SVC +**Switched Virtual Circuit.** A virtual circuit set up on demand for a call or transfer and cleared when it ends. + +See also: [PVC](p.md#pvc), [ATM](a.md#atm), [X.25](x.md#x25). + +## Swift +Apple's compiled, multi-paradigm language for its platforms, released in 2014 as the successor to Objective-C. It is open source and also runs on Linux. + +See also: [Objective C](o.md#objective-c). + +## Switch +A network device that connects devices on a LAN and forwards each frame only to the port where its destination MAC address is. It learns those addresses from the traffic it sees. A layer-3 switch can also route. + +See also: [Hub](h.md#hub), [Router](r.md#router), [VLAN](v.md#vlan). + +## System +A set of hardware, software and processes that work together. In a project, the system is what the project was set up to build. + +## System Test +A test of the whole system's behaviour, using a project build in the system test environment. + +See also: [Integration Test](i.md#integration-test), [System Test Environment](#system-test-environment). + +## System Test Environment +An environment that copies the live environment and its conditions as closely as possible, for system testing. + +See also: [System Test](#system-test). + +## SystemX +**System X.** The UK's national digital telephone exchange system, developed by the Post Office (later BT) with GEC, Plessey and STC, and introduced in the 1980s. It includes concentrators, digital local exchanges (DLEs) and trunk exchanges (DMSUs). + +See also: [DLE](d.md#dle), [PSTN](p.md#pstn). diff --git a/content/glossary/symbols.md b/content/glossary/symbols.md new file mode 100644 index 0000000..75fba3e --- /dev/null +++ b/content/glossary/symbols.md @@ -0,0 +1,14 @@ +--- +title: Symbols +--- + +# Symbols + +## .NET +Microsoft's software platform for building and running applications. Code compiles to an intermediate language that the Common Language Runtime runs, with a large class library. The original .NET Framework was Windows only; modern .NET is open source and cross-platform. + +![.NET](images/15007749.png) + +[[CAPTION:Figure]] .NET. + +See also: [C#](c.md#c_2), [VB](v.md#vb). diff --git a/content/glossary/t.md b/content/glossary/t.md new file mode 100644 index 0000000..babdefe --- /dev/null +++ b/content/glossary/t.md @@ -0,0 +1,240 @@ +--- +title: T +--- + +# T + +## T-Shirt Sizing +Estimating by sorting items into rough relative sizes, such as extra small, small, medium, large and extra large, rather than giving false precision. + +See also: [Relative Estimation](r.md#relative-estimation), [Points](p.md#points). + +## Task +A small, distinct unit of work in a project. + +See also: [Activity](a.md#activity), [Work Item](w.md#work-item). + +## Task Board +A board with columns such as "To do", "In progress" and "Done", on which cards for tasks move as work progresses. + +![Task Board](images/15009042.png) + +[[CAPTION:Figure]] Task Board. + +See also: [Kanban Board](k.md#kanban-board), [Information Radiators](i.md#information-radiators). + +## Task Estimate +An estimate for a single task, usually made during planning and tracking and fed into work plans and detailed schedules. + +See also: [Project Estimate](p.md#project-estimate), [Estimate](e.md#estimate). + +## TCAP +**Transaction Capabilities Application Part.** The SS7 layer that manages dialogues between applications, such as MAP, INAP and CAP, using transaction IDs to run many at once. It encodes messages with ASN.1 BER. + +See also: [SS7](s.md#ss7), [MAP](m.md#map), [INAP](i.md#inap), [CAP](c.md#cap). + +## Tcl/tk +Tcl is a dynamic scripting language, and Tk is the cross-platform GUI toolkit that grew up with it. Together they offer a quick way to build desktop interfaces, and other languages, such as Python's Tkinter, also use Tk. + +## TCP +**Transmission Control Protocol.** The internet's main transport protocol (RFC 9293). It gives applications a reliable, ordered, error-checked byte stream over IP, with flow and congestion control. The web, email and file transfer use it. + +![TCP](images/15009050.png) + +[[CAPTION:Figure]] TCP. + +See also: [UDP](u.md#udp), [Internet Protocol Suite](i.md#internet-protocol-suite), [Out-of-order delivery](o.md#out-of-order-delivery). + +## Tcsh +An enhanced C shell that adds command-line editing and completion. It was long the default shell for the root user on FreeBSD. + +See also: [Csh](c.md#csh). + +## TDD +**Test-Driven Development.** Writing a failing unit test first, then the simplest code that passes it, then refactoring, in short repeated cycles. + +See also: [Unit Testing](u.md#unit-testing), [Refactoring](r.md#refactoring), [BDD](b.md#bdd). + +## TDM +**Time-Division Multiplexing.** Sharing a channel by giving each signal its own repeating time slot. Digital telephony carries each call in a 64 kbit/s slot, for example 30 calls on an E1. + +![TDM](images/15009056.png) + +[[CAPTION:Figure]] TDM. + +See also: [Multiplex](m.md#multiplex), [PDH](p.md#pdh), [SDH](s.md#sdh). + +## Team +In agile terms, a small group of people working on the same product or goal, nearly all of them full time. + +## Team Room +A space set aside for a team for the length of a project, so that its members, including the product owner, can work together closely. + +See also: [Information Radiators](i.md#information-radiators). + +## Technical Debt +The future cost of rework caused by choosing a quick solution now over a better one that would take longer. Like money, it gathers interest: left unpaid, it makes every later change harder. + +See also: [Refactoring](r.md#refactoring). + +## Template +An outline or framework for creating an artefact. + +## Test +A single test case, or testing in general. + +See also: [Testing](#testing), [Test Case](#test-case). + +## Test Activity +A specific test procedure, combining one or more test techniques and test types. + +## Test Case +**TC.** The inputs, steps and expected results that check whether a feature works or a requirement is met. + +See also: [Test Case Specification](#test-case-specification), [Test Plan](#test-plan). + +## Test Case Specification +**TCS.** A document listing the test cases that verify one or more features. + +See also: [Test Case](#test-case). + +## Test Design +Working out how to test a product: linking test cases to its requirements and design, when testing needs real analysis and structure. + +See also: [Test Design Specification](#test-design-specification). + +## Test Design Specification +**TDS.** A document recording the test design for a group of test cases. + +See also: [Test Design](#test-design). + +## Test Level +See [Test Type](#test-type). + +## Test Plan +**TP.** A document setting out the scope, approach, resources, test items and schedule of testing. + +See also: [Quality Plan](q.md#quality-plan), [Test Case](#test-case). + +## Test Technique +The method used in a test activity, such as boundary value analysis or equivalence partitioning. + +## Test Type +A category of testing, such as unit, integration, system or acceptance testing, defined by what it tests, how, why and when. + +## Testing +Running software to find defects and to show that it meets its requirements. + +See also: [Quality Control](q.md#quality-control), [Test Plan](#test-plan). + +## Three Amigos +A meeting of the three main perspectives on a piece of work, business, development and testing, to agree what it should do before, during and after development. + +See also: [ATDD](a.md#atdd), [BDD](b.md#bdd). + +## Three C's +**Card, Conversation, Confirmation.** The three parts of a user story: a short note on a card, the conversation that fills in the detail, and the acceptance tests that confirm it is done. + +See also: [User Stories](u.md#user-stories). + +## Three Questions +The traditional structure of the daily stand-up: what did I complete, what will I do next, and what is in my way? + +See also: [Daily Scrum](d.md#daily-scrum). + +## TIA +**Telecommunications Industry Association.** The US trade body, accredited by ANSI, that develops standards such as TIA-568 for structured cabling. + +See also: [EIA](e.md#eia), [ANSI](a.md#ansi). + +## Timebox +A fixed, agreed period in which a person or team works towards a goal, and which ends on time whether or not everything is done. + +See also: [Sprints](s.md#sprints), [Iteration](i.md#iteration). + +## TKIP +**Temporal Key Integrity Protocol.** The stop-gap Wi-Fi security protocol, from 802.11i, that replaced WEP on existing hardware. It is no longer secure and was deprecated in 2012. + +See also: [WEP](w.md#wep), [WPA2](w.md#wpa2). + +## TLS +**Transport Layer Security.** The protocol that encrypts and authenticates connections between applications, such as HTTPS. A handshake authenticates the server, usually with an X.509 certificate, and agrees keys. Symmetric encryption then protects the data. TLS 1.3 (RFC 8446) is current, and it replaced SSL. + +![TLS](images/15009062.png) + +[[CAPTION:Figure]] TLS. + +```mermaid +sequenceDiagram + participant C as Client + participant S as Server + C->>S: ClientHello (versions, ciphers, key share) + S->>C: ServerHello, certificate, Finished + C->>S: Finished + C-->>S: Encrypted application data +``` + +[[CAPTION:Figure]] A TLS 1.3 handshake, simplified. + +See also: [HTTPS](h.md#https), [DTLS](d.md#dtls), [X.509](x.md#x509), [SSL](s.md#ssl). + +## TLV +**Type-Length-Value.** An encoding in which each element has a type code, a length and then the value. Receivers can skip elements they do not understand, so formats can grow. BER, IS-IS and many other protocols use it. + +![TLV](images/15009068.png) + +[[CAPTION:Figure]] TLV. + +See also: [BER](b.md#ber). + +## TOGAF +**The Open Group Architecture Framework.** A widely used enterprise architecture framework. Its Architecture Development Method (ADM) guides architecture work across the business, data, application and technology domains. + +See also: [Architecture](a.md#architecture), [ADR](a.md#adr). + +## Token Ring +A LAN technology from IBM, standardised as IEEE 802.5. A token passes round a ring of stations, and only the station holding it may transmit. There are therefore no collisions. Ethernet replaced it. + +![Token Ring](images/15009074.png) + +[[CAPTION:Figure]] Token Ring. + +See also: [FDDI](f.md#fddi). + +## TOML +**Tom's Obvious, Minimal Language.** A configuration file format, created by Tom Preston-Werner, designed to be easy to read and to map clearly to a dictionary. Python's `pyproject.toml` and Rust's `Cargo.toml` use it. + +See also: [YAML](y.md#yaml), [JSON](j.md#json). + +## Top Level Schedule +See [Business Schedule](b.md#business-schedule). + +## Top-Down Estimation +Estimating a whole system first, then dividing the total among its parts. + +See also: [Bottom-Up Estimation](b.md#bottom-up-estimation), [Macro Estimation](m.md#macro-estimation). + +## Triple DES +**3DES, or TDEA.** DES applied three times to each block with two or three keys, to extend its life after DES's 56-bit key became too short. It is slow and now deprecated in favour of AES. + +See also: [DES](d.md#des), [AES](a.md#aes). + +## TTY +**Teletypewriter.** Originally an electromechanical typewriter used as a terminal. On Unix the name now means any terminal device, including a terminal emulator window. + +See also: [Unix](u.md#unix). + +## TrustZone +Arm's hardware security extension that splits a processor into a secure world and a normal world. The secure world holds keys, secure boot and trusted applications, isolated from the main operating system. It exists on Cortex-A and, as Armv8-M, on Cortex-M. + +See also: [Root of Trust](r.md#root-of-trust), [Secure Element](s.md#secure-element). + +## TUP +**Telephone User Part.** The first SS7 call control protocol, for ordinary telephone calls without ISDN features. ISUP has largely replaced it. + +See also: [ISUP](i.md#isup), [SS7](s.md#ss7). + +## Twisted pair +Cable in which the two wires of each circuit are twisted together, which cuts interference and crosstalk. It is used for telephone lines and Ethernet. + +See also: [Cat 5](c.md#cat-5), [Ethernet over twisted pair](e.md#ethernet-over-twisted-pair). diff --git a/content/glossary/u.md b/content/glossary/u.md new file mode 100644 index 0000000..bc6de84 --- /dev/null +++ b/content/glossary/u.md @@ -0,0 +1,214 @@ +--- +title: U +--- + +# U + +## UA +**User Agent.** Software acting for a user. In HTTP, the browser or client, which names itself in the `User-Agent` header; in SIP, an endpoint that makes or receives calls. + +See also: [HTTP](h.md#http), [SIP](s.md#sip). + +## Ubiquitous Language +In domain-driven design, a shared vocabulary drawn from the business domain, used consistently by everyone, from conversations and requirements through to the code. + +## UDP +**User Datagram Protocol.** A simple, connectionless transport protocol (RFC 768) that sends datagrams with port numbers and a checksum, but no guarantee of delivery or order. DNS, VoIP media and streaming use it, along with protocols that add their own reliability. + +![UDP](images/15009084.png) + +[[CAPTION:Figure]] UDP. + +[[CAPTION:Table]] The UDP header: four 16-bit fields, then the data. + +| Bits 0 to 15 | Bits 16 to 31 | +| --- | --- | +| Source port | Destination port | +| Length | Checksum | +| Data | Data | + +See also: [TCP](t.md#tcp), [RTP](r.md#rtp), [DTLS](d.md#dtls). + +## UER +**Universal Edge Router.** Juniper's name for its broadband edge routers, which do the job of a BRAS. + +See also: [BRAS](b.md#bras), [BNG](b.md#bng). + +## UML + +**Unified Modeling Language.** + +The standard graphical language for modelling software systems, maintained by the Object Management Group. It was created by Grady Booch, Ivar Jacobson and James Rumbaugh in the mid-1990s. Its diagrams include class, sequence, activity, state and use case diagrams. + +![UML](images/15009092.png) + +[[CAPTION:Figure]] UML. + +![UML](images/15009098.png) + +[[CAPTION:Figure]] UML. + +![UML](images/15009104.png) + +[[CAPTION:Figure]] UML. + +See also: [Class Model](c.md#class-model), [Sequence Model](s.md#sequence-model), [Use Case](#use-case). + +## UMTS +**Universal Mobile Telecommunications System.** The 3GPP third-generation (3G) mobile system, which evolved from GSM. It uses wideband CDMA radio, and HSPA later speeded up its data. + +![UMTS](images/15008611.png) + +[[CAPTION:Figure]] UMTS. + +See also: [W-CDMA](w.md#w-cdma), [UTRAN](#utran), [GSM](g.md#gsm), [LTE (4G)](l.md#lte-4g). + +## UNI +**User-Network Interface.** The demarcation between a service provider's network and a customer's equipment, where the provider's responsibility ends. + +See also: [NNI](n.md#nni), [EVC](e.md#evc). + +## Unicast +Sending data from one sender to one receiver, each identified by its own address. Most internet traffic is unicast. + +![Unicast](images/15009110.png) + +[[CAPTION:Figure]] Unicast. + +See also: [Multicast](m.md#multicast), [Broadcast](b.md#broadcast), [Anycast](a.md#anycast). + +## Unicode +The standard that gives a unique number, or code point, to every character in the world's writing systems, plus symbols and emoji. It is kept in step with ISO/IEC 10646. UTF-8 and UTF-16 are ways of encoding it. + +See also: [UTF-8](#utf-8), [UTF-16](#utf-16), [ASCII](a.md#ascii). + +## Unified Communications +**UC.** Bringing an organisation's communication services, such as voice, video, messaging, presence, conferencing and voicemail, together behind one consistent user experience across devices. + +See also: [VoIP](v.md#voip), [SIP](s.md#sip). + +## Unit Test +The lowest level of component test: a test of one small piece of code on its own. + +See also: [Unit Testing](#unit-testing). + +## Unit Testing +Writing and running small automated tests, written by the developers, that each check a narrow piece of the code. + +See also: [Unit Test](#unit-test), [TDD](t.md#tdd), [Mock Objects](m.md#mock-objects). + +## Unix +A family of multi-user, multitasking operating systems descended from the original developed at Bell Labs by Ken Thompson, Dennis Ritchie and others from 1969. BSD, Solaris, AIX and macOS are Unix systems; Linux is Unix-like. + +![Unix](images/15009116.png) + +[[CAPTION:Figure]] Unix. + +See also: [Linux](l.md#linux), [POSIX](p.md#posix), [Sh](s.md#sh). + +## Update +Planned work to bring an artefact up to date with the project's current needs. + +See also: [Rework](r.md#rework). + +## Upstream +In a project, the early activities and artefacts, such as chartering, planning, requirements and architecture. In networking, the direction from the customer towards the network. + +See also: [Downstream](d.md#downstream). + +## URI +**Uniform Resource Identifier.** A string that identifies a resource (RFC 3986). URLs, which say where to find it, and URNs, which name it, are both kinds of URI. + +![URI](images/15009122.png) + +[[CAPTION:Figure]] URI. + +See also: [URL](#url), [URN](#urn). + +## URL +**Uniform Resource Locator.** A URI that says how and where to get a resource: a scheme, such as `https`, a host, a path and optional query and fragment, as in `https://example.com/page?id=1#top`. + +See also: [URI](#uri), [URN](#urn), [HTTP](h.md#http). + +## URN +**Uniform Resource Name.** A URI that names a resource permanently, without saying where it is, as in `urn:isbn:0451450523`. + +See also: [URI](#uri), [URL](#url). + +## Usability Testing +Watching real users try to do realistic tasks with a product, to find where it is hard to use. + +See also: [User Experience](#user-experience). + +## Use Case +A description of how a user (actor) interacts with a system to achieve a goal, including the main path and alternatives. + +See also: [Use Case Model](#use-case-model), [Use Case Specification](#use-case-specification), [Nominal Path](n.md#nominal-path). + +## Use Case Model +The set of use cases and actors that describes a system's behaviour from the outside. + +See also: [Use Case](#use-case), [UML](#uml). + +## Use Case Specification +The document that describes one use case in full. + +See also: [Use Case](#use-case). + +## User Experience +**UX.** The whole of a person's experience of using a product, including but going beyond its user interface: how useful, easy and pleasant it is. + +See also: [User Interface](#user-interface), [Usability Testing](#usability-testing), [Personas](p.md#personas). + +## User Interface +**UI.** The part of a system through which people interact with it, such as screens, controls and messages. + +See also: [User Interface Design](#user-interface-design), [User Experience](#user-experience). + +## User Interface Design +Designing how people interact with a system: layout, controls, flows and feedback. + +See also: [User Interface](#user-interface), [User Interface Prototype](#user-interface-prototype). + +## User Interface Prototype +A prototype built to explore a user interface's behaviour or look and feel. + +See also: [High Fidelity Prototype](h.md#high-fidelity-prototype), [Low Fidelity Prototype](l.md#low-fidelity-prototype). + +## User Plane +The part of a network that carries users' traffic, as opposed to the control plane, which carries signalling. In routers it is also called the data plane or forwarding plane. In mobile networks, separating the two planes lets each scale on its own. + +See also: [Data Plane](d.md#data-plane), [Control Plane](c.md#control-plane), [Management Plane](m.md#management-plane). + +## User Stories +Short descriptions of functionality from the user's point of view, agreed with the customer or product owner, that the team uses to divide up the work. + +See also: [Role-feature-reason](r.md#role-feature-reason), [INVEST](i.md#invest), [Three C's](t.md#three-cs), [Epic](e.md#epic). + +## UTC +**Coordinated Universal Time.** The world's primary time standard, kept by atomic clocks and adjusted with leap seconds to stay close to the Earth's rotation. It has no daylight saving time, and for everyday purposes it is the same as GMT. + +See also: [NTP](n.md#ntp). + +## UTF-16 +A Unicode encoding that uses one 16-bit unit for characters in the Basic Multilingual Plane and two units, a surrogate pair, for the rest. Windows, Java and JavaScript use it internally. UTF-32, by contrast, uses a fixed 32 bits for every code point. + +See also: [Unicode](#unicode), [UTF-8](#utf-8). + +## UTF-8 +The most widely used Unicode encoding, designed by Ken Thompson and Rob Pike. It uses one to four bytes per character, and the first 128 characters are the same as ASCII, so ASCII text is valid UTF-8. It is the default encoding of the web. + +See also: [Unicode](#unicode), [UTF-16](#utf-16), [ASCII](a.md#ascii). + +## UTRAN +**UMTS Terrestrial Radio Access Network.** The 3G radio access network: the Node Bs and the RNCs that control them. + +![UTRAN](images/15009128.png) + +[[CAPTION:Figure]] UTRAN. + +![UTRAN](images/15009134.png) + +[[CAPTION:Figure]] UTRAN. + +See also: [Node B](n.md#node-b), [RNC](r.md#rnc), [UMTS](#umts), [E-UTRAN](e.md#e-utran). diff --git a/content/glossary/v.md b/content/glossary/v.md new file mode 100644 index 0000000..a3f2ac0 --- /dev/null +++ b/content/glossary/v.md @@ -0,0 +1,181 @@ +--- +title: V +--- + +# V + +## V Model +A development model that extends waterfall by pairing each specification and design stage with the test stage that checks it. Requirements pair with acceptance testing, and so on down to unit tests. Drawn out, the stages form a V. + +![V Model](images/15009151.png) + +[[CAPTION:Figure]] V Model. + +```mermaid +flowchart LR + R[Requirements] --> SD[System design] --> AD[Architecture] --> MD[Module design] --> C[Coding] + C --> UT[Unit testing] --> IT[Integration testing] --> ST[System testing] --> AT[Acceptance testing] + R -. verified by .- AT + SD -. verified by .- ST + AD -. verified by .- IT + MD -. verified by .- UT +``` + +[[CAPTION:Figure]] The V model: each design stage is verified by a matching test stage. + +See also: [Waterfall](w.md#waterfall), [Testing](t.md#testing). + +## V.90 {#v90} +The ITU-T standard, from 1998, for 56 kbit/s dial-up modems. It achieves 56 kbit/s downstream and 33.6 kbit/s upstream by using the digital connection at the network end. + +See also: [V.92](#v92). + +## V.92 {#v92} +The ITU-T successor to V.90, which raised upstream speed to 48 kbit/s and added quicker connection and modem-on-hold. + +See also: [V.90](#v90). + +## VAD +**Voice Activity Detection.** Detecting whether speech is present in an audio signal. VoIP and mobile systems use it to stop sending during silence, which saves bandwidth, and generate comfort noise at the other end. + +See also: [CNG](c.md#cng), [VoIP](#voip). + +## VB +**Visual Basic.** Microsoft's event-driven language and IDE, first released in 1991, for building Windows applications quickly from forms and components. Classic VB became legacy in 2008; VB.NET is its .NET successor. + +See also: [BASIC](b.md#basic), [VBS](#vbs), [RAD](r.md#rad). + +## VBS +**VBScript.** Microsoft's scripting language based on Visual Basic, used for Windows administration and old web pages. Microsoft is removing it from Windows. + +See also: [VB](#vb). + +## VC +**Virtual Circuit.** A connection across a packet network that behaves like a dedicated circuit between two endpoints, set up before data flows. X.25, Frame Relay and ATM use virtual circuits. + +See also: [PVC](p.md#pvc), [SVC](s.md#svc), [VCI](#vci). + +## VCI +**Virtual Channel Identifier.** The 16-bit field in an ATM cell header that, with the VPI, identifies the virtual channel the cell belongs to. + +See also: [VPI](#vpi), [ATM](a.md#atm). + +## VCID +**Virtual Circuit Identifier.** The identifier that both ends of a pseudowire use to match up the same circuit, for example in an MPLS VPWS. + +![VCID](images/15009159.png) + +[[CAPTION:Figure]] VCID. + +See also: [PW](p.md#pw), [VPWS](#vpws). + +## VDSL +**Very-high-bit-rate DSL.** DSL using frequencies up to 30 MHz on short lines. VDSL2 (G.993.2) gives tens of megabits per second, and up to around 80 Mbit/s downstream from a street cabinet. It is the technology of fibre-to-the-cabinet broadband. + +See also: [ADSL](a.md#adsl), [G.Fast](g.md#gfast), [PCP](p.md#pcp). + +## Velocity +The amount of work, usually in story points, that a team completes in an iteration. Teams use it to forecast what they can take on next. + +See also: [Points](p.md#points), [Burndown Chart](b.md#burndown-chart). + +## Verifier +In an inspection, the person who checks that the rework has been done properly. + +See also: [Inspection](i.md#inspection). + +## Version +A formally released revision of an artefact, recognised by configuration management. + +See also: [Revision](r.md#revision), [Semantic Versioning](s.md#semantic-versioning). + +## Version Control +Recording every change to files, such as source code, so that any earlier version can be recovered and several people can work in parallel and merge their work. Git is the most widely used tool. + +See also: [Git](g.md#git), [Subversion](s.md#subversion), [SCM](s.md#scm). + +## VHDL +**VHSIC Hardware Description Language.** A language, standardised as IEEE 1076, for describing and simulating digital hardware, from systems down to logic gates. It came out of the US Very High-Speed Integrated Circuits programme in the 1980s. Verilog is its main alternative. + +## VISP +**Virtual ISP.** An internet service provider that sells another ISP's service under its own brand. + +See also: [ISP](i.md#isp), [MVNO](m.md#mvno). + +## VLAN +**Virtual LAN.** A separate broadcast domain created on shared switches, usually by tagging frames with an 802.1Q VLAN ID, so that one physical network carries several isolated LANs. + +![VLAN](images/15009165.png) + +[[CAPTION:Figure]] VLAN. + +See also: [Dot1Q](d.md#dot1q), [Dot1Q trunking](d.md#dot1q-trunking), [Switch](s.md#switch). + +## VLR +**Visitor Location Register.** The database, usually built into each MSC, of the subscribers currently in its area. It copies their details from their HLR. + +See also: [HLR](h.md#hlr), [MSC](m.md#msc). + +## VM +**Virtual Machine.** A software emulation of a computer, on which an operating system and applications run as if on real hardware. A hypervisor shares the physical machine between several VMs. + +See also: [Docker](d.md#docker), [IaaS](i.md#iaas). + +## VOD +**Video on Demand.** Letting viewers choose and watch video when they want, rather than at a broadcast time, by streaming or downloading. + +See also: [IPTV](i.md#iptv), [STB](s.md#stb). + +## VoIP +**Voice over IP.** Carrying voice calls over IP networks, with signalling protocols such as SIP to set up calls and RTP to carry the media. + +![VoIP](images/15009171.png) + +[[CAPTION:Figure]] VoIP. + +See also: [SIP](s.md#sip), [RTP](r.md#rtp), [SBC](s.md#sbc), [VoLTE](#volte). + +## VoLTE +**Voice over LTE.** Voice calls over an LTE network as IP packets, using IMS, following the GSMA IR.92 profile. It needs no circuit-switched network, connects calls faster and supports HD voice. + +See also: [IMS](i.md#ims), [LTE (4G)](l.md#lte-4g), [AMR-WB](a.md#amr-wb). + +## Vorbis +An open, royalty-free lossy audio codec from the Xiph.Org Foundation, usually used in the Ogg container (Ogg Vorbis). Opus has largely replaced it. + +See also: [OPUS](o.md#opus), [MP3](m.md#mp3). + +## VPI +**Virtual Path Identifier.** The field in an ATM cell header that identifies a virtual path, a bundle of virtual channels. See [VCI](#vci). + +## VPLS +**Virtual Private LAN Service.** A provider service that joins several customer sites into one Ethernet broadcast domain over an MPLS network, using a mesh of pseudowires. It is defined in RFCs 4761 and 4762. + +![VPLS](images/15009177.png) + +[[CAPTION:Figure]] VPLS. + +See also: [L2VPN](l.md#l2vpn), [PW](p.md#pw), [VPWS](#vpws). + +## VPN +**Virtual Private Network.** A private network built over a shared or public one, such as the internet or a provider's MPLS core. It uses tunnelling and often encryption, so that remote sites or users act as if directly connected. + +![VPN](images/15009183.png) + +[[CAPTION:Figure]] VPN. + +See also: [IPSec](i.md#ipsec), [L3VPN](l.md#l3vpn), [L2VPN](l.md#l2vpn). + +## VPWS +**Virtual Private Wire Service.** A point-to-point layer-2 service, such as Ethernet, carried over an MPLS network by a pseudowire. It is also called a virtual leased line or Ethernet over MPLS (EoMPLS). + +See also: [PW](p.md#pw), [VPLS](#vpls), [EPWS](e.md#epws). + +## VRF +**Virtual Routing and Forwarding.** Several separate routing tables in one router, so that different customers or networks, even with overlapping addresses, stay apart. It is the building block of MPLS layer-3 VPNs. + +![VRF](images/15009189.png) + +[[CAPTION:Figure]] VRF. + +See also: [L3VPN](l.md#l3vpn), [CE](c.md#ce). diff --git a/content/glossary/w.md b/content/glossary/w.md new file mode 100644 index 0000000..f857d28 --- /dev/null +++ b/content/glossary/w.md @@ -0,0 +1,204 @@ +--- +title: W +--- + +# W + +## W-CDMA +**Wideband CDMA.** The UMTS air interface, which uses direct-sequence CDMA over pairs of 5 MHz channels. + +See also: [UMTS](u.md#umts), [CDMA](c.md#cdma). + +## W3C +**World Wide Web Consortium.** The international body, founded by Tim Berners-Lee, that develops standards for the web, such as HTML, CSS and XML. + +See also: [XML](x.md#xml), [CSS](c.md#css). + +## Walkthrough +An informal review in which the author takes one or more people through an artefact to find defects. + +See also: [Review](r.md#review), [Desk Check](d.md#desk-check). + +## WAN +**Wide Area Network.** A network that spans a large area, linking sites in different cities or countries, usually over carriers' circuits. The internet is the largest WAN. + +![WAN](images/15009199.png) + +[[CAPTION:Figure]] WAN. + +See also: [LAN](l.md#lan), [MAN](m.md#man). + +## WANET +**Wireless Ad hoc Network.** A decentralised wireless network with no routers or access points, in which every node forwards traffic for the others. A MANET is the mobile kind. + +See also: [MANET](m.md#manet), [Wireless Mesh Network](#wireless-mesh-network). + +## WAP +**Wireless Access Point.** A device that lets Wi-Fi devices connect to a wired network. WAP also stood for Wireless Application Protocol, an early way of browsing on mobile phones. + +See also: [Wi-Fi](#wi-fi), [Wireless Repeater](#wireless-repeater). + +## Waterfall +A sequential development model in which work flows through requirements, design, construction, testing and deployment, each finished before the next starts. Winston Royce's 1970 paper, often cited as its origin, described it as risky. + +![Waterfall](images/15009207.png) + +[[CAPTION:Figure]] Waterfall. + +See also: [Waterfall Lifecycle](#waterfall-lifecycle), [V Model](v.md#v-model), [Spiral Model](s.md#spiral-model). + +## Waterfall Lifecycle +A lifecycle that moves through ordered phases, from concept to delivery, with a review at the end of each. + +See also: [Waterfall](#waterfall). + +## WBC +**Wholesale Broadband Connect.** BT Wholesale's broadband access product on the 21CN network. ISPs connect at regional aggregation points to collect their customers' traffic. + +![WBC](images/15009213.png) + +[[CAPTION:Figure]] WBC. + +See also: [WBMC](#wbmc), [BT 21CN](b.md#bt-21cn). + +## WBCC +**Wholesale Broadband Connect Converge.** A BT Wholesale package, based on WBC, that combined voice and broadband on one bill for the ISP. + +See also: [WBC](#wbc). + +## WBMC +**Wholesale Broadband Managed Connect.** A BT Wholesale product, built on WBC, in which BT carries the traffic end to end to a single handover point. It replaced IPStream. + +See also: [WBC](#wbc), [IPSC](i.md#ipsc). + +## Weekly Individual Plan +A plan in which each person sets their miniature milestones for the week. + +See also: [Miniature Milestone](m.md#miniature-milestone). + +## WEP +**Wired Equivalent Privacy.** The original Wi-Fi security from 1997, which is easily broken. WPA and WPA2 replaced it, and it was deprecated in 2004. + +See also: [WPA2](#wpa2), [TKIP](t.md#tkip). + +## What Requirement +See [Functional Requirement](f.md#functional-requirement). + +## Why Requirement +See [Business Requirement](b.md#business-requirement). + +## Wi-Fi +Wireless LAN technology based on the IEEE 802.11 standards. Wi-Fi is the Wi-Fi Alliance's trademark for certified products. Recent generations are numbered, such as Wi-Fi 6 (802.11ax) and Wi-Fi 7 (802.11be). + +See also: [802.11](numbers.md#80211), [WAP](#wap), [WPA2](#wpa2), [SSID](s.md#ssid). + +## Wiki +A website that its users edit together in the browser, usually with simple markup. + +See also: [Document Collaboration](d.md#document-collaboration), [Knowledge Management](k.md#knowledge-management). + +## WiMAX +**Worldwide Interoperability for Microwave Access.** Wireless broadband based on the IEEE 802.16 standards, promoted as an alternative to DSL and as a 4G candidate. LTE won out. + +See also: [LTE (4G)](l.md#lte-4g). + +## WIN-T +**Warfighter Information Network-Tactical.** The US Army's mobile tactical communications network, giving units secure voice, data and video on the move without fixed infrastructure. + +![WIN-T](images/15009219.png) + +[[CAPTION:Figure]] WIN-T. + +See also: [MANET](m.md#manet). + +## Wireless Mesh Network +A wireless network in which nodes connect in a mesh and relay traffic for each other, with some nodes acting as gateways to a wired network. The nodes rarely move, so routes stay stable. + +![Wireless Mesh Network](images/15009225.png) + +[[CAPTION:Figure]] Wireless Mesh Network. + +See also: [WANET](#wanet), [MANET](m.md#manet), [Network Topologies](n.md#network-topologies). + +## Wireless Repeater +A device that receives a Wi-Fi signal and rebroadcasts it to extend coverage. Also called a range extender. + +See also: [WAP](#wap). + +## Wireshark +A free, open-source packet analyser, formerly called Ethereal, used for troubleshooting networks and developing protocols. + +![Wireshark](images/15009231.png) + +[[CAPTION:Figure]] Wireshark. + +See also: [Packet sniffing](p.md#packet-sniffing). + +## WISP +**Wireless ISP.** An internet service provider whose access network is wireless, such as fixed wireless or Wi-Fi. + +![WISP](images/15009237.png) + +[[CAPTION:Figure]] WISP. + +See also: [ISP](i.md#isp). + +## WLR +**Wholesale Line Rental.** A UK product that let a provider rent a phone line from Openreach and bill the customer for line and calls. It is being withdrawn as the copper phone network closes. + +See also: [CPS](c.md#cps), [SMPF](s.md#smpf), [SoGEA](s.md#sogea). + +## WMA +**Windows Media Audio.** Microsoft's family of audio codecs, including lossy, professional, lossless and voice versions. + +See also: [MP3](m.md#mp3), [AAC](a.md#aac). + +## Work Breakdown Structure +**WBS.** A hierarchical breakdown of all a project's work into work packages. Larger projects keep it as a separate document; smaller ones hold it in the work plan or schedule. + +See also: [Work Package](#work-package), [Work Plan](#work-plan). + +## Work Item +A unit of tracked work, such as a task, story or defect. Also used for the artefact the work produces. + +See also: [Artefact](a.md#artefact), [Item](i.md#item). + +## Work Package +The lowest level of a work breakdown structure, often a tangible deliverable, which is broken down further into activities and tasks for estimating and scheduling. + +See also: [Work Breakdown Structure](#work-breakdown-structure). + +## Work Plan +A plan of the work packages and their task estimates, used to plan, track and control detailed work. + +See also: [Work Package](#work-package), [Detailed Schedule](d.md#detailed-schedule), [Earned Value](e.md#earned-value). + +## Work Product +See [Artefact](a.md#artefact). + +## WPA2 +**Wi-Fi Protected Access 2.** The Wi-Fi security certification, from 2004, based on the full IEEE 802.11i standard with AES encryption. It replaced WEP and WPA, and WPA3 now succeeds it. + +See also: [WEP](#wep), [TKIP](t.md#tkip), [EAP](e.md#eap). + +## WPS +**Wi-Fi Protected Setup.** A Wi-Fi Alliance scheme, from 2006, for adding devices to a network with a button or PIN rather than a passphrase. The PIN method is insecure and should be disabled. + +See also: [WPA2](#wpa2). + +## Write-only code +Code so hard to read that nobody, often not even its author, can understand it later. + +See also: [Construction Standard](c.md#construction-standard). + +## WS-BPEL +**Web Services Business Process Execution Language.** The full name of [BPEL](b.md#bpel). + +## WSDL +**Web Services Description Language.** An XML language for describing a web service's interface: its operations, messages, data types and endpoints. It is mostly used with SOAP. + +![WSDL](images/15009243.png) + +[[CAPTION:Figure]] WSDL. + +See also: [SOAP](s.md#soap), [XML Schema](x.md#xml-schema). diff --git a/content/glossary/x.md b/content/glossary/x.md new file mode 100644 index 0000000..72a3b73 --- /dev/null +++ b/content/glossary/x.md @@ -0,0 +1,54 @@ +--- +title: X +--- + +# X + +## X.25 {#x25} +An ITU-T packet-switching protocol suite for WANs, from 1976, using virtual circuits with error correction at every hop. It was widely used in the 1980s, for example for cash machines, and IP has replaced it. + +![X.25](images/15009253.png) + +[[CAPTION:Figure]] X.25. + +See also: [Frame Relay](f.md#frame-relay), [HDLC](h.md#hdlc). + +## X.509 {#x509} +The ITU-T standard format for public key certificates, which bind an identity, such as a host name, to a public key under a CA's signature. TLS and many other protocols use them. + +See also: [CA](c.md#ca), [TLS](t.md#tls), [DER](d.md#der). + +## XML +**Extensible Markup Language.** A W3C text format for structured documents and data, with user-defined, nested elements. It is the basis of many formats and protocols, such as SOAP and SVG. + +See also: [XML Schema](#xml-schema), [JSON](j.md#json), [SGML](s.md#sgml). + +## XML Schema +A description of the allowed structure and content of a type of XML document, used to validate documents. DTD, XSD and RELAX NG are schema languages. + +See also: [Xsd](#xsd), [XML](#xml). + +## XML-RPC +A simple remote procedure call protocol that encodes calls in XML and sends them over HTTP. SOAP grew out of it. + +See also: [JSON-RPC](j.md#json-rpc), [SOAP](s.md#soap), [RPC](r.md#rpc). + +## XMPP +**Extensible Messaging and Presence Protocol.** An open, XML-based protocol for instant messaging and presence, originally called Jabber. It has been extended for many other real-time uses. + +See also: [SIMPLE](s.md#simple). + +## XP +**Extreme Programming.** An agile method that focuses on engineering practices, such as pair programming, test-driven development, continuous integration, refactoring and small releases. + +See also: [Pair Programming](p.md#pair-programming), [TDD](t.md#tdd), [Agile development](a.md#agile-development). + +## Xsd +**XML Schema Definition.** The W3C's XML schema language. It describes the elements and attributes an XML document may contain, with data types, so documents can be validated. + +See also: [XML Schema](#xml-schema). + +## XSS +**Cross-Site Scripting.** A web vulnerability in which an attacker gets a site to include their script in pages other users view. The script then runs with the site's privileges in the victims' browsers. Encoding output and a content security policy prevent it. + +See also: [Pen testing](p.md#pen-testing). diff --git a/content/glossary/y.md b/content/glossary/y.md new file mode 100644 index 0000000..ecf7276 --- /dev/null +++ b/content/glossary/y.md @@ -0,0 +1,10 @@ +--- +title: Y +--- + +# Y + +## YAML +**YAML Ain't Markup Language.** A human-readable data format that uses indentation for nesting, widely used for configuration files. YAML 1.2 is a superset of JSON. + +See also: [JSON](j.md#json), [TOML](t.md#toml). diff --git a/content/glossary/z.md b/content/glossary/z.md new file mode 100644 index 0000000..66a6d38 --- /dev/null +++ b/content/glossary/z.md @@ -0,0 +1,10 @@ +--- +title: Z +--- + +# Z + +## ZigBee +A low-power, low-rate wireless mesh protocol based on IEEE 802.15.4, used in home automation and sensors. It is simpler and uses less power than Bluetooth or Wi-Fi. + +See also: [Bluetooth](b.md#bluetooth), [Wireless Mesh Network](w.md#wireless-mesh-network). diff --git a/content/guidelines/agent-rules.md b/content/guidelines/agent-rules.md new file mode 100644 index 0000000..6dd8757 --- /dev/null +++ b/content/guidelines/agent-rules.md @@ -0,0 +1,216 @@ +--- +title: "Agent rules (condensed)" +status: draft +owner: "Project owner" +last-reviewed: 2026-09-29 +--- + +# Agent rules (condensed) + +## How to use this file + +Read this file at the start of every task, together with `AGENTS.md`. Rules cite IDs. Each section links to the full guideline, which is authoritative. If a rule conflicts with the task, stop and ask (GOV-09). + +## Always + +1. Follow `AGENTS.md` and the commands in it. +1. Make the smallest change that meets the requirement (CP-02, CP-16), and state assumptions and questions first (CP-20). +1. Write Conventional Commits with `Refs:` and `Assisted-by:` footers (VCS-03, VCS-05, AGT-06). +1. Work on a short-lived branch and open a PR. Never push to `main`, and never approve or merge your own work (AGT-04, PRR-06). +1. Add or update tests with the change. Never skip, delete or weaken a test or check to pass a gate (TST-15, PIP-08). +1. Run the full local check command before proposing a change and include the results (AGT-09). +1. Update documentation in the same change (DOC-04). +1. Propose significant or hard-to-reverse decisions as an ADR with status `proposed` (ADR-03). +1. Never put secrets or personal data in code, logs, commits or prompts (SEC-05, AGT-08). +1. Stay out of the agent boundaries unless the task says so: guidelines, `AGENTS.md`, pipeline definitions, CODEOWNERS, secrets and signing material, generated and vendored code, security-relevant code, shared history and certified items (AGT-05). +1. When a rule is silent or unclear, apply the principles in [principles.md](principles-catalogue.md), starting with its start-here list. If principles conflict, or the answer is not clear, stop and ask. +1. Rules marked *(core)* cannot be tailored. Follow the product's conformance profile for the rest. + +## [EG-001 Guidelines governance](guidelines-governance.md) + +- Do not edit anything in `docs/guidelines/` unless the task explicitly says so (AGT-05). +- Cite rules by ID. +- If a rule blocks the task, stop and ask rather than deviating (GOV-09). + +## [EG-002 Engineering values and ways of working](values-and-ways-of-working.md) + +- Prefer small, reviewable changes (VAL-11). +- Record decisions and assumptions in the PR or an ADR, not only in chat (VAL-06). +- Raise risks and doubts early and do not hide uncertainty (VAL-12). + +## [EG-003 Quality policy and quality attributes](quality-policy-and-attributes.md) + +- When a task involves a trade-off, use the project's ranked quality attributes and say which one you optimised (QUA-03). +- Write measurable acceptance criteria (environment, stimulus, response, measure) and not adjectives (QUA-04). + +## [EG-004 Documentation as code and terminology](documentation-as-code.md) + +- Update docs in the same change as the behaviour they describe (DOC-04). +- Never hand-edit generated docs (DOC-09). +- Draw diagrams as Mermaid text and avoid semicolons in labels because they break rendering. +- Write plain British English with the conclusion first (DOC-11). +- Check every statement in a doc against the code before committing (AGT-15). + +## [EG-005 AI agent usage](ai-agent-usage.md) + +- Read `AGENTS.md` first and follow it (AGT-01). +- You may not approve or merge your own work (AGT-04). +- Do not touch governance files, secrets or signing material, generated or vendored code, security-relevant code, quality controls, shared history or certified items unless the task says so (AGT-05). +- Add an `Assisted-by:` trailer to your commits (AGT-06). +- Run the build, tests and lint before proposing a change and include the results (AGT-09). +- Keep changes small and single-purpose, and summarise intent and risk for larger ones (AGT-10). +- Do not send data to a tool that is not approved for its classification (AGT-08). + +## [EG-006 Requirements and traceability](requirements-and-traceability.md) + +- Before implementing, find the requirement ID or IDs the task serves and cite them in commits (`Refs: REQ-...`) and test names (REQ-06). +- If there is no requirement, say so and ask, or propose one for review (REQ-13). +- Never edit approved requirements without instruction, and flag the impact on tests and docs (REQ-09). + +## [EG-007 Architecture principles](architecture-principles.md) + +- Respect layering and dependency direction (AP-02) and keep third-party and hardware code behind our interfaces (AP-03). +- Choose the simplest design that meets the requirement (AP-05). +- When a design choice is significant or hard to reverse, propose an ADR instead of just implementing it (AP-13). +- Cite principle IDs in design notes (PRN-01). + +## [EG-008 Decision records: RFCs and ADRs](decision-records.md) + +- When a decision is significant, hard to reverse or crosses component boundaries, draft an ADR with status `proposed` from `docs/templates-reference/adr.md` instead of implementing silently (ADR-03). +- Do not edit accepted ADRs, supersede them (ADR-04). +- Read `docs/adr/` before changing behaviour it governs. +- Record at least two options (ADR-05). + +## [EG-009 Interfaces and hardware and software co-design](interfaces-and-co-design.md) + +- Do not change a documented interface without an ADR or RFC and a version bump (INT-04). +- Regenerate generated interface files instead of editing them (INT-03). +- Add or update contract tests on both sides when touching an interface (INT-05). +- Cite datasheets and errata by ID and version (INT-11). + +## [EG-010 Components, reuse and variants](components-reuse-and-variants.md) + +- Do not copy code from another component or repository into this one, depend on a pinned released version instead (COM-05, COM-03). +- Do not patch a component you consume, propose the change to its owner (COM-06). +- Do not depend on a branch or an unreleased commit (COM-03). +- When you change a component that has other consumers, state the consumer impact and compatibility (COM-07). + +## [EG-011 Technology selection and third-party components](technology-and-third-party.md) + +- Do not add a dependency or tool without asking or drafting an ADR (TEC-02). +- Pin exact versions and update the manifest (TEC-04, TEC-05). +- Check licences against the approved list (TEC-03). +- Do not paste code of unknown provenance (TEC-09). +- Wrap vendor code behind an interface (TEC-06) and never edit vendored code directly. + +## [EG-012 Coding principles](coding-principles.md) + +- Apply CP-02: write the minimum code, with no speculative features. +- Apply CP-16: change only what is needed and report unrelated problems instead of fixing them. +- Apply CP-20: state assumptions, ask when unclear and present alternatives. +- Handle every error path (CP-06), validate inputs at boundaries (CP-05) and keep warnings at zero (CP-15). +- Mark any deviation with the rule ID and a reason (CP-21). + +## [EG-013 Coding standard](coding-standard.md) + +- Run the formatter and linter before every commit and do not hand-format (COD-04). +- Do not add suppressions without a rule ID and justification (COD-05). +- Use the flags in the build configuration unchanged (COD-08). +- Do not reformat unrelated code (COD-09). +- Follow the condensed rule list in the repository (COD-11). + +## [EG-014 Version control, commits and versioning](version-control-and-versioning.md) + +- Write Conventional Commits with type, scope and the footers `Refs:` and `Assisted-by:` (VCS-03, VCS-05). +- Mark breaking changes with `!` and `BREAKING CHANGE:` (VCS-04). +- Keep commits atomic and buildable (VCS-06). +- Work on a short-lived branch and never push to `main` directly (VCS-02). +- Never force-push shared branches or rewrite history you did not create (VCS-08). +- Never edit version strings by hand (VCS-10). +- Never commit secrets, build outputs or large binaries (VCS-01). + +## [EG-015 Pull requests and code review](pull-requests-and-review.md) + +- Open a PR with a Conventional Commit title and the filled template: what and why, links, evidence, risk and checklist (PRR-02, PRR-03). +- Keep to one purpose per PR and roughly under 400 lines excluding generated files (PRR-04). +- Label your review comments blocking, suggestion, question or nit (PRR-10). +- Never approve or merge your own PR (PRR-06). +- Wait for required checks and fix failures instead of bypassing them (PRR-07). + +## [EG-016 Pipelines and quality gates](pipelines-and-quality-gates.md) + +- Do not modify pipeline definitions, required checks or gates unless the task says so (AGT-05). +- Never skip, disable or loosen a check or test to make a pipeline pass (PIP-08, TST-15). +- Run the same commands CI runs before proposing a change (PIP-01). +- If a test is flaky, report it and do not delete it (PIP-09). +- If your change breaks `main`, revert first (PIP-10). + +## [EG-017 Testing principles and strategy](testing-principles-and-strategy.md) + +- Write a failing test that reproduces a bug before fixing it (TP-08). +- Add tests in the same change as the code, named with requirement IDs (TST-03). +- Test behaviour and not implementation, and include negative and boundary cases (TP-04, TP-09). +- Never skip, delete or weaken a test to make a gate pass (TST-15). +- Check that generated tests verify the requirement and would fail if the code were wrong (AGT-11). + +## [EG-018 Defect and technical debt management](defects-and-technical-debt.md) + +- When fixing a defect, reproduce it with a test first, fix it, reference it in the commit (`Refs:`) and record the root cause category in the PR (DEF-05). +- Log unrelated problems you notice as tech-debt items instead of fixing them in passing (CP-16, DEF-09). +- Never close a defect without a regression test (DEF-04). + +## [EG-019 Logging, debug and fault diagnostics](logging-debug-and-diagnostics.md) + +- Use the project's logging facility with the correct level and module, and never `printf` or ad-hoc prints (LOG-03). +- Never log secrets, keys or personal data (LOG-08). +- Handle every fault path explicitly (DBG-03). +- Do not put side effects in assertions (DBG-04). +- Add event and fault codes to the registry and never invent them inline (LOG-05). + +## [EG-020 Observability and telemetry](observability-and-telemetry.md) + +- Do not add telemetry fields without updating the versioned schema and classification (OBS-02, OBS-05). +- Give every metric a name, unit and type per the convention (OBS-02). +- Never add personal data to telemetry (OBS-05). +- Do not add alerts without an owner, runbook and action (OBS-08). + +## [EG-021 Release, update and production](release-update-and-production.md) + +- Never create or move release tags, edit changelogs by hand or change version strings unless the task says so (VCS-10, VCS-11). +- Do not rebuild artefacts for release (REL-02). +- Put upgrade notes and breaking changes in commit footers so that release notes generate correctly (REL-05). + +## [EG-022 Security and data protection](security-and-data-protection.md) + +- Never write secrets, keys or tokens into code, config, logs, commits or prompts (SEC-05). +- If you find one, stop and report it. +- Never send Confidential or Restricted data to an unapproved tool (AGT-08, SEC-10). +- Do not use home-grown cryptography (SEC-03). +- Do not touch security-relevant code, authentication, key handling or debug protection without explicit instruction (AGT-05). +- Validate all input (CP-17). + +## [EG-023 Safety, regulatory and certification](safety-regulatory-and-certification.md) + +- Do not modify code, data or configuration in certified or regulated areas unless the task says so, and flag the change impact class in the PR (REG-05). +- Do not change tools or toolchain versions for certified builds (REG-08). + +## [EG-024 Planning, delivery and improvement](planning-delivery-and-improvement.md) + +- Treat the Definition of Done for the work item type as the finish line: reviewed, tested, documented, traced, with diagnostics and budgets checked (PLN-03). +- State uncertainty as a range and not a single date (PLN-04). +- Report blockers and risks in the PR or task summary and do not work around them silently (PLN-05, PLN-06). +- Draft postmortems from the template when asked (PLN-11). + +## [EG-025 Research, prototyping and experiments](research-and-prototyping.md) + +- State the question and time-box before starting a spike (RSH-01). +- Put spike code only under `prototypes/` or a `proto/` branch (RSH-02). +- Record results, including negative ones, in `docs/research/` using the experiment record template (RSH-05). +- Do not promote prototype code into the product by copying it, propose a graduation task instead (RSH-04). +- Prototype status does not relax agent boundaries (RSH-12). + +## [EG-026 Competence, tooling and onboarding](competence-tooling-and-onboarding.md) + +- Write or update a `docs/how-to/` page when you discover a non-obvious procedure (CMP-04, CMP-10). +- Keep the environment setup script working and tested (CMP-07). +- Point to ADRs and docs for rationale and do not rely on chat history (CMP-10). diff --git a/content/guidelines/ai-agent-usage.md b/content/guidelines/ai-agent-usage.md new file mode 100644 index 0000000..afe9ed3 --- /dev/null +++ b/content/guidelines/ai-agent-usage.md @@ -0,0 +1,130 @@ +--- +id: EG-005 +title: "AI agent usage" +status: draft +owner: "Build and tooling lead and Security lead" +version: 0.1.0 +part: "Foundations" +related: [EG-004, EG-011, EG-012, EG-015, EG-022] +review-by: 2027-09-30 +supersedes: +--- + +# EG-005: AI agent usage + +- **Owner:** Build and tooling lead and Security lead +- **Applies to:** All use of AI coding agents and assistants on team repositories +- **Enforcement:** CI, branch protection, review +- **Related:** [EG-004](documentation-as-code.md), [EG-011](technology-and-third-party.md), [EG-012](coding-principles.md), [EG-015](pull-requests-and-review.md), [EG-022](security-and-data-protection.md) +- **Principles:** AIP-01 to AIP-11 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Let agents raise productivity without lowering quality, security or traceability, by making standards machine-usable and enforcing them independently of the agent. + +## Rules + +1. **AGT-01** *(core)* Every repository MUST have an `AGENTS.md` at its root, reviewed and owned like code. +1. **AGT-02** `AGENTS.md` MUST state how to build, test, lint and format with exact commands, where the guidelines and ADRs are, the branching and commit conventions, and the agent boundaries (AGT-05). +1. **AGT-03** *(core)* Agent-authored changes MUST follow the same commit, PR, review and CI rules as human changes. There is no fast lane. +1. **AGT-04** *(core)* An agent MUST NOT approve or merge its own changes. At least one human reviewer who understands the change MUST approve. +1. **AGT-05** *(core)* Agents MUST NOT act in the areas listed in the boundaries table (below) without explicit human instruction in the task. +1. **AGT-06** *(core)* AI-assisted commits MUST carry a trailer, for example `Assisted-by: `, so that the extent of assistance is auditable. +1. **AGT-07** *(core)* Humans are accountable for all code they submit, whoever or whatever wrote it. The submitter MUST understand and be able to explain the change. +1. **AGT-08** *(core)* Data MUST NOT be sent to an agent service that is not approved for its classification ([EG-022](security-and-data-protection.md)). The approved tools and their permitted data classes are held in one place, owned by the security lead. +1. **AGT-09** Agents SHOULD run the local checks and tests before proposing a change, and the PR SHOULD include the evidence. +1. **AGT-10** Agents SHOULD work on small, single-purpose changes. A large agent-generated diff MUST be split or accompanied by a written summary of intent and risk. +1. **AGT-11** Agent-generated tests SHOULD be reviewed for whether they verify requirements, not just whether they pass. +1. **AGT-12** Shared prompts, agent configuration and skills SHOULD be stored in the repository, versioned and reviewed. +1. **AGT-13** Licence and provenance concerns for generated code MUST be handled under [EG-011](technology-and-third-party.md). +1. **AGT-14** `AGENTS.md` SHOULD link to the coding principles and name CP-02 (simplicity), CP-16 (surgical changes) and CP-20 (surface assumptions) explicitly. +1. **AGT-15** Agent-authored documentation MUST meet [EG-004](documentation-as-code.md), and MUST be checked for accuracy by a human against the code or design it describes. + +## Agent boundaries + +| Boundary | Examples | +|----------|----------| +| Governance files | Guidelines, `AGENTS.md`, CODEOWNERS, pipeline definitions, branch protection | +| Secrets and signing | Keys, certificates, tokens, provisioning data | +| Generated and vendored code | Generated files, third-party SDKs, vendored sources | +| Security-relevant code | Bootloader, update, cryptography, authentication, debug-port control | +| Quality controls | Disabling or skipping tests, analysis or gates, or lowering thresholds | +| History | Rewriting shared history, force-pushing | +| Certified or regulated items | Code and data in certified baselines ([EG-023](safety-regulatory-and-certification.md)) | + +## Minimum `AGENTS.md` contents + +```markdown +# AGENTS.md + +## Project +One paragraph: what this repository is and its target hardware. + +## Commands +- Build: `make build` +- Unit tests (host): `make test` +- Static analysis: `make analyse` +- Format and lint: `make lint` +- Docs: `make docs` + +## Conventions +- Commits: Conventional Commits, scopes listed in docs/guidelines +- Branches: trunk-based, short-lived `feat/` and `fix/` branches +- Requirements: reference IDs in commit footers as `Refs: REQ-...` +- Decisions: see docs/adr/, write a new ADR for significant decisions +- Principles: simplicity (CP-02), surgical changes (CP-16), surface assumptions (CP-20) + +## Boundaries +Do not modify: guidelines, pipeline definitions, CODEOWNERS, signing material, +vendor code, generated files. Ask a human first for anything security-relevant. + +## Definition of done +Builds cleanly, tests pass, analysis clean, docs updated, PR template completed. +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| AGT-01, AGT-02 | CI: `AGENTS.md` presence and required headings | +| AGT-03, AGT-04 | Branch protection: required human approvals for all authors | +| AGT-05 | CODEOWNERS on protected paths, review | +| AGT-06 | CI: commit trailer format check | +| AGT-08 | Approved tools register, security lead audit | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] `AGENTS.md` exists at the repository root with commands, conventions, boundaries and definition of done (AGT-01, AGT-02) +- [ ] Agent changes follow the same commit, PR, review and CI rules as human changes (AGT-03) +- [ ] A human approves every agent-authored change, and agents never approve or merge their own work (AGT-04) +- [ ] Agent boundaries are protected by CODEOWNERS or branch protection (AGT-05) +- [ ] AI-assisted commits carry an `Assisted-by:` trailer (AGT-06) +- [ ] The submitter understands and can explain every line submitted (AGT-07) +- [ ] An approved-tools register lists each tool and the data classes it may receive (AGT-08) +- [ ] Agents run local checks first and the PR includes the evidence (AGT-09) +- [ ] Large agent diffs are split or accompanied by a summary of intent and risk (AGT-10) +- [ ] Agent-generated tests are reviewed for whether they verify requirements (AGT-11) +- [ ] Shared prompts, agent configuration and skills are versioned and reviewed (AGT-12) +- [ ] Licence and provenance of generated code is handled under the technology guideline (AGT-13) +- [ ] `AGENTS.md` names CP-02, CP-16 and CP-20 (AGT-14) +- [ ] Agent-written documentation is checked by a human against the code or design (AGT-15) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Starter** profile (see [personal-projects.md](personal-projects.md)). This is the guideline that matters most for solo work with agents. You are the human approver. Keep every rule. CODEOWNERS is just you. Use PRs (or at least branches with CI) so that you review diffs before they reach `main`. + +### AI agents + +- Read `AGENTS.md` first and follow it (AGT-01). +- You may not approve or merge your own work (AGT-04). +- Do not touch governance files, secrets or signing material, generated or vendored code, security-relevant code, quality controls, shared history or certified items unless the task says so (AGT-05). +- Add an `Assisted-by:` trailer to your commits (AGT-06). +- Run the build, tests and lint before proposing a change and include the results (AGT-09). +- Keep changes small and single-purpose, and summarise intent and risk for larger ones (AGT-10). +- Do not send data to a tool that is not approved for its classification (AGT-08). diff --git a/content/guidelines/architecture-principles.md b/content/guidelines/architecture-principles.md new file mode 100644 index 0000000..98fa243 --- /dev/null +++ b/content/guidelines/architecture-principles.md @@ -0,0 +1,72 @@ +--- +id: EG-007 +title: "Architecture principles governance" +status: draft +owner: "Architecture group" +version: 0.1.0 +part: "Requirements, architecture and design" +related: [EG-003, EG-008, EG-012, EG-010] +review-by: 2027-09-30 +supersedes: +--- + +# EG-007: Architecture principles governance + +- **Owner:** Architecture group +- **Applies to:** Architecture governance and decision-making across products and teams +- **Enforcement:** Architecture review, ADRs, CI (dependency direction and resource budget checks) +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-008](decision-records.md), [EG-012](coding-principles.md), [EG-010](components-reuse-and-variants.md) + +> **Architectural Note:** Guidelines are generic process standards. Actual architectural principles (e.g. AP-01 to AP-17) are defined per business, product line, or vertical market in [Business Principles](business-principles.md). This guideline governs how business principles are declared, cited, and enforced. + +--- + +## Purpose + +Principles are the durable reasons behind architecture decisions. Guidelines define the generic governance framework; business verticals supply the domain-specific principles (e.g. automotive safety vs. cloud microservices). + +This guideline establishes the mandatory rules for citing, reviewing, and enforcing business principles across engineering teams and AI agents. + +--- + +## Governance Rules + +1. **PRN-01** ADRs and RFCs MUST cite the active business/vertical principle IDs (from the [business principles](business-principles.md) the product adopts) that drove or constrained the decision. +2. **PRN-02** A decision that deviates from an established business principle MUST explicitly record the deviation in its ADR, including rationale and risk mitigation. +3. **PRN-03** Where principles conflict, the business unit's precedence order applies unless an approved ADR defines a product-specific exception. +4. **PRN-04** Where a business principle can be checked automatically (dependency direction, resource budgets, schema compatibility), enforcement MUST be integrated into CI. +5. **PRN-05** Business principles MUST be reviewed annually by the business unit lead and modified only via an RFC. +6. **PRN-06** Products MAY tailor principle precedence using their quality attribute profile ([EG-003](quality-policy-and-attributes.md)). + +--- + +## Principle Enforcement Mechanics + +```text +Business Vertical Principles (e.g. AP-02 Layered Dependencies) + │ + ▼ +Generic Guideline Governance (EG-007 PRN-01 to PRN-06) + │ + ▼ +Automated Pipeline Enforcement (CI Check / Architecture Review / ADR) +``` + +--- + +## Compliance + +| Rule | Checked By | +|---|---| +| **PRN-01, PRN-02** | Architecture review & principles field in [ADR template](../templates/adr.md) | +| **PRN-04** | Pipeline checks (dependency direction, memory/CPU budgets, contract compatibility) | +| **PRN-05** | Annual governance review date | + +--- + +## Checklist + +- [ ] Active business principles are selected from [Business Principles](business-principles.md) for the product (PRN-05) +- [ ] ADRs and RFCs cite business principle IDs (PRN-01) +- [ ] Deviations from business principles are recorded in ADRs with trade-off rationale (PRN-02) +- [ ] Automated architecture rules (budgets, dependency direction) run in CI pipeline (PRN-04) diff --git a/content/guidelines/business-principles.md b/content/guidelines/business-principles.md new file mode 100644 index 0000000..8c9c09a --- /dev/null +++ b/content/guidelines/business-principles.md @@ -0,0 +1,31 @@ +# Business & Vertical Principles Overview + +## Generic Guidelines vs. Business & Vertical Principles + +In this engineering framework: + +- **Engineering Guidelines (`EG-001` – `EG-026`)** are **generic processes and mechanisms**. They define standard governance, version control rules, pull request workflows, CI/CD pipeline structures, test strategies, and compliance tracking that apply universally regardless of domain. +- **Principles (Architecture, Coding, Development)** are **specific to a business, product line, or industry vertical** (e.g., Automotive Embedded, Medical Devices, Cloud SaaS, High-Frequency Trading, Aerospace). + +> **Key Rule:** Principles MUST NOT be hardcoded into generic process guidelines. Instead, each business unit or vertical selects, tailors, or plugs in its domain principles into the generic guideline governance model. + +--- + +## Domain & Vertical Principle Catalogs + +Organizations and product teams adopt principle catalogs tailored to their specific domain: + +1. **[Reference Business Principles Catalogue](principles-catalogue.md):** Core guiding principles, architecture principles (AP-01 to AP-17), coding principles (CP-01 to CP-20), and lifecycle principles. +2. **Embedded & Real-Time Systems:** Prioritizes deterministic timing (AP-12), memory budgets (AP-06), safety containment (AP-10), and hardware isolation (AP-03). +3. **Cloud & Microservices:** Prioritizes horizontal scalability, event-driven decoupling (AsyncAPI), zero-downtime releases, and telemetry (EG-020). +4. **Regulated / Medical / Automotive:** Prioritizes strict requirement traceability (EG-006), hazard analysis, certification profiles (EG-023), and formal verification. + +--- + +## How Business Principles Integrate with Guidelines + +| Guideline Governance | How Business Principles Integrate | +|---|---| +| **[EG-007 Architecture Principles Governance](architecture-principles.md)** | Defines *how* ADRs cite principles and how CI enforces budget/dependency rules. The business supplies the active principle IDs (e.g., `AP-01`). | +| **[EG-012 Coding Principles Governance](coding-principles.md)** | Defines *how* code reviews and static analysis enforce coding rules. The vertical supplies language-specific coding standards (e.g., MISRA, AUTOSAR, PEP8). | +| **[EG-008 Decision Records (ADRs)](decision-records.md)** | ADRs cite active business principle IDs when recording architectural tradeoffs. | diff --git a/content/guidelines/checklists.md b/content/guidelines/checklists.md new file mode 100644 index 0000000..0bbca81 --- /dev/null +++ b/content/guidelines/checklists.md @@ -0,0 +1,490 @@ +--- +title: "Combined checklists" +status: draft +owner: "Project owner" +last-reviewed: 2026-09-29 +--- + +# Combined checklists + +## How to use these checklists + +Each guideline has its own checklist at the end of its file. This file combines them. Tick an item when it is true for your project. + +- **Adoption:** the unticked items are what is left to set up. +- **Assessment:** the percentage ticked per guideline shows where you are strong or weak. +- **Review:** revisit before each release and at each guideline review. + +For a personal project, use only the guidelines in your profile (see [personal-projects.md](personal-projects.md)). Copy a section into an issue or PR description to track it. + +## [EG-001 Guidelines governance](guidelines-governance.md) + +Profile: Standard. Owner: Architecture group. + +- [ ] Guidelines live in `docs/guidelines/`, one file per guideline, named `eg-nnn-short-title.md` (GOV-13) +- [ ] Every guideline has front matter with id, title, status, owner, version and review-by (GOV-01) +- [ ] Every guideline follows the standard template and states how compliance is checked (GOV-02, GOV-04) +- [ ] Rules have stable IDs and RFC 2119 keywords, with a rationale where it is not obvious (GOV-03) +- [ ] Each guideline is short enough to read in one sitting, with examples where they help (GOV-05, GOV-11) +- [ ] Changes are made by PR with owner and practitioner approval, and by RFC when material (GOV-06, GOV-07) +- [ ] Review-by dates are checked by CI and no guideline is overdue (GOV-08) +- [ ] Exceptions are recorded with owner, scope, reason and expiry (GOV-09) +- [ ] Deprecated guidelines are kept with a pointer to their replacement (GOV-10) +- [ ] Guideline versions follow semantic versioning (GOV-12) +- [ ] The index is generated from front matter (GOV-13) +- [ ] Each guideline has an accountable owner who handles questions and exception requests (GOV-14) +- [ ] Core rules are marked and applied identically across products, and all other rules are tailored only through the product profile (GOV-15) +- [ ] Each product has a conformance profile recording applicability, thresholds, exceptions, components and toolchain (GOV-16) + +## [EG-002 Engineering values and ways of working](values-and-ways-of-working.md) + +Profile: Full. Owner: Engineering manager. + +- [ ] Values are published and used as tie-breakers (VAL-01) +- [ ] Every component, pipeline, rig and tool has a named owner in CODEOWNERS or the ownership register (VAL-02) +- [ ] The decision rights table is current (VAL-03) +- [ ] Escalation steps and time limits are agreed, and decisions are committed to once made (VAL-04, VAL-05) +- [ ] Decisions, agreements and meeting outcomes are written down within one working day (VAL-06) +- [ ] Each component has an owning site and a deputy at another site (VAL-07) +- [ ] Cross-site agreements cover overlap hours, response time and the handover note (VAL-08) +- [ ] All sites use the same tools and conventions, and variants have an ADR (VAL-09) +- [ ] Work status is visible in the tracker and pipeline (VAL-10) +- [ ] Changes are small enough to review within one working day (VAL-11) +- [ ] Raising a risk, disagreement or mistake is treated as expected behaviour (VAL-12) + +## [EG-003 Quality policy and quality attributes](quality-policy-and-attributes.md) + +Profile: Standard. Owner: Test and quality lead. + +- [ ] A one-page quality policy is published (QUA-01) +- [ ] A quality attribute model exists with definitions and example measures (QUA-02) +- [ ] Each product has a ranked quality attribute profile, approved by the architecture lead and product owner (QUA-03) +- [ ] Quality attributes are stated as measurable scenarios in requirements (QUA-04) +- [ ] An NFR catalogue exists and requirements are selected from it (QUA-05) +- [ ] There is no separate quality phase, the gates are the quality controls (QUA-06) +- [ ] Each release has quality evidence in its evidence pack (QUA-07) +- [ ] The model and objectives are reviewed annually using field data (QUA-08) + +## [EG-004 Documentation as code and terminology](documentation-as-code.md) + +Profile: Starter. Owner: Build and tooling lead. + +- [ ] Documents are plain-text Markdown in the repository and linted (DOC-01, DOC-02) +- [ ] Documentation changes go through PR like code (DOC-03) +- [ ] Changes to behaviour, interfaces, configuration and build steps update docs in the same PR (DOC-04) +- [ ] Diagrams are Mermaid or PlantUML source (DOC-05) +- [ ] Every document has front matter with owner, status and review date (DOC-06) +- [ ] CI builds the docs, fails on broken links and lint errors, and publishes versioned docs with each release (DOC-07) +- [ ] The minimum document set exists: README, CONTRIBUTING, `AGENTS.md`, architecture overview, build and test instructions, `docs/adr/` (DOC-08) +- [ ] Generated docs are generated and never hand-edited (DOC-09) +- [ ] Documents state their question and audience, and are split by purpose (DOC-10) +- [ ] Writing is plain British English with the conclusion first (DOC-11) +- [ ] A single glossary exists and terms are used consistently (DOC-12) +- [ ] Naming conventions are followed for files, IDs and components (DOC-13) +- [ ] Approval is PR approval unless a regulation requires a signature record (DOC-14) +- [ ] Documents are structured with headings, front matter and IDs so people and agents can navigate them (DOC-15) + +## [EG-005 AI agent usage](ai-agent-usage.md) + +Profile: Starter. Owner: Build and tooling lead and Security lead. + +- [ ] `AGENTS.md` exists at the repository root with commands, conventions, boundaries and definition of done (AGT-01, AGT-02) +- [ ] Agent changes follow the same commit, PR, review and CI rules as human changes (AGT-03) +- [ ] A human approves every agent-authored change, and agents never approve or merge their own work (AGT-04) +- [ ] Agent boundaries are protected by CODEOWNERS or branch protection (AGT-05) +- [ ] AI-assisted commits carry an `Assisted-by:` trailer (AGT-06) +- [ ] The submitter understands and can explain every line submitted (AGT-07) +- [ ] An approved-tools register lists each tool and the data classes it may receive (AGT-08) +- [ ] Agents run local checks first and the PR includes the evidence (AGT-09) +- [ ] Large agent diffs are split or accompanied by a summary of intent and risk (AGT-10) +- [ ] Agent-generated tests are reviewed for whether they verify requirements (AGT-11) +- [ ] Shared prompts, agent configuration and skills are versioned and reviewed (AGT-12) +- [ ] Licence and provenance of generated code is handled under the technology guideline (AGT-13) +- [ ] `AGENTS.md` names CP-02, CP-16 and CP-20 (AGT-14) +- [ ] Agent-written documentation is checked by a human against the code or design (AGT-15) + +## [EG-006 Requirements and traceability](requirements-and-traceability.md) + +Profile: Standard. Owner: Systems engineering lead. + +- [ ] Requirement levels and allocation rules are defined (REQ-01) +- [ ] Each requirement is atomic, unambiguous, feasible, necessary, consistent and verifiable, with one verification method (REQ-02) +- [ ] Requirements are stored as text in the repository with stable `REQ--` IDs (REQ-03) +- [ ] Each requirement has a rationale, parent, owner and status (REQ-04) +- [ ] Requirements are baselined by tag at each release (REQ-05) +- [ ] IDs are referenced in commit footers, PRs and test names (REQ-06) +- [ ] Every approved requirement is allocated, implemented and verified, and every test traces to a requirement (REQ-07) +- [ ] The pipeline generates a trace report with gap gates (REQ-08) +- [ ] A requirement change triggers a change impact check (REQ-09) +- [ ] Requirements are reviewed by author, implementer and verifier before approval (REQ-10) +- [ ] Non-functional requirements are selected from the NFR catalogue (REQ-11) +- [ ] Regulatory and security requirements are captured as requirements (REQ-12) +- [ ] Work items link to a requirement or defect, or state their justification (REQ-13) + +## [EG-007 Architecture principles](architecture-principles.md) + +Profile: Standard. Owner: Architecture group. + +- [ ] Principles are reviewed and adopted for the product, and changed only by RFC (PRN-05) +- [ ] ADRs and RFCs cite the principles that drove them (PRN-01) +- [ ] Decisions that go against a principle say so and record the consequences (PRN-02) +- [ ] The precedence order is applied, and tailored per quality profile where needed (PRN-03, PRN-06) +- [ ] Dependency direction, budgets and interface compatibility are checked in CI (PRN-04) +- [ ] Layers are defined and no application code touches hardware directly (AP-02) +- [ ] Silicon, SDKs and protocols sit behind interfaces we own (AP-03) +- [ ] Flash, RAM, CPU, power and latency budgets are allocated and tracked (AP-06) +- [ ] Diagnosis, security, fail-safe behaviour and lifecycle are designed in from the start (AP-08 to AP-11) + +## [EG-008 Decision records: RFCs and ADRs](decision-records.md) + +Profile: Standard. Owner: Architecture group. + +- [ ] ADRs live in `docs/adr/`, are numbered and use the template (ADR-01, ADR-02) +- [ ] ADRs are proposed by PR and merged as `accepted` (ADR-03) +- [ ] Accepted ADRs are immutable apart from status and links, and superseding ADRs point back (ADR-04) +- [ ] At least two options and the reasons for rejection are recorded (ADR-05) +- [ ] Consequences and verification are stated (ADR-06) +- [ ] ADRs link to requirements, principles, RFCs and PRs, and commits use the `ADR:` footer (ADR-07) +- [ ] ADRs are written before or during implementation, and retrospective ones are marked (ADR-08) +- [ ] A decision log index is generated (ADR-09) +- [ ] RFCs are used for guideline changes, cross-team interfaces, major tools and release changes (RFC-01) +- [ ] RFCs are stored in `docs/rfc/`, proposed by PR, short, and discussed in writing by default (RFC-02, RFC-09, RFC-10) +- [ ] Each RFC names an author, sponsor, affected teams, comment period and decision date (RFC-03, RFC-04) +- [ ] Affected teams are notified explicitly (RFC-05) +- [ ] Each RFC covers problem, goals, proposal, alternatives, impact, risks and open questions (RFC-06) +- [ ] Outcome and rationale are recorded and ADRs created (RFC-07, RFC-08) +- [ ] An architecture forum meets on a fixed cadence and every RFC is decided, or deferred with a reason, by its decision date (RFC-11) +- [ ] Architecture review is limited to architecturally significant changes, and other decisions belong to component owners (RFC-12) + +## [EG-009 Interfaces and hardware and software co-design](interfaces-and-co-design.md) + +Profile: Standard. Owner: Architecture group. + +- [ ] Every interface has an owner and a specification in the repository (INT-01, INT-02) +- [ ] Machine-readable definitions generate code and docs, with no hand-maintained copies (INT-03) +- [ ] Interfaces are versioned, a compatibility matrix is kept, and breaking changes go through RFC (INT-04) +- [ ] Contract tests run on both sides in CI (INT-05) +- [ ] The hardware interface agreement is versioned and firmware headers are generated where possible (INT-06) +- [ ] Hardware changes are reviewed for firmware impact and recorded in the bill of materials (INT-07) +- [ ] An errata register is maintained (INT-08) +- [ ] A bring-up plan and emulation exist for new hardware (INT-09) +- [ ] Cross-site interface changes are announced with lead time (INT-10) +- [ ] Vendor documents are version-controlled or cited by exact version (INT-11) + +## [EG-010 Components, reuse and variants](components-reuse-and-variants.md) + +Profile: Full. Owner: Architecture group. + +- [ ] A component register lists every component with owner, repository, version, status and consumers (COM-01) +- [ ] Every component has a named owner and deputy (COM-02) +- [ ] Consumers depend on released, pinned versions and never on branches or unreleased commits (COM-03) +- [ ] Interfaces follow the interface guideline and versions follow semantic versioning (COM-04) +- [ ] The register is checked before building or copying, and forks are recorded in an ADR (COM-05) +- [ ] Consumers do not patch components locally, and contribute changes through the owner (COM-06) +- [ ] Changes affecting several consumers are assessed for impact, with migration guidance and deprecation notice (COM-07) +- [ ] Each component builds and tests on its own and publishes versioned artefacts (COM-08) +- [ ] Variation is expressed through documented variation points, and supported variants are recorded (COM-09) +- [ ] Each product's bill of materials lists component versions (COM-10) +- [ ] Supported combinations are tested and recorded in the compatibility matrix, and kept few (COM-11) +- [ ] Transfer, deprecation and retirement are recorded and communicated (COM-12) +- [ ] Consumers report through the owner's tracker and severity 1 and 2 defects are visible to all consumers (COM-13) + +## [EG-011 Technology selection and third-party components](technology-and-third-party.md) + +Profile: Standard. Owner: Architecture group. + +- [ ] A technology radar exists with adopt, trial, assess and hold (TEC-01) +- [ ] New dependencies and tools have an evaluation recorded in an ADR (TEC-02) +- [ ] A licence policy lists approved and prohibited licences (TEC-03) +- [ ] A manifest of third-party components exists and an SBOM is produced per release (TEC-04) +- [ ] Versions are pinned and checksums verified (TEC-05) +- [ ] Vendor SDKs are wrapped behind our interfaces and patches are tracked (TEC-06) +- [ ] Dependency updates are proposed automatically, with security patch times (TEC-07) +- [ ] Each dependency has an owner, and critical ones have an exit strategy (TEC-08) +- [ ] AI-generated code is checked for licence and provenance (TEC-09) +- [ ] Toolchain versions are pinned in containers and listed in the radar (TEC-10) +- [ ] End-of-life dates are monitored (TEC-11) + +## [EG-012 Coding principles](coding-principles.md) + +Profile: Starter. Owner: Firmware lead. + +- [ ] The coding standard derives from the principles and cites them (CPR-01) +- [ ] Review comments cite principle or rule IDs (CPR-02) +- [ ] Deviations are recorded in the code and in the deviation log (CPR-03) +- [ ] Principles that a tool can check are enforced in CI (CPR-04) +- [ ] Changes are the minimum needed and every changed line traces to the request (CP-02, CP-16) +- [ ] Wrong code is hard to write: distinct types, `const`, narrow interfaces (CP-04) +- [ ] Input is validated at boundaries and invariants are asserted (CP-05) +- [ ] There is no silent failure and every error path is defined (CP-06) +- [ ] Resource use is bounded and undefined behaviour is avoided (CP-07, CP-08) +- [ ] Warnings are treated as errors (CP-15) +- [ ] Assumptions are stated and questions raised before implementing (CP-20) + +## [EG-013 Coding standard](coding-standard.md) + +Profile: Standard. Owner: Firmware lead. + +- [ ] There is one coding standard per language in use, with rule IDs that cite principles (COD-01) +- [ ] Language version, subset and compiler are specified (COD-02) +- [ ] The standard covers every required topic or says why not (COD-03) +- [ ] The formatter is configured in the repository and checked in CI (COD-04) +- [ ] Static analysis is configured and pinned, warnings are errors, and suppressions are justified and audited (COD-05) +- [ ] Rules are classified mandatory, required or advisory, with a deviation log (COD-06) +- [ ] Each rule has a rationale, examples and a checking tool (COD-07) +- [ ] Compiler flags are standardised and identical locally and in CI (COD-08) +- [ ] A legacy baseline is tracked and there is no drive-by reformatting (COD-09) +- [ ] The standard is reviewed annually through RFC (COD-10) +- [ ] A quick reference, review checklist and agent rule list exist (COD-11) +- [ ] Scripts and tooling code also have a proportionate standard (COD-12) + +## [EG-014 Version control, commits and versioning](version-control-and-versioning.md) + +Profile: Starter. Owner: Build and tooling lead. + +- [ ] Git is the single source of truth, the structure is documented, and no secrets or build outputs are committed (VCS-01) +- [ ] Branching is trunk-based with short-lived, correctly named branches (VCS-02) +- [ ] Commits follow Conventional Commits with scopes mapped to components (VCS-03) +- [ ] Breaking changes use `!` and a `BREAKING CHANGE:` footer (VCS-04) +- [ ] Footers carry `Refs:`, `ADR:` and `Assisted-by:` where relevant (VCS-05) +- [ ] Commits are atomic and buildable (VCS-06) +- [ ] A commit-msg hook and a CI lint enforce the format (VCS-07) +- [ ] `main` has linear history and shared branches are never force-pushed (VCS-08) +- [ ] Semantic versioning is defined for the product (VCS-09) +- [ ] The version comes from the tag, is injected by the build and is never hand-edited (VCS-10) +- [ ] Release tags are annotated, signed and immutable (VCS-11) +- [ ] Version bump, changelog and release notes are generated from commits (VCS-12) +- [ ] Back-ports use `cherry-pick -x` onto supported release branches (VCS-13) +- [ ] The team understands git well enough to use it safely (VCS-14) +- [ ] Commit subjects are imperative and short, and bodies say why (VCS-15) + +## [EG-015 Pull requests and code review](pull-requests-and-review.md) + +Profile: Standard. Owner: Firmware lead. + +- [ ] All changes to protected branches go through PR with no direct pushes (PRR-01) +- [ ] PR titles follow Conventional Commits (PRR-02) +- [ ] PR descriptions follow the template (PRR-03) +- [ ] PRs have one purpose and stay around 400 changed lines or fewer, excluding generated files (PRR-04) +- [ ] CODEOWNERS define required reviewers, with two for safety, security and certified areas (PRR-05) +- [ ] Reviewers are not the author and understand the area, and agents do not approve (PRR-06) +- [ ] Required checks pass before human review (PRR-07) +- [ ] Reviewers use the review focus for the change type (PRR-08) +- [ ] First response is within one working day and stale PRs are escalated (PRR-09) +- [ ] Comments are labelled and cite rule IDs (PRR-10) +- [ ] Blocking comments are resolved and stale approvals dismissed (PRR-11) +- [ ] Draft and stacked PRs are used where helpful, with documented order (PRR-12) +- [ ] PRs merge by squash and branches are deleted (PRR-13) +- [ ] PR links are kept as review evidence (PRR-14) +- [ ] Architecture review is not a routine PR requirement, and other PRs are reviewed by component owners (PRR-15) + +## [EG-016 Pipelines and quality gates](pipelines-and-quality-gates.md) + +Profile: Standard. Owner: Build and tooling lead. + +- [ ] The pipeline is defined as code, with logic in scripts that run locally by the same command (PIP-01) +- [ ] Toolchains are pinned in versioned container images and builds are hermetic (PIP-02) +- [ ] Reproducibility is verified periodically (PIP-03) +- [ ] Stages are ordered for fast feedback and stages 1 to 4 finish within ten minutes (PIP-04) +- [ ] Artefacts are built once, promoted and stored immutably with metadata (PIP-05) +- [ ] Gates are defined by branch type (PIP-06) +- [ ] Mandatory gate content is present: warnings, analysis, budgets, coverage, docs, trace, licence and SBOM, secrets (PIP-07) +- [ ] Gates cannot be bypassed without a recorded exception (PIP-08) +- [ ] Flaky tests are quarantined within one working day (PIP-09) +- [ ] The policy is revert first, and `main` is restored within one working day (PIP-10) +- [ ] The pipeline has an owner, a rotation and service levels (PIP-11) +- [ ] SBOM, signed artefacts and provenance are produced per release (PIP-12) +- [ ] Repository settings and required checks are held as code (PIP-13) +- [ ] Shared HIL rigs are accessed through the pipeline (PIP-14) +- [ ] Pipeline duration, queue time and failure reasons are monitored (PIP-15) +- [ ] Credentials are least-privilege and short-lived, and release signing uses the two-person rule (PIP-16) + +## [EG-017 Testing principles and strategy](testing-principles-and-strategy.md) + +Profile: Standard. Owner: Test and quality lead. + +- [ ] A test strategy exists per product with levels, entry and exit criteria, tools and environments (TST-01) +- [ ] Testing is weighted towards host-level tests (TST-02) +- [ ] Test names or metadata carry requirement IDs (TST-03) +- [ ] Coverage is measured with thresholds and a ratchet, and requirement coverage is reported (TST-04) +- [ ] Simulators and test doubles are maintained and validated against hardware (TST-05) +- [ ] Negative tests and fault injection cover parsers, comms, update, storage and security (TST-06) +- [ ] Every fixed defect has a regression test and the suite runs on every merge to `main` (TST-07) +- [ ] Test data and fixtures are versioned (TST-08) +- [ ] Rigs are managed as products (TST-09) +- [ ] Results are archived per release and linked to requirements (TST-10) +- [ ] Non-functional tests run nightly or continuously (TST-11) +- [ ] Verification of high-risk requirements is independent of the author (TST-12) +- [ ] Upgrade and rollback are tested from all supported versions (TST-13) +- [ ] Exploratory testing is planned and time-boxed, with findings recorded (TST-14) +- [ ] No failing test is skipped or deleted to pass a gate (TST-15) + +## [EG-018 Defect and technical debt management](defects-and-technical-debt.md) + +Profile: Standard. Owner: Test and quality lead. + +- [ ] All defects are in the tracker with the required fields (DEF-01) +- [ ] Severity uses the agreed definitions, with priority set separately (DEF-02) +- [ ] New defects are triaged within two working days (DEF-03) +- [ ] The lifecycle is followed, verification is by someone else and closure needs a regression test (DEF-04) +- [ ] Root cause analysis is done for severity 1 and 2 defects and escapes (DEF-05) +- [ ] Each escape asks which gate should have caught it (DEF-06) +- [ ] Commits reference defects and release notes list fixes and known issues (DEF-07) +- [ ] A field defect route exists, with reproduction and release manager notification (DEF-08) +- [ ] Technical debt is recorded with label, impact, estimate and owner (DEF-09) +- [ ] Capacity is reserved for debt and old debt is reviewed (DEF-10) +- [ ] Deliberate debt has a repayment plan and expiry (DEF-11) +- [ ] Suppressions and deviations are tracked as debt (DEF-12) +- [ ] Metrics are used for improvement and not for ranking individuals (DEF-13) + +## [EG-019 Logging, debug and fault diagnostics](logging-debug-and-diagnostics.md) + +Profile: Standard. Owner: Firmware lead. + +- [ ] Logging purposes are separated, each with its own levels and retention (LOG-01) +- [ ] Log levels are defined and DEBUG and TRACE are off in production (LOG-02) +- [ ] There is one logging facility, no ad-hoc printf, and every record has a module ID and severity (LOG-03) +- [ ] The format is compact, structured and machine-parseable, with an off-target decoder if needed (LOG-04) +- [ ] A versioned event and fault code registry has an owner (LOG-05) +- [ ] Logging cost is budgeted, non-blocking and rate-limited (LOG-06) +- [ ] Persistent logs are wear-aware (LOG-07) +- [ ] Logs contain no secrets or personal data (LOG-08) +- [ ] Messages say what happened with context, and there is no logging in tight loops (LOG-09) +- [ ] Log decoding tools are versioned with the firmware (LOG-10) +- [ ] A standard debug setup is documented per platform and starts with one command (DBG-01) +- [ ] Debug and production builds are defined, debug builds are never released, and production interfaces are locked (DBG-02) +- [ ] Every fault path has defined behaviour, recorded in an ADR (DBG-03) +- [ ] Assertions guard invariants, have defined production behaviour and no side effects (DBG-04) +- [ ] The reset reason is captured at every boot (DBG-05) +- [ ] The crash dump format is defined, a build ID is embedded and symbols are archived per release (DBG-06, DBG-07) +- [ ] Runtime diagnostics exist and the watchdog design is reviewed (DBG-08, DBG-09) +- [ ] A procedure turns a field dump into a local reproduction, which becomes a test (DBG-10) +- [ ] Trace and timing tools are approved and the cost of instrumentation is documented (DBG-11) + +## [EG-020 Observability and telemetry](observability-and-telemetry.md) + +Profile: Full. Owner: Platform lead. + +- [ ] Observability requirements are defined per product (OBS-01) +- [ ] The telemetry schema is versioned, owned and follows naming conventions (OBS-02) +- [ ] Every record carries device, firmware, hardware, configuration and time context (OBS-03) +- [ ] Bandwidth, battery, storage and cost budgets exist with a sampling policy (OBS-04) +- [ ] Data minimisation, classification and retention are applied (OBS-05) +- [ ] Transport handles offline buffering, ordering, de-duplication and loss detection (OBS-06) +- [ ] Backend, dashboards and alerts are defined as code (OBS-07) +- [ ] Health indicators and objectives are defined, and every alert has an owner, runbook and action (OBS-08, OBS-13) +- [ ] Rollouts are gated by telemetry with automatic halt criteria (OBS-09) +- [ ] Field insights are reviewed monthly and fed into triage and tests (OBS-10) +- [ ] Lab and HIL environments emit the same telemetry (OBS-11) +- [ ] Data quality checks are in place (OBS-12) + +## [EG-021 Release, update and production](release-update-and-production.md) + +Profile: Standard. Owner: Release manager. + +- [ ] Release stages, roles and a checklist are defined in the repository (REL-01) +- [ ] The candidate is a tagged commit built once, and that artefact is promoted (REL-02) +- [ ] Release criteria are met: gates, no open severity 1 or 2, trace, NFRs, budgets, docs, SBOM, security, regulatory (REL-03) +- [ ] The evidence pack is generated automatically and retained (REL-04) +- [ ] Release notes are generated from commits and defects (REL-05) +- [ ] A bill of materials defines each baseline (REL-06) +- [ ] A compatibility matrix and supported upgrade paths are maintained (REL-07) +- [ ] The update mechanism is recorded in an ADR: authenticated, atomic, power-fail safe, with rollback and a downgrade policy (REL-08) +- [ ] Rollouts are staged with health gates and the ability to halt (REL-09) +- [ ] Upgrade and rollback are tested from supported versions (REL-10) +- [ ] A hotfix process is defined (REL-11) +- [ ] A support and end-of-life policy is published (REL-12) +- [ ] Production programming and provisioning are controlled, with feedback, unit traceability and secure key injection (REL-13, REL-14, REL-15) +- [ ] Artefacts, symbols and toolchains are retained for the supported life (REL-16) +- [ ] A post-release review is held (REL-17) + +## [EG-022 Security and data protection](security-and-data-protection.md) + +Profile: Starter. Owner: Security lead. + +- [ ] Security requirements and a threat model exist and are updated on architecture change (SEC-01) +- [ ] The security architecture covers trust boundaries, secure boot, encrypted comms, secure update, protected storage and debug protection (SEC-02) +- [ ] Only approved cryptography and vetted libraries are used (SEC-03) +- [ ] Key hierarchy, protected signing, provisioning, rotation and revocation are documented (SEC-04) +- [ ] No secrets are in source, and scanning runs pre-commit and in CI (SEC-05) +- [ ] Secure coding rules and security-focused review apply, and parsers are fuzzed (SEC-06) +- [ ] Security testing is performed and findings are tracked (SEC-07) +- [ ] Vulnerability intake, disclosure policy, patch times and SBOM monitoring exist (SEC-08) +- [ ] Supply chain controls are in place (SEC-09) +- [ ] A data classification scheme has handling rules, including for AI tools (SEC-10) +- [ ] Privacy by design, retention and impact assessment are applied where needed (SEC-11) +- [ ] Access is least-privilege with multi-factor authentication and quarterly review (SEC-12) +- [ ] Debug and manufacturing interfaces are protected in production (SEC-13) +- [ ] A security incident procedure is defined (SEC-14) + +## [EG-023 Safety, regulatory and certification](safety-regulatory-and-certification.md) + +Profile: Full. Owner: Regulatory lead. + +- [ ] Applicable standards and regulations are identified in a compliance register (REG-01) +- [ ] The register maps obligations to requirements, evidence, owner and status (REG-02) +- [ ] Regulatory obligations are captured as requirements and traced (REG-03) +- [ ] Evidence is produced by the pipeline into the evidence pack (REG-04) +- [ ] Changes to certified items have a change impact assessment (REG-05) +- [ ] The certified baseline is recorded and regulated software is separated where applicable (REG-06) +- [ ] The safety lifecycle is followed where an integrity level applies (REG-07) +- [ ] Tools for certified builds are qualified or validated and frozen (REG-08) +- [ ] Records are retained, audits held annually and findings tracked (REG-09) +- [ ] Test house and notified body contact is owned by a named person (REG-10) +- [ ] Regulatory changes are reviewed quarterly (REG-11) +- [ ] Technical files are maintained as docs as code (REG-12) +- [ ] Compliance training is recorded (REG-13) + +## [EG-024 Planning, delivery and improvement](planning-delivery-and-improvement.md) + +Profile: Full. Owner: Engineering manager. + +- [ ] Work is planned in time-boxed iterations or flow with a visible backlog, and every item is traceable (PLN-01) +- [ ] A Definition of Ready exists per work item type (PLN-02) +- [ ] A Definition of Done exists per type and the PR template reflects it (PLN-03) +- [ ] Estimates are ranges and forecasts state confidence (PLN-04) +- [ ] A risk register is maintained and reviewed fortnightly (PLN-05) +- [ ] Dependencies are recorded with an owner and need-by date (PLN-06) +- [ ] Escalation thresholds are defined (PLN-07) +- [ ] Capacity is reserved for support, debt and improvement (PLN-08) +- [ ] A small metric set is tracked and not used to rank individuals (PLN-09) +- [ ] Retrospectives are held and actions are tracked (PLN-10) +- [ ] Blameless postmortems follow severity 1 and 2 escapes, failed releases, outages and security incidents (PLN-11) +- [ ] Improvements are first-class backlog items (PLN-12) +- [ ] Friction with a guideline is raised as an issue or RFC (PLN-13) +- [ ] Status is derived from tracker and pipeline data (PLN-14) + +## [EG-025 Research, prototyping and experiments](research-and-prototyping.md) + +Profile: Standard. Owner: Research lead. + +- [ ] Work is classified as exploratory or production, with a question and a time-box (RSH-01) +- [ ] Exploratory code lives in designated locations and is excluded from release builds (RSH-02) +- [ ] Exemptions are understood and the non-negotiables are kept: no secrets, licences, data rules, version control, a working build (RSH-03) +- [ ] Prototype code is never copied into the product and the graduation checklist is used (RSH-04) +- [ ] An experiment record captures question, method, setup, results, conclusion and next step (RSH-05) +- [ ] Experiments are reproducible (RSH-06) +- [ ] Each exploration ends with a decision to stop, extend or graduate (RSH-07) +- [ ] The maturity level is stated (RSH-08) +- [ ] Prototypes are reviewed at the end of the time-box and unowned ones archived after 90 days (RSH-09) +- [ ] Lab equipment and boards are recorded with an owner (RSH-10) +- [ ] Findings are shared with the team (RSH-11) +- [ ] Agent use in prototypes follows the same boundaries (RSH-12) + +## [EG-026 Competence, tooling and onboarding](competence-tooling-and-onboarding.md) + +Profile: Full. Owner: Engineering manager. + +- [ ] A competence matrix covers languages, RTOS, platforms, protocols, tools, domains, security and test (CMP-01) +- [ ] Single-person areas are tracked as risks (CMP-02) +- [ ] Expected depth per tool is defined and used in training (CMP-03) +- [ ] Each tool and domain area has a champion and a how-to page (CMP-04) +- [ ] Domain experts and deputies are named and primers written (CMP-05) +- [ ] An onboarding path is documented and the environment is set up in under one day (CMP-06) +- [ ] Environment setup is tested in CI (CMP-07) +- [ ] An annual training plan has protected learning time (CMP-08) +- [ ] Knowledge sharing is regular and sessions are recorded (CMP-09) +- [ ] Rationale is captured in ADRs and docs, and a handover checklist exists for leavers (CMP-10) +- [ ] Key areas are known at two or more sites (CMP-11) +- [ ] The team is trained in safe and effective use of AI agents (CMP-12) +- [ ] New tools are introduced with training and a radar update (CMP-13) diff --git a/content/guidelines/coding-principles.md b/content/guidelines/coding-principles.md new file mode 100644 index 0000000..2f09fb5 --- /dev/null +++ b/content/guidelines/coding-principles.md @@ -0,0 +1,45 @@ +--- +id: EG-012 +title: "Coding principles governance" +status: draft +owner: "Firmware and Software lead" +version: 0.1.0 +part: "Code and build" +related: [EG-005, EG-013, EG-015] +review-by: 2027-09-30 +supersedes: +--- + +# EG-012: Coding principles governance + +- **Owner:** Firmware and Software lead +- **Applies to:** Code development by human engineers and AI agents +- **Enforcement:** Code review, static analysis, linter gates +- **Related:** [EG-005](ai-agent-usage.md), [EG-013](coding-standard.md), [EG-015](pull-requests-and-review.md) + +> **Development Note:** Generic guidelines govern code review workflows, PR quality gates, and surgical change boundaries. Specific coding principles (e.g. CP-01 to CP-20) are provided per business vertical or programming language domain in [Business Principles](business-principles.md). + +--- + +## Purpose + +This guideline defines the generic governance for coding practices across projects. Specific coding rules (e.g., MISRA for embedded C, PEP8 for Python, or strict concurrency rules for cloud microservices) are configured per business vertical and programming domain. + +--- + +## Governance Rules + +1. **CPR-01** Every project MUST select and adopt a coding principles profile from [Business Principles](business-principles.md) matching its tech stack and business vertical. +2. **CPR-02** Code changes MUST be surgical: touch only code necessary for the requirement or defect fix, preserving unrelated formatting and comments. +3. **CPR-03** Automated static analysis and linting MUST enforce coding principles in the local build and CI pipeline ([EG-016](pipelines-and-quality-gates.md)). +4. **CPR-04** AI coding agents MUST cite active coding principle IDs when explaining implementation choices in pull requests or agent task briefs. + +--- + +## Compliance + +| Rule | Checked By | +|---|---| +| **CPR-01** | Product conformance profile ([EG-001](guidelines-governance.md)) | +| **CPR-02, CPR-04** | Peer review & PR review checklist ([EG-015](pull-requests-and-review.md)) | +| **CPR-03** | CI pipeline linter & static analysis quality gate | diff --git a/content/guidelines/coding-standard.md b/content/guidelines/coding-standard.md new file mode 100644 index 0000000..d23f78b --- /dev/null +++ b/content/guidelines/coding-standard.md @@ -0,0 +1,102 @@ +--- +id: EG-013 +title: "Coding standard" +status: draft +owner: "Firmware lead" +version: 0.1.0 +part: "Code and build" +related: [EG-012, EG-016, EG-019, EG-026] +review-by: 2027-09-30 +supersedes: +--- + +# EG-013: Coding standard + +- **Owner:** Firmware lead +- **Applies to:** All languages used in product and tooling code +- **Enforcement:** Formatter, compiler, static analysis, review +- **Related:** [EG-012](coding-principles.md), [EG-016](pipelines-and-quality-gates.md), [EG-019](logging-debug-and-diagnostics.md), [EG-026](competence-tooling-and-onboarding.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Require a coding standard to exist for every language in use, and define what it must contain. This guideline does not contain the coding rules themselves. It says what the coding standard must decide, how it is enforced and how it evolves. + +## Rules + +1. **COD-01** *(core)* The team MUST maintain one coding standard per language in use (for example C, C++, Python, shell, build scripts). Each MUST derive from the coding principles ([EG-012](coding-principles.md)), and each rule MUST have an ID and cite the principle it serves. +1. **COD-02** Each standard MUST specify the language version, the permitted subset or rule set (for example MISRA C or a C++ subset), and the compiler and version. +1. **COD-03** Each standard MUST cover the topics in the table below, or state explicitly why a topic does not apply. +1. **COD-04** *(core)* Formatting MUST be applied by a formatter, not by hand. Its configuration lives in the repository, and CI checks it. +1. **COD-05** *(core)* Static analysis tools, rule sets and severities MUST be configured in the repository and pinned. Warnings MUST be errors. Suppressions MUST be inline, carry the rule ID and a justification, and be audited. +1. **COD-06** Rules MUST be classified as `mandatory`, `required` or `advisory`. Deviations follow GOV-09 and are recorded in a deviation log. +1. **COD-07** Each rule MUST give its rationale, a compliant and a non-compliant example, and identify the tool that checks it or state that it is checked in review. +1. **COD-08** Compiler flags MUST be standardised: a baseline warning set, hardening options, sanitisers for host builds, and documented effects of optimisation and link-time optimisation on debuggability. Flags are identical locally and in CI. +1. **COD-09** Standards MUST apply to new and changed code. Legacy code MUST be tracked against a baseline that is reduced over time. Unrelated legacy code MUST NOT be reformatted in feature PRs (CP-16). +1. **COD-10** Standards MUST be reviewed at least annually and changed through an RFC. Each change MUST include a migration approach. +1. **COD-11** Standards MUST be short enough to use. A one-page quick reference and a review checklist SHOULD be generated from them, and an agent-oriented condensed rule list SHOULD be kept in the repository ([EG-005](ai-agent-usage.md)). +1. **COD-12** Tooling code and scripts MUST have a proportionate standard. Lack of a standard for scripts is not an exemption. + +## What a coding standard must decide + +| Topic | The standard must decide | +|-------|--------------------------| +| Naming | Conventions for files, types, functions, variables, constants, macros, and module prefixes | +| Layout | Formatting configuration, line length, file organisation | +| Structure | Module and header structure, include rules, dependency direction | +| Types | Fixed-width integers, enumerations, booleans, signed and unsigned arithmetic, casts | +| Constants and macros | When macros are allowed, constant definitions, magic numbers | +| Error handling | Return conventions, error code scheme, propagation, failure behaviour | +| Assertions | Where required, production behaviour, side-effect ban ([EG-019](logging-debug-and-diagnostics.md)) | +| Memory | Allocation policy, stack limits, alignment, buffer handling, initialisation | +| Pointers | Permitted uses, ownership, null handling, pointer arithmetic | +| Concurrency | Rules for ISRs, shared data, locking, RTOS primitives, priorities | +| Hardware access | Register access, `volatile`, memory barriers, DMA and cache handling | +| Portability | Compiler extensions, endianness, target-specific code isolation | +| Undefined behaviour | Specific hazards to avoid and how they are detected | +| Comments | What to comment, format, requirement and ADR references | +| Banned constructs | Unsafe functions and language features, with alternatives | +| Testing hooks | Seams for test doubles, test-only code rules | +| Security | Secret handling, constant-time operations, input validation | + +## Compliance + +| Rule | Checked by | +|------|------------| +| COD-04 | CI: formatter check | +| COD-05, COD-06 | CI: analyser with warnings as errors, suppression audit report | +| COD-08 | Build configuration shared by local and CI builds | +| COD-09 | Baseline tracking report | +| COD-10 | Review-by date and RFC history | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] There is one coding standard per language in use, with rule IDs that cite principles (COD-01) +- [ ] Language version, subset and compiler are specified (COD-02) +- [ ] The standard covers every required topic or says why not (COD-03) +- [ ] The formatter is configured in the repository and checked in CI (COD-04) +- [ ] Static analysis is configured and pinned, warnings are errors, and suppressions are justified and audited (COD-05) +- [ ] Rules are classified mandatory, required or advisory, with a deviation log (COD-06) +- [ ] Each rule has a rationale, examples and a checking tool (COD-07) +- [ ] Compiler flags are standardised and identical locally and in CI (COD-08) +- [ ] A legacy baseline is tracked and there is no drive-by reformatting (COD-09) +- [ ] The standard is reviewed annually through RFC (COD-10) +- [ ] A quick reference, review checklist and agent rule list exist (COD-11) +- [ ] Scripts and tooling code also have a proportionate standard (COD-12) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Choose a formatter and a linter per language, commit their configuration and fail CI on warnings. That is most of the value. Write topic entries only for topics you actually hit. + +### AI agents + +- Run the formatter and linter before every commit and do not hand-format (COD-04). +- Do not add suppressions without a rule ID and justification (COD-05). +- Use the flags in the build configuration unchanged (COD-08). +- Do not reformat unrelated code (COD-09). +- Follow the condensed rule list in the repository (COD-11). diff --git a/content/guidelines/competence-tooling-and-onboarding.md b/content/guidelines/competence-tooling-and-onboarding.md new file mode 100644 index 0000000..1d79e60 --- /dev/null +++ b/content/guidelines/competence-tooling-and-onboarding.md @@ -0,0 +1,94 @@ +--- +id: EG-026 +title: "Competence, tooling and onboarding" +status: draft +owner: "Engineering manager" +version: 0.1.0 +part: "Delivery, research and people" +related: [EG-002, EG-005, EG-011, EG-013, EG-014] +review-by: 2027-09-30 +supersedes: +--- + +# EG-026: Competence, tooling and onboarding + +- **Owner:** Engineering manager +- **Applies to:** All team members, all sites, and all tools and domains used +- **Enforcement:** Competence matrix reviews, onboarding audit, review +- **Related:** [EG-002](values-and-ways-of-working.md), [EG-005](ai-agent-usage.md), [EG-011](technology-and-third-party.md), [EG-013](coding-standard.md), [EG-014](version-control-and-versioning.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Keep skills, tool mastery and domain knowledge deliberately maintained. The team should understand the tools and domain it depends on, and no capability should sit with a single person or site. + +## Rules + +1. **CMP-01** A competence matrix MUST be maintained covering languages, RTOS, silicon platforms, protocols, tools, domain areas, security and test, with levels 0 to 4. It is reviewed twice a year. +1. **CMP-02** Areas with a single competent person MUST be tracked as risks (PLN-05), with a plan to build cover. +1. **CMP-03** Expected depth for each tool MUST be defined (below) and used in onboarding and training plans. +1. **CMP-04** Each tool and domain area MUST have a champion who maintains a short "how we use X" page in `docs/how-to/` and answers questions. Champions are listed in the ownership register. +1. **CMP-05** Each domain area (protocols, standards, regulation, hardware, security, customer use) MUST have a named subject matter expert and deputy, and a short primer document. +1. **CMP-06** Onboarding MUST follow a documented path: environment set up in under one day by one command, a first-week checklist, a buddy, a guided walkthrough of these guidelines (not just a link) and a first PR through the full process. +1. **CMP-07** The environment setup MUST be tested in CI, so that onboarding does not rot. +1. **CMP-08** An annual training plan MUST exist, with protected learning time (proposed 5 percent), recorded by person. +1. **CMP-09** Knowledge sharing MUST be regular: technical talks, cross-site pairing or rotations and design walkthroughs of major components. Sessions SHOULD be recorded and stored. +1. **CMP-10** Rationale MUST be captured in ADRs and documents, not in people's heads. A handover checklist MUST be used when someone leaves or changes role. +1. **CMP-11** Key areas MUST have knowledge at two or more sites (VAL-07). +1. **CMP-12** Team members MUST be trained in effective and safe use of AI agents, including reviewing agent output and handling data ([EG-005](ai-agent-usage.md), [EG-022](security-and-data-protection.md)). +1. **CMP-13** New tool versions or tools MUST be introduced with training and a radar update (TEC-01). + +## Expected depth by tool and domain + +| Area | Expected understanding | +|------|------------------------| +| Git | Object model (commits, refs, the graph), rebase versus merge, reflog for recovery, bisect, worktrees, rerere, `cherry-pick -x`, history rewriting rules | +| Build system and linker | Build system internals, linker scripts, map files, memory layout, reproducibility | +| Compiler | Flags, optimisation and link-time optimisation effects, undefined behaviour, debuggability | +| Static analysis and test tools | What each checks, false positive profile, configuration, suppression rules | +| Debug and trace | Probes, breakpoints and trace, crash dump decoding, timing analysis | +| Target platform | Peripherals, memory map, errata, boot process | +| RTOS | Scheduling, priorities, synchronisation, memory model, interrupt handling | +| Bootloader and update | Chain of trust, image formats, atomic update, rollback | +| CI and containers | Pipeline structure, container toolchains, artefact handling | +| Domain and standards | Protocols, regulatory context and use cases for our products | + +## Compliance + +| Rule | Checked by | +|------|------------| +| CMP-01, CMP-02 | Competence matrix review, risk register | +| CMP-04, CMP-05 | Ownership register and how-to page check | +| CMP-06, CMP-07 | Onboarding feedback, CI environment setup job | +| CMP-08 | Training plan review | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] A competence matrix covers languages, RTOS, platforms, protocols, tools, domains, security and test (CMP-01) +- [ ] Single-person areas are tracked as risks (CMP-02) +- [ ] Expected depth per tool is defined and used in training (CMP-03) +- [ ] Each tool and domain area has a champion and a how-to page (CMP-04) +- [ ] Domain experts and deputies are named and primers written (CMP-05) +- [ ] An onboarding path is documented and the environment is set up in under one day (CMP-06) +- [ ] Environment setup is tested in CI (CMP-07) +- [ ] An annual training plan has protected learning time (CMP-08) +- [ ] Knowledge sharing is regular and sessions are recorded (CMP-09) +- [ ] Rationale is captured in ADRs and docs, and a handover checklist exists for leavers (CMP-10) +- [ ] Key areas are known at two or more sites (CMP-11) +- [ ] The team is trained in safe and effective use of AI agents (CMP-12) +- [ ] New tools are introduced with training and a radar update (CMP-13) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Full** profile (see [personal-projects.md](personal-projects.md)). Treat the expected-depth table as your learning plan, especially git internals and the debug tools. Keep a short learning log and a `docs/how-to/` page for anything you had to relearn twice. A one-command environment setup is worth doing even solo, and it is what lets agents work reliably. + +### AI agents + +- Write or update a `docs/how-to/` page when you discover a non-obvious procedure (CMP-04, CMP-10). +- Keep the environment setup script working and tested (CMP-07). +- Point to ADRs and docs for rationale and do not rely on chat history (CMP-10). diff --git a/content/guidelines/components-reuse-and-variants.md b/content/guidelines/components-reuse-and-variants.md new file mode 100644 index 0000000..ef761e6 --- /dev/null +++ b/content/guidelines/components-reuse-and-variants.md @@ -0,0 +1,106 @@ +--- +id: EG-010 +title: "Components, reuse and variants" +status: draft +owner: "Architecture group" +version: 0.1.0 +part: "Requirements, architecture and design" +related: [EG-007, EG-009, EG-011, EG-014, EG-021] +review-by: 2027-09-30 +supersedes: +--- + +# EG-010: Components, reuse and variants + +- **Owner:** Architecture group +- **Applies to:** Every component, whether built in the product, reused from another product, or third party +- **Enforcement:** Component register checks in CI, dependency checks in the pipeline, review +- **Related:** [EG-007](architecture-principles.md), [EG-009](interfaces-and-co-design.md), [EG-011](technology-and-third-party.md), [EG-014](version-control-and-versioning.md), [EG-021](release-update-and-production.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Define how the parts of a product are named, owned, versioned and consumed. The same rules work whether a product is self-contained, shares components with other products, or uses third-party code. Sharing is treated as a spectrum and not as a requirement: the rules apply to a component with one consumer and scale up as consumers are added. + +## Rules + +1. **COM-01** *(core)* Every component MUST be listed in a component register recording its owner, deputy, repository, current version, status (`active`, `maintenance`, `deprecated`, `retired`) and consumers. Third-party components are listed with the same fields ([EG-011](technology-and-third-party.md)). A product keeps its own register, or uses a shared register where products share components. +1. **COM-02** *(core)* Every component MUST have a named owner and a deputy. A component with no owner MUST be escalated to the architecture group within one month. +1. **COM-03** *(core)* Consumers MUST depend on released, versioned artefacts. They MUST NOT depend on a branch, an unreleased commit or a copy of the source. +1. **COM-04** Component interfaces MUST follow [EG-009](interfaces-and-co-design.md), and component versions MUST follow semantic versioning ([EG-014](version-control-and-versioning.md)). +1. **COM-05** Before building or copying functionality, the author MUST check the register. A fork of a component MUST be recorded in an ADR, with either a plan to converge or the reason not to. +1. **COM-06** A consumer MUST NOT patch a component locally. Changes are contributed to the owner, who reviews them under the same rules as any other change. An urgent local patch needs an exception (GOV-09) with an expiry date. +1. **COM-07** A change to a component with more than one consumer MUST include a consumer impact assessment. A breaking change MUST come with migration guidance and a deprecation notice agreed with the consumers. +1. **COM-08** Each component MUST build and pass its tests without any consumer present, and MUST publish versioned artefacts with release notes generated from Conventional Commits ([EG-014](version-control-and-versioning.md)). +1. **COM-09** Variation between products or configurations SHOULD be expressed through documented variation points (build-time configuration, runtime configuration or an extension interface) and not by branching or copying. Supported variants MUST be recorded in the register. +1. **COM-10** Each product MUST have a bill of materials listing the version of every component it ships ([EG-021](release-update-and-production.md)). +1. **COM-11** Where a component is used in more than one configuration, the supported combinations MUST be tested and recorded in a compatibility matrix. The number of supported combinations SHOULD be kept small. +1. **COM-12** Transfer of ownership, deprecation and retirement of a component MUST be recorded in the register and communicated to every consumer. +1. **COM-13** Consumers MUST report defects and requests through the owner's tracker. Severity 1 and 2 defects ([EG-018](defects-and-technical-debt.md)) SHOULD be visible to all consumers. + +## How the rules scale + +| Situation | What applies | What can be relaxed | +|-----------|--------------|---------------------| +| No sharing: the component sits inside one product | COM-01, COM-02, COM-03, COM-10 | Compatibility matrix, deprecation notices and consumer impact assessment | +| A few internal consumers | Also COM-04 to COM-08 and COM-12 | Formal variation points, if there is one variant | +| Many consumers across products | All rules | Nothing, and the owner needs time set aside to serve consumers | +| Third-party component | COM-01, COM-03, COM-05, COM-06, COM-10 and [EG-011](technology-and-third-party.md) | Owner is the internal sponsor, who tracks upstream releases and defects | + +A component that gains a second consumer moves down this table. The owner and deputy then agree the additional rules with the new consumer and record the change in the register. + +## Component register example + +```yaml +components: + - name: bootloader + owner: "A. Owner" + deputy: "B. Deputy" + repository: "git@example.com:firmware/bootloader.git" + version: 2.4.1 + status: active + consumers: [product-a, product-b] + interfaces: [IF-BOOT-001] + variation-points: [flash-layout, signature-scheme] +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| COM-01, COM-02, COM-12 | CI: register schema validation and ownership check | +| COM-03, COM-08, COM-10 | CI: dependency resolution rejects branch and unreleased references, and generates the bill of materials | +| COM-04, COM-07, COM-11 | Interface compatibility tests, review by the component owner | +| COM-05, COM-06, COM-09, COM-13 | Review, ADR history, tracker audit | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] A component register lists every component with owner, repository, version, status and consumers (COM-01) +- [ ] Every component has a named owner and deputy (COM-02) +- [ ] Consumers depend on released, pinned versions and never on branches or unreleased commits (COM-03) +- [ ] Interfaces follow the interface guideline and versions follow semantic versioning (COM-04) +- [ ] The register is checked before building or copying, and forks are recorded in an ADR (COM-05) +- [ ] Consumers do not patch components locally, and contribute changes through the owner (COM-06) +- [ ] Changes affecting several consumers are assessed for impact, with migration guidance and deprecation notice (COM-07) +- [ ] Each component builds and tests on its own and publishes versioned artefacts (COM-08) +- [ ] Variation is expressed through documented variation points, and supported variants are recorded (COM-09) +- [ ] Each product's bill of materials lists component versions (COM-10) +- [ ] Supported combinations are tested and recorded in the compatibility matrix, and kept few (COM-11) +- [ ] Transfer, deprecation and retirement are recorded and communicated (COM-12) +- [ ] Consumers report through the owner's tracker and severity 1 and 2 defects are visible to all consumers (COM-13) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Full** profile (see [personal-projects.md](personal-projects.md)). Only needed if you reuse your own code across projects. Publish it as a versioned package or tagged repository and pin it, instead of copying it between projects. Keep a short list of the components you maintain and their versions. + +### AI agents + +- Do not copy code from another component or repository into this one, depend on a pinned released version instead (COM-05, COM-03). +- Do not patch a component you consume, propose the change to its owner (COM-06). +- Do not depend on a branch or an unreleased commit (COM-03). +- When you change a component that has other consumers, state the consumer impact and compatibility (COM-07). diff --git a/content/guidelines/core-rules.md b/content/guidelines/core-rules.md new file mode 100644 index 0000000..bd31351 --- /dev/null +++ b/content/guidelines/core-rules.md @@ -0,0 +1,131 @@ +--- +title: "Core rules" +status: draft +owner: "Architecture group" +last-reviewed: 2026-09-29 +--- + +# Core rules + +## Purpose + +Core rules apply to every product without tailoring. They are the small set of rules whose value comes from being the same everywhere: traceability, commit and version conventions, secrets, supply chain and AI agent controls. Everything else is a default that a product may tailor through its conformance profile (GOV-15, GOV-16). + +There are 61 core rules out of 348 in total. The full wording, rationale and compliance checks are in each guideline. Core rules are marked *(core)* there. + +## How core rules change + +- A new core rule or a change to one needs an RFC ([EG-008](decision-records.md)) with every product's technical lead consulted, and a MAJOR or MINOR version change to the guideline (GOV-12). +- A product cannot tailor a core rule. A time-limited exception needs the guideline owner's approval (GOV-09). +- If products keep asking for exceptions to a core rule, the rule is probably wrong. Review it. + +## Core rules by guideline + +### [EG-001 Guidelines governance](guidelines-governance.md) + +- **GOV-01** Every guideline MUST have a unique ID (`EG-nnn`), a named owner, a status (`draft`, `active`, `deprecated`), a version and a review-by date in its front matter. +- **GOV-06** New guidelines and material changes MUST go through an RFC ([EG-008](decision-records.md)), unless the change is editorial. +- **GOV-07** Changes MUST be made by pull request, approved by the owner and by at least one practitioner subject to the rule. +- **GOV-09** A deviation MUST be recorded as an exception with owner approval, scope, reason and expiry date. A permanent deviation means the rule is wrong and SHOULD be changed. +- **GOV-15** Rules MUST be classified as core or tailorable. Core rules apply to every product without tailoring and are marked *(core)*. All other rules, and all numeric values, are defaults that a product MAY tailor through its conformance profile (GOV-16). The core set is kept small so that products stay consistent where it matters and free elsewhere. +- **GOV-16** Each product MUST have a conformance profile recording which guidelines apply, any tailored rules and values with the reason, approved exceptions, the components it uses and provides (COM-01), and its toolchain. The product's technical lead owns the profile and approves tailoring of non-core rules, and the architecture group is informed at its forum (RFC-11). Exceptions to core rules need the guideline owner's approval (GOV-09). + +### [EG-004 Documentation as code and terminology](documentation-as-code.md) + +- **DOC-01** Documentation MUST be stored as plain text in the same repository as the thing it describes, or in a designated documentation repository referenced from it. +- **DOC-03** Changes to documentation MUST go through the same PR and review process as code. +- **DOC-04** A change that alters behaviour, an interface, a configuration option or a build step MUST update the affected documentation in the same PR. +- **DOC-08** Each repository MUST provide the minimum document set: `README`, `CONTRIBUTING`, `AGENTS.md`, architecture overview, build and test instructions, and `docs/adr/`. + +### [EG-005 AI agent usage](ai-agent-usage.md) + +- **AGT-01** Every repository MUST have an `AGENTS.md` at its root, reviewed and owned like code. +- **AGT-03** Agent-authored changes MUST follow the same commit, PR, review and CI rules as human changes. There is no fast lane. +- **AGT-04** An agent MUST NOT approve or merge its own changes. At least one human reviewer who understands the change MUST approve. +- **AGT-05** Agents MUST NOT act in the areas listed in the boundaries table (below) without explicit human instruction in the task. +- **AGT-06** AI-assisted commits MUST carry a trailer, for example `Assisted-by: `, so that the extent of assistance is auditable. +- **AGT-07** Humans are accountable for all code they submit, whoever or whatever wrote it. The submitter MUST understand and be able to explain the change. +- **AGT-08** Data MUST NOT be sent to an agent service that is not approved for its classification ([EG-022](security-and-data-protection.md)). The approved tools and their permitted data classes are held in one place, owned by the security lead. + +### [EG-006 Requirements and traceability](requirements-and-traceability.md) + +- **REQ-03** Requirements MUST be stored as text in the repository beside the design, with stable IDs of the form `REQ--`. IDs are never reused. +- **REQ-06** Requirement IDs MUST be referenced in commit footers, PR descriptions, test names and, where non-obvious, code comments and ADRs. + +### [EG-008 Decision records: RFCs and ADRs](decision-records.md) + +- **ADR-01** ADRs MUST be stored in `docs/adr/` in the repository they affect, numbered sequentially and named `NNNN-short-title.md`. +- **ADR-03** An ADR MUST be proposed by PR. Merging with status `accepted` is the record of approval. +- **ADR-04** An accepted ADR MUST be immutable apart from status and links. A changed decision is a new ADR that supersedes the old one, and the old ADR MUST be updated to point to it. +- **RFC-01** An RFC MUST be used for new or changed guidelines, cross-team interface changes, adoption of major tools or dependencies, and changes to the release or update process. + +### [EG-010 Components, reuse and variants](components-reuse-and-variants.md) + +- **COM-01** Every component MUST be listed in a component register recording its owner, deputy, repository, current version, status (`active`, `maintenance`, `deprecated`, `retired`) and consumers. Third-party components are listed with the same fields ([EG-011](technology-and-third-party.md)). A product keeps its own register, or uses a shared register where products share components. +- **COM-02** Every component MUST have a named owner and a deputy. A component with no owner MUST be escalated to the architecture group within one month. +- **COM-03** Consumers MUST depend on released, versioned artefacts. They MUST NOT depend on a branch, an unreleased commit or a copy of the source. + +### [EG-011 Technology selection and third-party components](technology-and-third-party.md) + +- **TEC-03** A licence policy MUST list approved licences and prohibited ones. Any other licence needs legal review before use. +- **TEC-04** All third-party components MUST be recorded in a manifest (name, version, source, licence, checksum, owner). An SBOM MUST be generated for each release. +- **TEC-05** Versions MUST be pinned and checksums verified. Floating versions and unpinned downloads in builds MUST NOT be used. + +### [EG-012 Coding principles](coding-principles.md) + +- **CPR-01** The coding standard MUST derive from these principles, and each of its rules SHOULD cite the principle it serves. + +### [EG-013 Coding standard](coding-standard.md) + +- **COD-01** The team MUST maintain one coding standard per language in use (for example C, C++, Python, shell, build scripts). Each MUST derive from the coding principles ([EG-012](coding-principles.md)), and each rule MUST have an ID and cite the principle it serves. +- **COD-04** Formatting MUST be applied by a formatter, not by hand. Its configuration lives in the repository, and CI checks it. +- **COD-05** Static analysis tools, rule sets and severities MUST be configured in the repository and pinned. Warnings MUST be errors. Suppressions MUST be inline, carry the rule ID and a justification, and be audited. + +### [EG-014 Version control, commits and versioning](version-control-and-versioning.md) + +- **VCS-01** Git MUST be the single source of truth. The repository structure MUST be documented, and the choice of monorepo or multi-repo (with its multi-repo tool) recorded in an ADR. Large binaries MUST use LFS or an artefact store. Secrets and build outputs MUST NOT be committed. +- **VCS-03** Commit messages MUST follow Conventional Commits, using the allowed types (below) and scopes mapped to components in a scope file. +- **VCS-04** A breaking change MUST be marked with `!` and a `BREAKING CHANGE:` footer. +- **VCS-05** Footers MUST include `Refs:` for requirement or ticket IDs. They SHOULD include `ADR:` where relevant, and MUST include `Assisted-by:` where AI assisted (AGT-06). +- **VCS-07** Commit format MUST be enforced by a commit-msg hook (installed by one command) and by a CI lint. +- **VCS-08** `main` MUST have linear history through squash or rebase merging. Force-push to shared branches MUST NOT be used. History rewriting is allowed only on your own unmerged branches. +- **VCS-09** Firmware versions MUST follow semantic versioning as defined below. +- **VCS-10** The version MUST have a single source, derived from the Git tag by the build and injected into the artefact. Version strings MUST NOT be edited by hand. Builds MUST embed the commit hash and a dirty flag, and the device MUST report its version. +- **VCS-11** Release tags MUST be annotated, signed, immutable, and named `vX.Y.Z` (pre-releases `vX.Y.Z-rc.N`). Tags trigger the release pipeline. + +### [EG-015 Pull requests and code review](pull-requests-and-review.md) + +- **PRR-01** All changes to protected branches MUST be made by PR. Direct pushes MUST NOT be allowed, including for owners. +- **PRR-02** The PR title MUST follow Conventional Commits and becomes the squash commit message. +- **PRR-05** CODEOWNERS MUST define required reviewers for each path. At least one owner approves every PR, and safety, security and certified areas need two. +- **PRR-06** Reviewers MUST be someone other than the author who understands the area. An agent MUST NOT count as an approver (AGT-04). + +### [EG-016 Pipelines and quality gates](pipelines-and-quality-gates.md) + +- **PIP-01** Pipelines MUST be defined as code in the repository. Logic MUST live in scripts and the build system, runnable locally with the same command. CI only orchestrates. +- **PIP-02** Toolchains MUST be pinned in versioned container images. Builds MUST be hermetic, with network access limited to declared package caches. +- **PIP-05** Artefacts MUST be built once and promoted. The artefact that is tested MUST be the artefact that is released. It is stored immutably with metadata (commit, pipeline run, toolchain digest). +- **PIP-08** Gates MUST NOT be bypassed. An override needs a recorded exception (GOV-09) approved by the pipeline owner. +- **PIP-13** Repository settings, branch protection and required checks MUST be held as code and applied by tooling (policy as code). + +### [EG-017 Testing principles and strategy](testing-principles-and-strategy.md) + +- **TST-03** Test names or metadata MUST include the requirement IDs they verify (REQ-06). The verification method recorded for each requirement MUST be executed. +- **TST-07** Every fixed defect MUST have a regression test (DEF-04). The regression suite MUST run on every merge to `main`. + +### [EG-018 Defect and technical debt management](defects-and-technical-debt.md) + +- **DEF-01** All defects MUST be recorded in the tracker. Private lists MUST NOT be used. A record includes version found, environment, steps, expected and actual results, severity, component, linked requirement and logs or dumps. +- **DEF-02** Severity MUST follow the definitions below. Priority is set separately by the product owner. + +### [EG-021 Release, update and production](release-update-and-production.md) + +- **REL-02** A release candidate MUST be a tagged commit built once by the pipeline. Only that artefact is promoted (PIP-05). +- **REL-05** Release notes MUST be generated from conventional commits and closed defects, and MUST include breaking changes, upgrade notes and known issues. + +### [EG-022 Security and data protection](security-and-data-protection.md) + +- **SEC-03** Only approved cryptographic algorithms and vetted libraries MUST be used, from a list owned by the security lead. Home-grown cryptography MUST NOT be used. +- **SEC-05** Secrets MUST NOT be in source. Secret scanning MUST run pre-commit and in CI. A leaked secret is an incident and MUST be rotated. +- **SEC-08** Vulnerability management MUST have an intake channel, a coordinated disclosure policy, severity assessment and patch times by severity (below). The SBOM MUST be monitored against vulnerability feeds. +- **SEC-10** A data classification scheme MUST be in place (below), with handling rules for storage, logs, telemetry, sharing and AI tools. diff --git a/content/guidelines/decision-records.md b/content/guidelines/decision-records.md new file mode 100644 index 0000000..30edda4 --- /dev/null +++ b/content/guidelines/decision-records.md @@ -0,0 +1,155 @@ +--- +id: EG-008 +title: "Decision records: RFCs and ADRs" +status: draft +owner: "Architecture group" +version: 0.1.0 +part: "Requirements, architecture and design" +related: [EG-001, EG-002, EG-004, EG-007] +review-by: 2027-09-30 +supersedes: +--- + +# EG-008: Decision records: RFCs and ADRs + +- **Owner:** Architecture group +- **Applies to:** All significant technical, process and guideline decisions +- **Enforcement:** Review, CI (format and link checks) +- **Related:** [EG-001](guidelines-governance.md), [EG-002](values-and-ways-of-working.md), [EG-004](documentation-as-code.md), [EG-007](architecture-principles.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Give significant proposals a structured, time-boxed and inclusive way to gather input across sites and disciplines (the RFC). Then record the outcome with its context, so that it is not re-litigated, misremembered or lost (the ADR). + +## RFC or ADR? + +The RFC is the discussion. The ADR is the outcome. Small decisions with an obvious owner go straight to an ADR. Anything with real disagreement, cross-team impact or a high cost of being wrong starts as an RFC, and produces one or more ADRs. + +A decision needs an ADR if it is hard or costly to reverse, or affects more than one team or component. Examples are the choice of RTOS, bootloader, update mechanism, cryptographic scheme or flash layout, and an interface between components. Adopting or dropping a significant tool or dependency needs one too, as does a deviation from a guideline that is likely to recur. + +## Rules + +1. **ADR-01** *(core)* ADRs MUST be stored in `docs/adr/` in the repository they affect, numbered sequentially and named `NNNN-short-title.md`. +1. **ADR-02** ADRs MUST use the standard template (below). +1. **ADR-03** *(core)* An ADR MUST be proposed by PR. Merging with status `accepted` is the record of approval. +1. **ADR-04** *(core)* An accepted ADR MUST be immutable apart from status and links. A changed decision is a new ADR that supersedes the old one, and the old ADR MUST be updated to point to it. +1. **ADR-05** Each ADR MUST record at least two considered options, including "do nothing" where relevant, with reasons for rejection. +1. **ADR-06** Each ADR MUST state its consequences, including negative ones, and how the decision will be verified. +1. **ADR-07** ADRs SHOULD link to the requirements they satisfy, the principles that apply (PRN-01), the RFC that led to them, and the PRs that implement them. Implementing commits SHOULD carry an `ADR:` footer. +1. **ADR-08** ADRs SHOULD be written before or during implementation. Retrospective ADRs are acceptable for undocumented existing decisions and MUST be marked as such. +1. **ADR-09** A decision log index MUST be generated from ADR front matter. +1. **RFC-01** *(core)* An RFC MUST be used for new or changed guidelines, cross-team interface changes, adoption of major tools or dependencies, and changes to the release or update process. +1. **RFC-02** RFCs MUST be stored in `docs/rfc/`, numbered, and proposed by PR so that discussion is attached to the text. +1. **RFC-03** Each RFC MUST name an author, a sponsor with authority to decide, and the affected teams and sites. +1. **RFC-04** Each RFC MUST have a comment period of at least five working days (longer when several time zones are involved) and a stated decision date. +1. **RFC-05** Affected teams MUST be notified explicitly. Silence from a team that was not told is not consent. +1. **RFC-06** Each RFC MUST include: problem statement, goals and non-goals, proposal, alternatives, impact (including migration and cost), risks and open questions. +1. **RFC-07** The sponsor MUST record the outcome (`accepted`, `rejected`, `withdrawn`) with a short rationale and a summary of unresolved objections. +1. **RFC-08** An accepted RFC MUST result in one or more ADRs and, where relevant, guideline updates. The RFC then becomes read-only. +1. **RFC-09** RFCs SHOULD be short. A proposal that needs more than five pages should be split. +1. **RFC-10** Written comment is the default. Meetings are for resolving disagreements, and their outcomes MUST be written back to the RFC (VAL-06). +1. **RFC-11** The architecture group MUST run a forum on a fixed cadence (proposed: fortnightly) where open RFCs are decided. Every RFC has a decision date, and an RFC not decided by then MUST be decided, deferred with a written reason, or withdrawn. +1. **RFC-12** Architecture review MUST be limited to architecturally significant changes: a cross-component interface, an architecture principle, a quality attribute target, a component boundary or the adoption of a technology. Other decisions belong to the component owner (VAL-02). + +## ADR template + +```markdown +--- +id: ADR-0000 +title: Short imperative title +status: proposed | accepted | rejected | deprecated | superseded +date: YYYY-MM-DD +deciders: [names or roles] +principles: [AP-00] +requirements: [REQ-...] +rfc: +supersedes: +superseded-by: +--- + +## Context + +What is the problem, and what forces (technical, regulatory, schedule) apply? + +## Decision drivers + +- Driver 1 +- Driver 2 + +## Options considered + +1. Option A: summary, pros, cons +1. Option B: summary, pros, cons + +## Decision + +We will do X, because Y. + +## Consequences + +- Positive: +- Negative: +- Follow-up actions: + +## Verification + +How will we know this decision is working? +``` + +## RFC lifecycle + +```mermaid +stateDiagram-v2 + [*] --> Draft + Draft --> InReview: PR opened + InReview --> Accepted: Sponsor decides + InReview --> Rejected: Sponsor decides + InReview --> Withdrawn: Author withdraws + Accepted --> Implemented: ADRs and work items created +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| ADR-01, ADR-02, ADR-09 | CI: ADR front matter and heading validation, index generation | +| ADR-03, ADR-04 | Branch protection, CI: accepted ADR content immutability check | +| ADR-07, RFC-08 | Review, PR template item | +| RFC-11, RFC-12 | Forum calendar and decision log review | +| RFC-01 to RFC-07 | RFC template validation in CI, sponsor review | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] ADRs live in `docs/adr/`, are numbered and use the template (ADR-01, ADR-02) +- [ ] ADRs are proposed by PR and merged as `accepted` (ADR-03) +- [ ] Accepted ADRs are immutable apart from status and links, and superseding ADRs point back (ADR-04) +- [ ] At least two options and the reasons for rejection are recorded (ADR-05) +- [ ] Consequences and verification are stated (ADR-06) +- [ ] ADRs link to requirements, principles, RFCs and PRs, and commits use the `ADR:` footer (ADR-07) +- [ ] ADRs are written before or during implementation, and retrospective ones are marked (ADR-08) +- [ ] A decision log index is generated (ADR-09) +- [ ] RFCs are used for guideline changes, cross-team interfaces, major tools and release changes (RFC-01) +- [ ] RFCs are stored in `docs/rfc/`, proposed by PR, short, and discussed in writing by default (RFC-02, RFC-09, RFC-10) +- [ ] Each RFC names an author, sponsor, affected teams, comment period and decision date (RFC-03, RFC-04) +- [ ] Affected teams are notified explicitly (RFC-05) +- [ ] Each RFC covers problem, goals, proposal, alternatives, impact, risks and open questions (RFC-06) +- [ ] Outcome and rationale are recorded and ADRs created (RFC-07, RFC-08) +- [ ] An architecture forum meets on a fixed cadence and every RFC is decided, or deferred with a reason, by its decision date (RFC-11) +- [ ] Architecture review is limited to architecturally significant changes, and other decisions belong to component owners (RFC-12) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Keep ADRs. They are your memory and your agents' memory. Replace RFCs with a short design note in `docs/rfc/` for large changes, with no comment period and you as sponsor. + +### AI agents + +- When a decision is significant, hard to reverse or crosses component boundaries, draft an ADR with status `proposed` from `docs/templates-reference/adr.md` instead of implementing silently (ADR-03). +- Do not edit accepted ADRs, supersede them (ADR-04). +- Read `docs/adr/` before changing behaviour it governs. +- Record at least two options (ADR-05). diff --git a/content/guidelines/defects-and-technical-debt.md b/content/guidelines/defects-and-technical-debt.md new file mode 100644 index 0000000..2ae1e92 --- /dev/null +++ b/content/guidelines/defects-and-technical-debt.md @@ -0,0 +1,88 @@ +--- +id: EG-018 +title: "Defect and technical debt management" +status: draft +owner: "Test and quality lead" +version: 0.1.0 +part: "Verification" +related: [EG-017, EG-019, EG-020, EG-021, EG-024] +review-by: 2027-09-30 +supersedes: +--- + +# EG-018: Defect and technical debt management + +- **Owner:** Test and quality lead +- **Applies to:** All defects and technical debt in product and tooling +- **Enforcement:** Tracker configuration, review, release criteria +- **Related:** [EG-017](testing-principles-and-strategy.md), [EG-019](logging-debug-and-diagnostics.md), [EG-020](observability-and-telemetry.md), [EG-021](release-update-and-production.md), [EG-024](planning-delivery-and-improvement.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Handle defects and technical debt consistently: visible, classified, analysed and closed with evidence, so that each one improves the process and none is hidden. + +## Rules + +1. **DEF-01** *(core)* All defects MUST be recorded in the tracker. Private lists MUST NOT be used. A record includes version found, environment, steps, expected and actual results, severity, component, linked requirement and logs or dumps. +1. **DEF-02** *(core)* Severity MUST follow the definitions below. Priority is set separately by the product owner. +1. **DEF-03** New defects MUST be triaged within two working days. Fix targets are set by severity. +1. **DEF-04** A defect lifecycle is `new`, `triaged`, `in progress`, `fixed`, `verified`, `closed`. Verification MUST be done by someone other than the fixer, and closure requires a regression test (TST-07). +1. **DEF-05** Root cause analysis MUST be done for severity 1 and 2 defects and for escapes. The cause is categorised as requirement, design, code, test gap, tooling or process. +1. **DEF-06** For each escape, the team MUST ask which gate should have caught it, and record the improvement to that gate. +1. **DEF-07** Commits and PRs MUST reference the defect (`Refs:`). Release notes MUST list fixed defects and known issues. +1. **DEF-08** Field defects MUST have a defined intake route, reproduction using diagnostics ([EG-019](logging-debug-and-diagnostics.md)), and notification to the release manager. Security defects follow [EG-022](security-and-data-protection.md). +1. **DEF-09** Technical debt MUST be recorded in the tracker with a `tech-debt` label, impact, cost estimate and owner, and be visible in a debt register. +1. **DEF-10** Capacity MUST be reserved for debt reduction each iteration (proposed 10 to 20 percent). Debt older than six months MUST be reviewed. +1. **DEF-11** Debt taken deliberately (for example to meet a date) MUST be recorded with a repayment plan and expiry. +1. **DEF-12** Static analysis suppressions and guideline deviations count as debt and MUST be tracked (COD-05, GOV-09). +1. **DEF-13** Defect and debt metrics (trends, escape rate, age, reopen rate) MUST be reviewed for improvement ([EG-024](planning-delivery-and-improvement.md)) and MUST NOT be used to rank individuals. + +## Severity definitions + +| Severity | Definition | Examples | +|----------|------------|----------| +| 1 Critical | Safety hazard, security compromise, data loss, or fleet-wide failure without recovery | Bricking update failure, key exposure | +| 2 Major | Requirement not met with no workaround, or widespread degradation | Missed real-time deadline in normal use | +| 3 Minor | Requirement not met with a workaround, or limited impact | Incorrect diagnostic counter | +| 4 Trivial | Cosmetic or documentation defect | Typo in a log message | + +## Compliance + +| Rule | Checked by | +|------|------------| +| DEF-01, DEF-02, DEF-09 | Tracker required fields and workflow | +| DEF-04 | Workflow enforcement and CI regression test check | +| DEF-05, DEF-06 | Postmortem records ([EG-024](planning-delivery-and-improvement.md)) | +| DEF-10 | Iteration planning review | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] All defects are in the tracker with the required fields (DEF-01) +- [ ] Severity uses the agreed definitions, with priority set separately (DEF-02) +- [ ] New defects are triaged within two working days (DEF-03) +- [ ] The lifecycle is followed, verification is by someone else and closure needs a regression test (DEF-04) +- [ ] Root cause analysis is done for severity 1 and 2 defects and escapes (DEF-05) +- [ ] Each escape asks which gate should have caught it (DEF-06) +- [ ] Commits reference defects and release notes list fixes and known issues (DEF-07) +- [ ] A field defect route exists, with reproduction and release manager notification (DEF-08) +- [ ] Technical debt is recorded with label, impact, estimate and owner (DEF-09) +- [ ] Capacity is reserved for debt and old debt is reviewed (DEF-10) +- [ ] Deliberate debt has a repayment plan and expiry (DEF-11) +- [ ] Suppressions and deviations are tracked as debt (DEF-12) +- [ ] Metrics are used for improvement and not for ranking individuals (DEF-13) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). An issue list with `bug` and `tech-debt` labels is enough. Reproduce, add a regression test, and note the root cause for anything serious. Keep a short debt list and clear it regularly. + +### AI agents + +- When fixing a defect, reproduce it with a test first, fix it, reference it in the commit (`Refs:`) and record the root cause category in the PR (DEF-05). +- Log unrelated problems you notice as tech-debt items instead of fixing them in passing (CP-16, DEF-09). +- Never close a defect without a regression test (DEF-04). diff --git a/content/guidelines/documentation-as-code.md b/content/guidelines/documentation-as-code.md new file mode 100644 index 0000000..014056c --- /dev/null +++ b/content/guidelines/documentation-as-code.md @@ -0,0 +1,115 @@ +--- +id: EG-004 +title: "Documentation as code and terminology" +status: draft +owner: "Build and tooling lead" +version: 0.1.0 +part: "Foundations" +related: [EG-001, EG-005, EG-008] +review-by: 2027-09-30 +supersedes: +--- + +# EG-004: Documentation as code and terminology + +- **Owner:** Build and tooling lead +- **Applies to:** All documentation describing the product, its design or the way it is built +- **Enforcement:** CI (lint, link check, build), review +- **Related:** [EG-001](guidelines-governance.md), [EG-005](ai-agent-usage.md), [EG-008](decision-records.md) +- **Principles:** WP-01 to WP-10 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Treat documentation with the same discipline as source code so that it stays accurate, reviewable and close to what it describes, and give the team one shared vocabulary. + +## Rules + +1. **DOC-01** *(core)* Documentation MUST be stored as plain text in the same repository as the thing it describes, or in a designated documentation repository referenced from it. +1. **DOC-02** Documents MUST be written in Markdown with a defined style guide enforced by a linter. +1. **DOC-03** *(core)* Changes to documentation MUST go through the same PR and review process as code. +1. **DOC-04** *(core)* A change that alters behaviour, an interface, a configuration option or a build step MUST update the affected documentation in the same PR. +1. **DOC-05** Diagrams MUST be authored as text (Mermaid or PlantUML) and stored as source. Exported images MUST NOT be the only copy. +1. **DOC-06** Every document MUST have front matter identifying its owner, status and last review date. +1. **DOC-07** The pipeline MUST build the documentation, fail on broken links and lint errors, and publish it as a versioned artefact with each release. +1. **DOC-08** *(core)* Each repository MUST provide the minimum document set: `README`, `CONTRIBUTING`, `AGENTS.md`, architecture overview, build and test instructions, and `docs/adr/`. +1. **DOC-09** Generated documentation (API reference, register maps, trace reports, decision log index) MUST be generated by the pipeline and MUST NOT be edited by hand. +1. **DOC-10** Documents SHOULD state the question they answer and their audience in the first paragraph, and SHOULD be split by purpose: tutorial, how-to, reference or explanation. +1. **DOC-11** Writing SHOULD be plain British English: short sentences, one idea per paragraph, active voice, conclusion first, acronyms defined at first use. +1. **DOC-12** The team MUST maintain a single glossary at `docs/reference/glossary.md`. A term is defined once and used consistently in requirements, code identifiers and documents. New terms are added by PR. +1. **DOC-13** File names, requirement IDs, component names and identifiers MUST follow the naming conventions in the style guide (kebab-case for files, stable IDs for artefacts). +1. **DOC-14** Approval of a document SHOULD be the PR approval. Separate signature records SHOULD NOT be used unless a regulation requires them ([EG-023](safety-regulatory-and-certification.md)). +1. **DOC-15** Documents MUST be structured (headings, front matter, IDs) so that both people and agents can navigate them. Long documents SHOULD be split rather than nested deeply. + +## Layout + +```text +repo/ + README.md + CONTRIBUTING.md + AGENTS.md + docs/ + architecture/ + adr/ + rfc/ + requirements/ + research/ + guidelines/ + how-to/ + reference/ +``` + +## Change flow + +```mermaid +flowchart LR + A[Change to code or design] --> B[Update docs in same PR] + B --> C[Lint and link check] + C --> D[Review] + D --> E[Merge] + E --> F[Build and publish versioned docs] +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| DOC-02, DOC-06, DOC-13 | Markdown linter and front matter schema check in CI | +| DOC-05, DOC-07, DOC-09 | Documentation build job in CI | +| DOC-04 | PR template checklist item | +| DOC-08 | Repository template check | +| DOC-12 | Glossary term check (optional linter) and review | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Documents are plain-text Markdown in the repository and linted (DOC-01, DOC-02) +- [ ] Documentation changes go through PR like code (DOC-03) +- [ ] Changes to behaviour, interfaces, configuration and build steps update docs in the same PR (DOC-04) +- [ ] Diagrams are Mermaid or PlantUML source (DOC-05) +- [ ] Every document has front matter with owner, status and review date (DOC-06) +- [ ] CI builds the docs, fails on broken links and lint errors, and publishes versioned docs with each release (DOC-07) +- [ ] The minimum document set exists: README, CONTRIBUTING, `AGENTS.md`, architecture overview, build and test instructions, `docs/adr/` (DOC-08) +- [ ] Generated docs are generated and never hand-edited (DOC-09) +- [ ] Documents state their question and audience, and are split by purpose (DOC-10) +- [ ] Writing is plain British English with the conclusion first (DOC-11) +- [ ] A single glossary exists and terms are used consistently (DOC-12) +- [ ] Naming conventions are followed for files, IDs and components (DOC-13) +- [ ] Approval is PR approval unless a regulation requires a signature record (DOC-14) +- [ ] Documents are structured with headings, front matter and IDs so people and agents can navigate them (DOC-15) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Starter** profile (see [personal-projects.md](personal-projects.md)). Start with a README and `AGENTS.md`. Add `docs/adr/` and Mermaid diagrams at the Standard profile. Front matter and a glossary become worthwhile once several people or agents share the same terms. + +### AI agents + +- Update docs in the same change as the behaviour they describe (DOC-04). +- Never hand-edit generated docs (DOC-09). +- Draw diagrams as Mermaid text and avoid semicolons in labels because they break rendering. +- Write plain British English with the conclusion first (DOC-11). +- Check every statement in a doc against the code before committing (AGT-15). diff --git a/content/guidelines/guidelines-governance.md b/content/guidelines/guidelines-governance.md new file mode 100644 index 0000000..2b5a6fc --- /dev/null +++ b/content/guidelines/guidelines-governance.md @@ -0,0 +1,109 @@ +--- +id: EG-001 +title: "Guidelines governance" +status: draft +owner: "Architecture group" +version: 0.1.0 +part: "Foundations" +related: [EG-004, EG-008] +review-by: 2027-09-30 +supersedes: +--- + +# EG-001: Guidelines governance + +- **Owner:** Architecture group +- **Applies to:** All engineering teams, sites and agents +- **Enforcement:** CI (front matter and link checks), review +- **Related:** [EG-004](documentation-as-code.md), [EG-008](decision-records.md) +- **Principles:** GP-01 to GP-10 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Define how guidelines are written, approved, changed, retired and enforced, so that every other guideline is consistent. + +## Rules + +1. **GOV-01** *(core)* Every guideline MUST have a unique ID (`EG-nnn`), a named owner, a status (`draft`, `active`, `deprecated`), a version and a review-by date in its front matter. +1. **GOV-02** Every guideline MUST follow the standard template (below). +1. **GOV-03** Every rule MUST have a stable ID and use RFC 2119 keywords. A rule SHOULD carry a brief rationale where the reason is not obvious. +1. **GOV-04** Every guideline MUST state how compliance is checked: tooling, review, or manual audit. A rule with no check MUST be marked as advisory. +1. **GOV-05** A guideline SHOULD be short enough to read in one sitting. Longer material belongs in supporting how-to documents. +1. **GOV-06** *(core)* New guidelines and material changes MUST go through an RFC ([EG-008](decision-records.md)), unless the change is editorial. +1. **GOV-07** *(core)* Changes MUST be made by pull request, approved by the owner and by at least one practitioner subject to the rule. +1. **GOV-08** Guidelines MUST be reviewed at least every 12 months. CI reports overdue reviews. +1. **GOV-09** *(core)* A deviation MUST be recorded as an exception with owner approval, scope, reason and expiry date. A permanent deviation means the rule is wrong and SHOULD be changed. +1. **GOV-10** Deprecated guidelines MUST remain in the repository with a pointer to their replacement. +1. **GOV-11** A guideline SHOULD include one correct and one incorrect example where this aids understanding. +1. **GOV-12** Guideline versions follow semantic versioning: MAJOR when a rule makes previously compliant work non-compliant, MINOR for new rules, PATCH for wording. +1. **GOV-13** Guidelines MUST be stored in `docs/guidelines/`, one file per guideline, named `eg-nnn-short-title.md`. The index MUST be generated from front matter. +1. **GOV-14** Each guideline MUST be assigned to a named owner who is accountable for keeping it useful, and who is the first contact for questions and exception requests. +1. **GOV-15** *(core)* Rules MUST be classified as core or tailorable. Core rules apply to every product without tailoring and are marked *(core)*. All other rules, and all numeric values, are defaults that a product MAY tailor through its conformance profile (GOV-16). The core set is kept small so that products stay consistent where it matters and free elsewhere. +1. **GOV-16** *(core)* Each product MUST have a conformance profile recording which guidelines apply, any tailored rules and values with the reason, approved exceptions, the components it uses and provides (COM-01), and its toolchain. The product's technical lead owns the profile and approves tailoring of non-core rules, and the architecture group is informed at its forum (RFC-11). Exceptions to core rules need the guideline owner's approval (GOV-09). + +## Standard template + +1. Header block: owner, applies to, enforcement, related guidelines +1. Purpose (why this exists, in two or three sentences) +1. Rules (with IDs, keywords and rationale where needed) +1. Supporting material (tables, templates, diagrams, examples) +1. Compliance (how each rule is checked, and by what) + +## Front matter + +```yaml +id: EG-000 +title: Short title +status: draft +owner: Role +version: 0.1.0 +part: Foundations +related: [EG-000] +review-by: 2027-09-30 +supersedes: +``` + +The header block at the top of each guideline body carries the applies-to, enforcement and related guidelines for human readers. + +## Compliance + +| Rule | Checked by | +|------|------------| +| GOV-01, GOV-08, GOV-13 | CI: front matter schema validation and index generation | +| GOV-02, GOV-04 | CI: required headings check | +| GOV-06, GOV-07 | Branch protection and CODEOWNERS on `docs/guidelines/` | +| GOV-09 | Exception register review at each guideline review | +| GOV-15, GOV-16 | CI: core rule list generated from the guidelines, profile schema validation and comparison against the core list | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Guidelines live in `docs/guidelines/`, one file per guideline, named `eg-nnn-short-title.md` (GOV-13) +- [ ] Every guideline has front matter with id, title, status, owner, version and review-by (GOV-01) +- [ ] Every guideline follows the standard template and states how compliance is checked (GOV-02, GOV-04) +- [ ] Rules have stable IDs and RFC 2119 keywords, with a rationale where it is not obvious (GOV-03) +- [ ] Each guideline is short enough to read in one sitting, with examples where they help (GOV-05, GOV-11) +- [ ] Changes are made by PR with owner and practitioner approval, and by RFC when material (GOV-06, GOV-07) +- [ ] Review-by dates are checked by CI and no guideline is overdue (GOV-08) +- [ ] Exceptions are recorded with owner, scope, reason and expiry (GOV-09) +- [ ] Deprecated guidelines are kept with a pointer to their replacement (GOV-10) +- [ ] Guideline versions follow semantic versioning (GOV-12) +- [ ] The index is generated from front matter (GOV-13) +- [ ] Each guideline has an accountable owner who handles questions and exception requests (GOV-14) +- [ ] Core rules are marked and applied identically across products, and all other rules are tailored only through the product profile (GOV-15) +- [ ] Each product has a conformance profile recording applicability, thresholds, exceptions, components and toolchain (GOV-16) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Keep the files and IDs so rules can be cited. Skip owners, review-by cycles and RFCs, because you are the owner. Record which guidelines you adopted in ADR-0001. + +### AI agents + +- Do not edit anything in `docs/guidelines/` unless the task explicitly says so (AGT-05). +- Cite rules by ID. +- If a rule blocks the task, stop and ask rather than deviating (GOV-09). diff --git a/content/guidelines/index.md b/content/guidelines/index.md index e0d5dd2..e46e360 100644 --- a/content/guidelines/index.md +++ b/content/guidelines/index.md @@ -5,6 +5,97 @@ author: Mark Bacon # Guidelines -Engineering guidelines: how to plan, design, build, review, release and run software. +## About these guidelines -This section is moving in from EngineeringProcess (eg- series) and the generic GU- guidelines in technical-documentation. +These are 26 engineering guidelines for an embedded R&D function, one guideline per file. They work for one product or many, and do not assume that products share code. They describe the practices a mature team follows and, for each one, what the guideline must contain and how compliance is checked. They are not coding rules: those belong in the coding standard that [EG-013](coding-standard.md) requires. + +Each guideline has rules with stable IDs, a compliance table, a checklist, and notes for applying it to personal projects and to AI agents. Cite rules by ID (for example `VCS-03`) in reviews, commits and agent instructions. + +> **Note:** All guidelines are `draft` at version 0.1.0. Numeric thresholds (review turnaround, patch times, coverage targets, size limits) are proposed starting points. Set `status: active` when you adopt a guideline. + +## Using these guidelines + +- **In an organisation with several products:** read [organisation-adoption.md](organisation-adoption.md). Core rules apply everywhere, and each product records its tailoring in a conformance profile. +- **In a single team:** copy the guidelines and the [templates](../templates/index.md) you need into the repository's `docs/` folder, confirm the owner roles, and adopt in the order given below. +- **In a personal project:** read [personal-projects.md](personal-projects.md), pick a profile (Starter, Standard or Full) and follow the bootstrap steps. +- **With AI agents:** put an [`AGENTS.md`](../templates/agents-md.md) at the repository root and keep [agent-rules.md](agent-rules.md) in the repository. Each guideline also has an AI agents section. +- **To assess adoption:** use the checklist at the end of each guideline, or the combined [checklists.md](checklists.md). + +## How to read the rules + +The keywords MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are used as defined in RFC 2119. + +- **MUST**: mandatory. Deviation needs a recorded, approved exception. +- **SHOULD**: expected. Deviation needs a stated reason in the PR or ADR. +- **MAY**: optional and left to judgement. + +Every rule has a stable ID (for example `DOC-03`). IDs are never reused. Reviews, commits, tooling output and AI agents cite rules by ID. + +## Guiding principles for consistency + +1. **GP-01 One way to do each thing.** Where two approaches exist, the guideline picks one and records why. +1. **GP-02 If a tool can check it, a tool checks it.** Reviewers spend attention on design, not formatting. +1. **GP-03 Conventions drive outcomes.** A convention that does not feed an automated result (version, changelog, trace report, release) is etiquette and will decay. Prefer conventions that machines can read. +1. **GP-04 Everything is text in version control.** Requirements, decisions, diagrams, pipelines, repository settings and guidelines are reviewed, versioned and diffable. +1. **GP-05 Record decisions once, at the right level.** Context in an RFC, outcome in an ADR, rules in a guideline, enforcement in tooling. +1. **GP-06 Local equals CI.** Any check that runs in the pipeline runs identically on a developer machine with one command. +1. **GP-07 Guidelines are short and owned.** A few pages each, one named owner, a review date, and a stated way of checking compliance. +1. **GP-08 Understand the tool.** Using a tool without knowing how it works (git, the linker, the RTOS, the debug probe) is a risk. Each guideline names the competence expected. +1. **GP-09 Explain the why.** Rules carry a rationale where it is not obvious, so that people and agents can apply them sensibly at the edges. +1. **GP-10 Sequence adoption.** Start with what other practices depend on: commit conventions, a single version source and a reproducible pipeline. + +## Designing for AI agents + +AI coding agents and assistants are treated as team members with no tribal knowledge, no memory between sessions, and perfect literalism. Every guideline is therefore written with explicit, numbered, testable rules and stable IDs, so that an agent can follow and cite them. Enforcement never depends on the agent complying: hooks, gates, branch protection and CODEOWNERS apply equally to agent-authored changes. The specific rules are in [EG-005](ai-agent-usage.md). The coding principles CP-02 (simplicity), CP-16 (surgical changes) and CP-20 (surface assumptions) in [EG-012](coding-principles.md) are the most important for agents. + +## Guideline index + +| ID | Guideline | Part | Owner | Core rules | Personal projects start at | +|----|-----------|------|-------|------------|----------------------------| +| [EG-001](guidelines-governance.md) | Guidelines governance | Foundations | Architecture group | 6 of 16 | Standard | +| [EG-002](values-and-ways-of-working.md) | Engineering values and ways of working | Foundations | Engineering manager | 0 of 12 | Full | +| [EG-003](quality-policy-and-attributes.md) | Quality policy and quality attributes | Foundations | Test and quality lead | 0 of 8 | Standard | +| [EG-004](documentation-as-code.md) | Documentation as code and terminology | Foundations | Build and tooling lead | 4 of 15 | Starter | +| [EG-005](ai-agent-usage.md) | AI agent usage | Foundations | Build and tooling lead and Security lead | 7 of 15 | Starter | +| [EG-006](requirements-and-traceability.md) | Requirements and traceability | Requirements, architecture and design | Systems engineering lead | 2 of 13 | Standard | +| [EG-007](architecture-principles.md) | Architecture principles | Requirements, architecture and design | Architecture group | 0 of 6 | Standard | +| [EG-008](decision-records.md) | Decision records: RFCs and ADRs | Requirements, architecture and design | Architecture group | 4 of 21 | Standard | +| [EG-009](interfaces-and-co-design.md) | Interfaces and hardware and software co-design | Requirements, architecture and design | Architecture group | 0 of 11 | Standard | +| [EG-010](components-reuse-and-variants.md) | Components, reuse and variants | Requirements, architecture and design | Architecture group | 3 of 13 | Full | +| [EG-011](technology-and-third-party.md) | Technology selection and third-party components | Requirements, architecture and design | Architecture group | 3 of 11 | Standard | +| [EG-012](coding-principles.md) | Coding principles | Code and build | Firmware lead | 1 of 4 | Starter | +| [EG-013](coding-standard.md) | Coding standard | Code and build | Firmware lead | 3 of 12 | Standard | +| [EG-014](version-control-and-versioning.md) | Version control, commits and versioning | Code and build | Build and tooling lead | 9 of 15 | Starter | +| [EG-015](pull-requests-and-review.md) | Pull requests and code review | Code and build | Firmware lead | 4 of 15 | Standard | +| [EG-016](pipelines-and-quality-gates.md) | Pipelines and quality gates | Code and build | Build and tooling lead | 5 of 16 | Standard | +| [EG-017](testing-principles-and-strategy.md) | Testing principles and strategy | Verification | Test and quality lead | 2 of 15 | Standard | +| [EG-018](defects-and-technical-debt.md) | Defect and technical debt management | Verification | Test and quality lead | 2 of 13 | Standard | +| [EG-019](logging-debug-and-diagnostics.md) | Logging, debug and fault diagnostics | Diagnostics and field insight | Firmware lead | 0 of 21 | Standard | +| [EG-020](observability-and-telemetry.md) | Observability and telemetry | Diagnostics and field insight | Platform lead | 0 of 13 | Full | +| [EG-021](release-update-and-production.md) | Release, update and production | Release, risk and compliance | Release manager | 2 of 17 | Standard | +| [EG-022](security-and-data-protection.md) | Security and data protection | Release, risk and compliance | Security lead | 4 of 14 | Starter | +| [EG-023](safety-regulatory-and-certification.md) | Safety, regulatory and certification | Release, risk and compliance | Regulatory lead | 0 of 13 | Full | +| [EG-024](planning-delivery-and-improvement.md) | Planning, delivery and improvement | Delivery, research and people | Engineering manager | 0 of 14 | Full | +| [EG-025](research-and-prototyping.md) | Research, prototyping and experiments | Delivery, research and people | Research lead | 0 of 12 | Standard | +| [EG-026](competence-tooling-and-onboarding.md) | Competence, tooling and onboarding | Delivery, research and people | Engineering manager | 0 of 13 | Full | + +## Supporting files + +- [Principles catalogue](principles-catalogue.md): the principle sets (guiding, values, requirements, architecture, coding, testing, security, diagnostics, lifecycle, documentation, AI agents) and a start-here list +- [core-rules.md](core-rules.md): the rules that apply to every product without tailoring +- [organisation-adoption.md](organisation-adoption.md): two-layer model, conformance profile, owner roles, governance cadence and rollout +- [checklists.md](checklists.md): all checklists in one file +- [personal-projects.md](personal-projects.md): profiles, what to scale down, what never scales down, bootstrap steps +- [agent-rules.md](agent-rules.md): condensed rules for AI agents +- [Templates](../templates/index.md): ADR, RFC, requirement, interface specification, NFR scenario, experiment record, postmortem, pull request, agent task brief, product conformance profile and component register templates +- [AGENTS.md](../templates/agents-md.md): agent instructions to place at the repository root + +## Suggested adoption sequence + +1. **Mechanics first:** [EG-001](guidelines-governance.md) governance, [EG-014](version-control-and-versioning.md) commits and versioning, [EG-016](pipelines-and-quality-gates.md) reproducible pipeline +1. **Shared thinking:** [EG-002](values-and-ways-of-working.md) values, [EG-003](quality-policy-and-attributes.md) quality attributes, [EG-007](architecture-principles.md) architecture principles, [EG-012](coding-principles.md) coding principles +1. **Recording and communicating:** [EG-004](documentation-as-code.md) docs as code, [EG-008](decision-records.md) RFCs and ADRs, [EG-005](ai-agent-usage.md) AI agent usage, [EG-015](pull-requests-and-review.md) PR conventions +1. **Traceable engineering:** [EG-006](requirements-and-traceability.md) requirements, [EG-013](coding-standard.md) coding standard, [EG-009](interfaces-and-co-design.md) interfaces, [EG-010](components-reuse-and-variants.md) components and reuse, [EG-011](technology-and-third-party.md) technology and third-party +1. **Evidence and insight:** [EG-017](testing-principles-and-strategy.md) testing, [EG-018](defects-and-technical-debt.md) defects and debt, [EG-019](logging-debug-and-diagnostics.md) logging and debug, [EG-020](observability-and-telemetry.md) observability +1. **Assurance and delivery:** [EG-021](release-update-and-production.md) release, [EG-022](security-and-data-protection.md) security, [EG-023](safety-regulatory-and-certification.md) regulatory, [EG-024](planning-delivery-and-improvement.md) planning +1. **Growth:** [EG-025](research-and-prototyping.md) research and prototyping, [EG-026](competence-tooling-and-onboarding.md) competence and onboarding diff --git a/content/guidelines/interfaces-and-co-design.md b/content/guidelines/interfaces-and-co-design.md new file mode 100644 index 0000000..7a26777 --- /dev/null +++ b/content/guidelines/interfaces-and-co-design.md @@ -0,0 +1,92 @@ +--- +id: EG-009 +title: "Interfaces and hardware and software co-design" +status: draft +owner: "Architecture group" +version: 0.1.0 +part: "Requirements, architecture and design" +related: [EG-007, EG-011, EG-017, EG-021, EG-010] +review-by: 2027-09-30 +supersedes: +--- + +# EG-009: Interfaces and hardware and software co-design + +- **Owner:** Architecture group +- **Applies to:** All software, protocol, hardware and cross-site interfaces +- **Enforcement:** CI (contract tests, generation checks), review +- **Related:** [EG-007](architecture-principles.md), [EG-011](technology-and-third-party.md), [EG-017](testing-principles-and-strategy.md), [EG-021](release-update-and-production.md), [EG-010](components-reuse-and-variants.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Make every boundary an owned, versioned, verified contract, and keep hardware and firmware teams working from one shared, controlled description of the platform. + +## Rules + +1. **INT-01** Every interface (software API, inter-processor link, protocol, hardware and software boundary, external system) MUST have an owner and a specification in the repository. +1. **INT-02** An interface specification MUST state its purpose, roles, data formats, timing, error handling, versioning and compatibility rules, security properties and verification method. +1. **INT-03** Interfaces SHOULD be defined in machine-readable form (headers, IDL, schemas, register descriptions such as SVD) and used to generate code and documentation. Hand-maintained copies MUST NOT exist. +1. **INT-04** Interfaces MUST be versioned with semantic versioning. A compatibility matrix MUST be maintained, and a breaking change needs an RFC and a coordinated release. +1. **INT-05** Provider and consumer MUST both run interface contract tests in CI. +1. **INT-06** A hardware interface agreement (pin map, memory map, clocks, power, boot configuration, peripherals) MUST be kept as a versioned document. Firmware headers SHOULD be generated from it where possible. +1. **INT-07** Every hardware revision or component change MUST be reviewed for firmware impact, and the firmware team notified before release of the change. The hardware revision MUST be recorded in the bill of materials ([EG-021](release-update-and-production.md)). +1. **INT-08** A silicon errata register MUST be kept. Each erratum is reviewed when a silicon revision is adopted, and tracked with affected components, workaround and references to code or ADRs. +1. **INT-09** A bring-up plan MUST exist for new hardware: board-level test firmware, checklists, and emulation or simulation so that firmware work starts before hardware arrives. +1. **INT-10** Cross-site interfaces are the highest-risk contracts. Changes MUST be announced by change notice or RFC with agreed lead time. +1. **INT-11** Datasheets, reference manuals and vendor documents MUST be version-controlled or referenced by exact version, and cited by ID in the design. + +## Interface specification skeleton + +```markdown +# Interface: + +- Owner: +- Version: +- Providers and consumers: + +## Purpose +## Data and encoding +## Timing and ordering +## Errors and recovery +## Versioning and compatibility +## Security properties +## Verification +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| INT-03 | CI: generation check fails if generated files differ from committed sources | +| INT-04, INT-05 | CI: contract tests on both sides, compatibility matrix check | +| INT-07, INT-08 | Hardware change review, errata register audit | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Every interface has an owner and a specification in the repository (INT-01, INT-02) +- [ ] Machine-readable definitions generate code and docs, with no hand-maintained copies (INT-03) +- [ ] Interfaces are versioned, a compatibility matrix is kept, and breaking changes go through RFC (INT-04) +- [ ] Contract tests run on both sides in CI (INT-05) +- [ ] The hardware interface agreement is versioned and firmware headers are generated where possible (INT-06) +- [ ] Hardware changes are reviewed for firmware impact and recorded in the bill of materials (INT-07) +- [ ] An errata register is maintained (INT-08) +- [ ] A bring-up plan and emulation exist for new hardware (INT-09) +- [ ] Cross-site interface changes are announced with lead time (INT-10) +- [ ] Vendor documents are version-controlled or cited by exact version (INT-11) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Write down any boundary that another module, script, agent or device relies on (an API, a file format, a register map). Keep datasheet versions in `docs/reference/` or link them by version. + +### AI agents + +- Do not change a documented interface without an ADR or RFC and a version bump (INT-04). +- Regenerate generated interface files instead of editing them (INT-03). +- Add or update contract tests on both sides when touching an interface (INT-05). +- Cite datasheets and errata by ID and version (INT-11). diff --git a/content/guidelines/logging-debug-and-diagnostics.md b/content/guidelines/logging-debug-and-diagnostics.md new file mode 100644 index 0000000..718b4c7 --- /dev/null +++ b/content/guidelines/logging-debug-and-diagnostics.md @@ -0,0 +1,110 @@ +--- +id: EG-019 +title: "Logging, debug and fault diagnostics" +status: draft +owner: "Firmware lead" +version: 0.1.0 +part: "Diagnostics and field insight" +related: [EG-007, EG-012, EG-020, EG-022] +review-by: 2027-09-30 +supersedes: +--- + +# EG-019: Logging, debug and fault diagnostics + +- **Owner:** Firmware lead +- **Applies to:** All firmware, debug tooling and production builds +- **Enforcement:** Build configuration, CI, review +- **Related:** [EG-007](architecture-principles.md), [EG-012](coding-principles.md), [EG-020](observability-and-telemetry.md), [EG-022](security-and-data-protection.md) +- **Principles:** OP-01 to OP-10 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Make the device explain its own behaviour and failures, cheaply and safely, during development and in the field, with a single framework for logging, debugging, fault handling and crash capture. + +## Logging rules + +1. **LOG-01** Logging purposes MUST be kept separate, each with its own level and retention policy: development trace, field diagnostics and security or audit events. +1. **LOG-02** Log levels MUST follow the definitions below. The compiled or default level for production MUST be defined, and `DEBUG` and `TRACE` MUST be compiled out or disabled in production. +1. **LOG-03** All logging MUST use one common logging facility. Ad-hoc `printf` MUST NOT be used. Every record has a module ID and severity, with compile-time filtering per module. +1. **LOG-04** The record format MUST be compact, structured and machine-parseable. Where flash is constrained, records SHOULD be tokenised or binary with an off-target decoder. Each record has a timestamp (source and unit defined), module, level, event ID and parameters. +1. **LOG-05** A single, versioned event and fault code registry MUST exist with an owner. Each code has a description, severity, likely cause and recommended action. The registry is the source of truth from which code and documentation are generated. +1. **LOG-06** Logging cost MUST be budgeted (flash, RAM, CPU). Logging MUST be non-blocking and safe in the contexts it is used in, rate-limited and de-duplicated. Logging from time-critical interrupts MUST be deferred. +1. **LOG-07** Persistent logs MUST use a wear-aware policy, with documented write limits. RAM ring buffers hold recent history. +1. **LOG-08** Logs MUST NOT contain secrets, keys or personal data, and MUST follow the data classification ([EG-022](security-and-data-protection.md)). +1. **LOG-09** Messages MUST state what happened and give useful context. Logging inside tight loops MUST be avoided. +1. **LOG-10** Log decoding tools MUST be versioned with the firmware and retained (DBG-07). + +## Debug and fault rules + +1. **DBG-01** A standard debug setup per platform MUST be documented and kept in the repository: approved probes and adapters, debug server, IDE configuration and trace tools, starting with one command. +1. **DBG-02** Debug builds and production builds MUST be defined, with their differences documented. Production images MUST have debug interfaces disabled or protected ([EG-022](security-and-data-protection.md)). Debug builds MUST NOT be released. +1. **DBG-03** Every fault path (hard fault, assertion, watchdog, stack overflow, brownout, memory error) MUST have defined behaviour: capture state, log, enter a safe state and reset if appropriate. The policy is recorded in an ADR. +1. **DBG-04** Assertions MUST be used for invariants (CP-05). Their production behaviour MUST be defined (for example log and reset), and they MUST NOT have side effects. +1. **DBG-05** The reset reason and last-known state MUST be captured at every boot and reported. +1. **DBG-06** A crash dump MUST have a defined format (registers, stack excerpt, fault status registers, version and build ID, key state), stored in reserved memory with a defined retrieval procedure and size budget. +1. **DBG-07** A build ID MUST be embedded in every image. Map files, ELF files and symbols MUST be archived per release for the supported life so that dumps can be decoded. +1. **DBG-08** Runtime diagnostics SHOULD include health counters (stack high-water marks, heap use, queue depths, CPU load), self-tests and a diagnostic command interface with access control. +1. **DBG-09** Watchdog design MUST be reviewed as a system property: window, feeding rules and reset handling. Watchdogs MUST NOT be fed blindly from timer interrupts. +1. **DBG-10** A documented procedure MUST exist to turn a field dump or log into a local reproduction, and the reproduction MUST become a test (TP-08). +1. **DBG-11** Trace and timing tools for analysis MUST be approved, and the cost of instrumentation documented. + +## Log levels + +| Level | Meaning | Production | +|-------|---------|------------| +| ERROR | An operation failed and the system could not recover it locally | Enabled | +| WARN | Unexpected condition handled, but worth attention | Enabled | +| INFO | Significant normal event (boot, update, state change) | Enabled at defined volume | +| DEBUG | Detail for development | Compiled out or disabled | +| TRACE | Fine-grained flow for development | Compiled out | + +## Compliance + +| Rule | Checked by | +|------|------------| +| LOG-02, LOG-03 | CI: build configuration check and lint for banned `printf` use | +| LOG-05 | CI: registry schema and uniqueness check, generation check | +| LOG-06 | CI: size and timing budget report | +| DBG-02 | Release criteria and production image verification ([EG-021](release-update-and-production.md)) | +| DBG-07 | Release evidence pack completeness check | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Logging purposes are separated, each with its own levels and retention (LOG-01) +- [ ] Log levels are defined and DEBUG and TRACE are off in production (LOG-02) +- [ ] There is one logging facility, no ad-hoc printf, and every record has a module ID and severity (LOG-03) +- [ ] The format is compact, structured and machine-parseable, with an off-target decoder if needed (LOG-04) +- [ ] A versioned event and fault code registry has an owner (LOG-05) +- [ ] Logging cost is budgeted, non-blocking and rate-limited (LOG-06) +- [ ] Persistent logs are wear-aware (LOG-07) +- [ ] Logs contain no secrets or personal data (LOG-08) +- [ ] Messages say what happened with context, and there is no logging in tight loops (LOG-09) +- [ ] Log decoding tools are versioned with the firmware (LOG-10) +- [ ] A standard debug setup is documented per platform and starts with one command (DBG-01) +- [ ] Debug and production builds are defined, debug builds are never released, and production interfaces are locked (DBG-02) +- [ ] Every fault path has defined behaviour, recorded in an ADR (DBG-03) +- [ ] Assertions guard invariants, have defined production behaviour and no side effects (DBG-04) +- [ ] The reset reason is captured at every boot (DBG-05) +- [ ] The crash dump format is defined, a build ID is embedded and symbols are archived per release (DBG-06, DBG-07) +- [ ] Runtime diagnostics exist and the watchdog design is reviewed (DBG-08, DBG-09) +- [ ] A procedure turns a field dump into a local reproduction, which becomes a test (DBG-10) +- [ ] Trace and timing tools are approved and the cost of instrumentation is documented (DBG-11) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Use one logging function with levels and module tags from the start, and no stray prints. Embed a build ID. For hardware projects, keep a one-page debug setup note. Skip binary log tokenisation unless flash is tight. + +### AI agents + +- Use the project's logging facility with the correct level and module, and never `printf` or ad-hoc prints (LOG-03). +- Never log secrets, keys or personal data (LOG-08). +- Handle every fault path explicitly (DBG-03). +- Do not put side effects in assertions (DBG-04). +- Add event and fault codes to the registry and never invent them inline (LOG-05). diff --git a/content/guidelines/observability-and-telemetry.md b/content/guidelines/observability-and-telemetry.md new file mode 100644 index 0000000..5350fa8 --- /dev/null +++ b/content/guidelines/observability-and-telemetry.md @@ -0,0 +1,88 @@ +--- +id: EG-020 +title: "Observability and telemetry" +status: draft +owner: "Platform lead" +version: 0.1.0 +part: "Diagnostics and field insight" +related: [EG-003, EG-018, EG-019, EG-021, EG-022] +review-by: 2027-09-30 +supersedes: +--- + +# EG-020: Observability and telemetry + +- **Owner:** Platform lead +- **Applies to:** Devices, backend platform and the tooling that receives and displays their data +- **Enforcement:** CI (schema checks), review, release criteria +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-018](defects-and-technical-debt.md), [EG-019](logging-debug-and-diagnostics.md), [EG-021](release-update-and-production.md), [EG-022](security-and-data-protection.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Define what we must be able to know about devices in the field and in the lab. Say how that data is collected, structured and protected, and how it feeds back into engineering decisions. + +## Rules + +1. **OBS-01** Each product MUST define its observability requirements: the questions the team must be able to answer about a device and the fleet (is it healthy, which version, what failed, did the update succeed, what are the resource margins). These are captured as requirements ([EG-006](requirements-and-traceability.md)). +1. **OBS-02** The telemetry schema MUST be documented and versioned with an owner. Metric names, units and types (counter, gauge, event) MUST follow naming conventions, and schema changes follow semantic versioning with backward compatibility. +1. **OBS-03** Every record MUST carry context: pseudonymous device ID, firmware version, hardware revision, configuration version, boot count and time source. +1. **OBS-04** Bandwidth, battery, storage and cost budgets per device MUST be defined, with sampling and aggregation policy to stay within them. +1. **OBS-05** Telemetry MUST follow data minimisation and the data classification ([EG-022](security-and-data-protection.md)). Personal data MUST NOT be collected without approval. Retention periods MUST be defined. +1. **OBS-06** Transport MUST handle offline buffering, back-pressure, ordering, de-duplication and loss detection. +1. **OBS-07** Backend ingestion, storage, dashboards and alerts MUST be defined as code and versioned. +1. **OBS-08** Fleet health indicators and objectives MUST be defined (for example update success rate, crash-free rate, connectivity). Every alert has an owner, a runbook and a required action. Alerts without actions MUST NOT exist. +1. **OBS-09** Update rollouts MUST use telemetry health gates with automatic halt criteria ([EG-021](release-update-and-production.md)). +1. **OBS-10** Field insights MUST be reviewed at least monthly and fed into defect triage, requirements and test gaps ([EG-018](defects-and-technical-debt.md), [EG-024](planning-delivery-and-improvement.md)). +1. **OBS-11** Lab and HIL environments SHOULD emit the same telemetry, so that parsers and dashboards are exercised before release. +1. **OBS-12** Telemetry data quality MUST be checked: schema validation, gap and anomaly detection, and time source checks. +1. **OBS-13** Operational runbooks MUST exist for the top alerts and be kept with the dashboards and alert definitions. + +## Signals + +| Signal | Answers | Examples | +|--------|---------|----------| +| Logs | What happened, in what order | Event records, fault codes | +| Metrics | How much, how often, how healthy | Counters, gauges, resource margins | +| Events | Significant state changes | Boot, update result, connectivity change | +| Dumps | Why did it fail | Crash dump with build ID | + +## Compliance + +| Rule | Checked by | +|------|------------| +| OBS-02, OBS-03 | CI: schema validation and compatibility check | +| OBS-05 | Privacy review, telemetry field register | +| OBS-07, OBS-08 | Code review of dashboards and alerts, alert audit | +| OBS-10 | Monthly review record | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Observability requirements are defined per product (OBS-01) +- [ ] The telemetry schema is versioned, owned and follows naming conventions (OBS-02) +- [ ] Every record carries device, firmware, hardware, configuration and time context (OBS-03) +- [ ] Bandwidth, battery, storage and cost budgets exist with a sampling policy (OBS-04) +- [ ] Data minimisation, classification and retention are applied (OBS-05) +- [ ] Transport handles offline buffering, ordering, de-duplication and loss detection (OBS-06) +- [ ] Backend, dashboards and alerts are defined as code (OBS-07) +- [ ] Health indicators and objectives are defined, and every alert has an owner, runbook and action (OBS-08, OBS-13) +- [ ] Rollouts are gated by telemetry with automatic halt criteria (OBS-09) +- [ ] Field insights are reviewed monthly and fed into triage and tests (OBS-10) +- [ ] Lab and HIL environments emit the same telemetry (OBS-11) +- [ ] Data quality checks are in place (OBS-12) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Full** profile (see [personal-projects.md](personal-projects.md)). Needed only if the project runs somewhere you cannot see. Define three or four health questions, emit versioned metrics and set a few alerts that each have an action. Skip it for local tools. + +### AI agents + +- Do not add telemetry fields without updating the versioned schema and classification (OBS-02, OBS-05). +- Give every metric a name, unit and type per the convention (OBS-02). +- Never add personal data to telemetry (OBS-05). +- Do not add alerts without an owner, runbook and action (OBS-08). diff --git a/content/guidelines/organisation-adoption.md b/content/guidelines/organisation-adoption.md new file mode 100644 index 0000000..e1f8424 --- /dev/null +++ b/content/guidelines/organisation-adoption.md @@ -0,0 +1,93 @@ +--- +title: "Adopting the guidelines across several products" +status: draft +owner: "Architecture group" +last-reviewed: 2026-09-29 +--- + +# Adopting the guidelines across several products + +## Purpose + +This page explains how an R&D function with several products and a small architecture group can adopt the guidelines. It does not assume that products share code. Where they do, the components guideline ([EG-010](components-reuse-and-variants.md)) adds the rules for it. Where they do not, the same rules apply within each product. + +## Two layers + +| Layer | What it contains | Who decides | How it changes | +|-------|------------------|-------------|----------------| +| Core | The 61 rules marked *(core)*, listed in [core-rules.md](core-rules.md) | Guideline owners, with the architecture forum | RFC, consulting every product's technical lead | +| Product | Everything else: applicability, thresholds, toolchain choices, exceptions | The product's technical lead | Change to the conformance profile, reported at the architecture forum | + +The core layer keeps the products consistent where it matters most: anyone can move between products and find the same commit conventions, traceability, release evidence and AI agent controls. The product layer lets each product fit the rules to its hardware, certification needs and team size. + +## Product conformance profile + +Each product keeps a conformance profile in its repository (GOV-16), using [the template](../templates/product-conformance-profile.md). It records: + +- which guidelines apply, and why any do not +- tailored rules and values, with the reason and who approved them +- approved exceptions with expiry dates +- the toolchain and the components the product uses and provides +- the top quality attributes and any change to the precedence order + +The profile is checked in CI against the core list, so a profile cannot switch off a core rule. + +## Sharing code is optional + +Nothing here requires products to share code. The components guideline treats sharing as a spectrum, and its "How the rules scale" table says which rules apply when a component has one consumer, several, or is third party. A product that shares nothing keeps a register of its own components and complies with the same core rules. A product that starts to share a component then adopts the extra rules for that component, and not for the whole product. + +## Owner roles + +Owning a guideline means keeping it useful, answering questions and deciding on exceptions (GOV-14). Ownership is deliberately spread so that the architecture group is not a bottleneck. The architecture group owns the guidelines that set the direction of the design ([EG-001](guidelines-governance.md), [EG-007](architecture-principles.md), [EG-008](decision-records.md), [EG-009](interfaces-and-co-design.md), [EG-010](components-reuse-and-variants.md), [EG-011](technology-and-third-party.md)). The rest sit with part-time leads drawn from the developers. + +| Role | Guidelines owned | Typically drawn from | Time | +|------|------------------|----------------------|------| +| Architecture group | [EG-001](guidelines-governance.md), [EG-007](architecture-principles.md), [EG-008](decision-records.md), [EG-009](interfaces-and-co-design.md), [EG-010](components-reuse-and-variants.md), [EG-011](technology-and-third-party.md) | The architects, with a lead architect named per guideline | Core part of their work | +| Engineering manager | [EG-002](values-and-ways-of-working.md), [EG-024](planning-delivery-and-improvement.md), [EG-026](competence-tooling-and-onboarding.md) | Line management | About 10 percent | +| Build and tooling lead | [EG-004](documentation-as-code.md), [EG-005](ai-agent-usage.md), [EG-014](version-control-and-versioning.md), [EG-016](pipelines-and-quality-gates.md) | Senior developer | 20 to 30 percent | +| Firmware lead | [EG-012](coding-principles.md), [EG-013](coding-standard.md), [EG-015](pull-requests-and-review.md), [EG-019](logging-debug-and-diagnostics.md) | Senior developer | 10 to 20 percent | +| Test and quality lead | [EG-003](quality-policy-and-attributes.md), [EG-017](testing-principles-and-strategy.md), [EG-018](defects-and-technical-debt.md) | Senior developer or test engineer | About 20 percent | +| Systems engineering lead | [EG-006](requirements-and-traceability.md) | An architect or a senior systems engineer | 10 to 20 percent | +| Platform lead | [EG-020](observability-and-telemetry.md) | Senior developer (platform or backend) | 5 to 10 percent | +| Release manager | [EG-021](release-update-and-production.md) | Senior developer or programme role | About 10 percent, more near releases | +| Security lead | [EG-005](ai-agent-usage.md), [EG-022](security-and-data-protection.md) | Senior developer with a security focus | About 10 percent | +| Regulatory lead | [EG-023](safety-regulatory-and-certification.md) | May sit outside R&D | As the products require | +| Research lead | [EG-025](research-and-prototyping.md) | Senior developer or architect | 5 to 10 percent | + +> **Note:** Roles are proposals to be confirmed. One person may hold several roles, and a role may be shared. Each owner has a named deputy. + +## Governance cadence + +| Activity | Proposed cadence | Owner | +|----------|------------------|-------| +| Architecture forum: decide or defer open RFCs (RFC-11) | Fortnightly | Architecture group | +| Owners' check-in: exceptions, tailoring reports and open questions | Monthly | Architecture group | +| Exception register review (GOV-09) | Quarterly | Guideline owners | +| Guideline review (GOV-08) | Each guideline yearly, spread across the year | Guideline owners | +| Adoption assessment using the checklists | Twice a year per product | Product technical lead | + +Keep architecture review for architecturally significant changes (RFC-12), and do not make it a routine step in every pull request (PRR-15). With three architects this is what keeps the workload manageable. + +## Rollout + +Adopt in the order of the sequence in the [guideline index](index.md). These phases group the steps. Durations are proposals and depend on the number of products and the starting point. + +| Phase | Guidelines | Outcome | Proposed duration | +|-------|------------|---------|-------------------| +| 0. Decide | [EG-001](guidelines-governance.md), [EG-002](values-and-ways-of-working.md) | Owners named, core set agreed, conformance profile template agreed | 1 month | +| 1. Mechanics | [EG-014](version-control-and-versioning.md), [EG-016](pipelines-and-quality-gates.md), [EG-015](pull-requests-and-review.md) | Commit and version conventions, reproducible pipeline and PR template in every product | 2 to 3 months | +| 2. Shared thinking | [EG-003](quality-policy-and-attributes.md), [EG-007](architecture-principles.md), [EG-012](coding-principles.md) | Quality attributes, architecture and coding principles agreed and cited in reviews | 2 months | +| 3. Recording | [EG-004](documentation-as-code.md), [EG-008](decision-records.md), [EG-005](ai-agent-usage.md) | Docs as code, ADRs and RFCs, AI agent controls in place | 2 to 3 months | +| 4. Traceable engineering | [EG-006](requirements-and-traceability.md), [EG-013](coding-standard.md), [EG-009](interfaces-and-co-design.md), [EG-010](components-reuse-and-variants.md), [EG-011](technology-and-third-party.md) | Requirements traced, coding standard, interfaces and components registered | 3 to 6 months | +| 5. Evidence and insight | [EG-017](testing-principles-and-strategy.md) to [EG-020](observability-and-telemetry.md) | Test strategy, defect management, diagnostics and observability | 3 to 6 months | +| 6. Assurance and delivery | [EG-021](release-update-and-production.md) to [EG-024](planning-delivery-and-improvement.md) | Release and update safety, security, regulatory evidence, planning | Ongoing | +| 7. Growth | [EG-025](research-and-prototyping.md), [EG-026](competence-tooling-and-onboarding.md) | Research routes and onboarding | Ongoing | + +Start each product with its core rules and the guidelines its risks demand, not with all of them. Use the [checklists](checklists.md) to see where a product is, and tick items only when they are true. + +## Scaling notes + +- **When a product joins:** create its conformance profile, register its components and run the checklists to set a baseline. +- **When a component gains a consumer:** move it down the scaling table in [EG-010](components-reuse-and-variants.md) and agree the extra rules with the new consumer. +- **When the architecture group is overloaded:** narrow the scope of architecture review (RFC-12), delegate more decisions to component owners (VAL-02) and check that owners have time for the role. +- **When a core rule is disputed by several products:** treat it as a signal and open an RFC rather than granting exceptions one by one. diff --git a/content/guidelines/personal-projects.md b/content/guidelines/personal-projects.md new file mode 100644 index 0000000..7530ff7 --- /dev/null +++ b/content/guidelines/personal-projects.md @@ -0,0 +1,115 @@ +--- +title: "Using the guidelines on personal projects" +status: draft +owner: "Project owner" +last-reviewed: 2026-09-29 +--- + +# Using the guidelines on personal projects + +## Purpose + +The guidelines are written for a team, but the ideas scale down. This page says which guidelines to adopt for a personal project, what to simplify, and what never to drop, especially when AI agents do part of the work. + +Working alone does not mean working without controls. When an agent writes code, you are also the reviewer, the test lead and the release manager, and the guidelines are the checks that keep those roles honest. + +## Profiles + +| Profile | Use it for | Adopt | +|---------|------------|-------| +| Starter | Scripts, experiments and weekend projects | The Starter guidelines in the table below | +| Standard | Any project you intend to maintain, or where agents work regularly | Starter plus the Standard guidelines | +| Full | Multi-person, deployed or regulated work | Everything, taking items as they apply | + +## What to adopt and how to simplify + +| ID | Guideline | Starts at | What to keep or simplify | +|----|-----------|-----------|--------------------------| +| [EG-001](guidelines-governance.md) | Guidelines governance | Standard | Keep the files and IDs so rules can be cited. Skip owners, review-by cycles and RFCs, because you are the owner. Record which guidelines you adopted in ADR-0001. | +| [EG-002](values-and-ways-of-working.md) | Engineering values and ways of working | Full | Keep written-by-default, small steps, simplicity and evidence over opinion. Decision rights collapse to you. Skip the cross-site rules. | +| [EG-003](quality-policy-and-attributes.md) | Quality policy and quality attributes | Standard | Pick your top three quality attributes (for example correctness, simplicity, resource use) and write them in the README. Use the NFR scenario format only for requirements that matter. | +| [EG-004](documentation-as-code.md) | Documentation as code and terminology | Starter | Start with a README and `AGENTS.md`. Add `docs/adr/` and Mermaid diagrams at the Standard profile. Front matter and a glossary become worthwhile once several people or agents share the same terms. | +| [EG-005](ai-agent-usage.md) | AI agent usage | Starter | This is the guideline that matters most for solo work with agents. You are the human approver. Keep every rule. CODEOWNERS is just you. Use PRs (or at least branches with CI) so that you review diffs before they reach `main`. | +| [EG-006](requirements-and-traceability.md) | Requirements and traceability | Standard | Keep a short `docs/requirements/` list with IDs and a one-line verification each, and reference IDs with `Refs:` in commits. Skip formal levels and three-way review. For a small script, a 'what it must do' list in the README is enough. | +| [EG-007](architecture-principles.md) | Architecture principles | Standard | Adopt five or six principles that fit and write them in the README. Good defaults are AP-02, AP-03, AP-05, AP-07, AP-08 and AP-13. | +| [EG-008](decision-records.md) | Decision records: RFCs and ADRs | Standard | Keep ADRs. They are your memory and your agents' memory. Replace RFCs with a short design note in `docs/rfc/` for large changes, with no comment period and you as sponsor. | +| [EG-009](interfaces-and-co-design.md) | Interfaces and hardware and software co-design | Standard | Write down any boundary that another module, script, agent or device relies on (an API, a file format, a register map). Keep datasheet versions in `docs/reference/` or link them by version. | +| [EG-010](components-reuse-and-variants.md) | Components, reuse and variants | Full | Only needed if you reuse your own code across projects. Publish it as a versioned package or tagged repository and pin it, instead of copying it between projects. Keep a short list of the components you maintain and their versions. | +| [EG-011](technology-and-third-party.md) | Technology selection and third-party components | Standard | Keep pinning, checksums, a dependency list with licences, and automated update PRs. The radar can be a table in the README. | +| [EG-012](coding-principles.md) | Coding principles | Starter | Use as is. It is language-neutral and costs nothing. Paste CP-02, CP-16 and CP-20 into every agent brief. | +| [EG-013](coding-standard.md) | Coding standard | Standard | Choose a formatter and a linter per language, commit their configuration and fail CI on warnings. That is most of the value. Write topic entries only for topics you actually hit. | +| [EG-014](version-control-and-versioning.md) | Version control, commits and versioning | Starter | Use Conventional Commits from day one, because they drive your version and changelog. Signed tags are recommended but optional. Use release branches only if you support old versions. | +| [EG-015](pull-requests-and-review.md) | Pull requests and code review | Standard | Still use PRs, mainly for agent work and for reviewing your own diff. Self-review with the template checklist. Waive CODEOWNERS and turnaround rules, but keep 'CI must pass before merge' and 'small PRs'. | +| [EG-016](pipelines-and-quality-gates.md) | Pipelines and quality gates | Standard | Use one workflow that runs the same `make` targets as your machine: format check, build, tests, lint and secret scan. Pin the toolchain with a container or lockfile. Skip rotations, service levels and HIL queueing. | +| [EG-017](testing-principles-and-strategy.md) | Testing principles and strategy | Standard | Write unit tests on the host for logic, a regression test for every bug, and a smoke test for the whole thing. A coverage ratchet is cheap and effective. Skip rig management unless you own a rig. | +| [EG-018](defects-and-technical-debt.md) | Defect and technical debt management | Standard | An issue list with `bug` and `tech-debt` labels is enough. Reproduce, add a regression test, and note the root cause for anything serious. Keep a short debt list and clear it regularly. | +| [EG-019](logging-debug-and-diagnostics.md) | Logging, debug and fault diagnostics | Standard | Use one logging function with levels and module tags from the start, and no stray prints. Embed a build ID. For hardware projects, keep a one-page debug setup note. Skip binary log tokenisation unless flash is tight. | +| [EG-020](observability-and-telemetry.md) | Observability and telemetry | Full | Needed only if the project runs somewhere you cannot see. Define three or four health questions, emit versioned metrics and set a few alerts that each have an action. Skip it for local tools. | +| [EG-021](release-update-and-production.md) | Release, update and production | Standard | Tag, build once, generate the changelog from commits and publish. Keep a one-line release checklist and write down which versions you support. Skip staged rollout and production provisioning unless you ship devices. | +| [EG-022](security-and-data-protection.md) | Security and data protection | Starter | Non-negotiable even solo: secret scanning, multi-factor authentication, dependency updates and a written list of what data may go to which AI tool. Do a one-page threat model only if the project is internet-facing or handles other people's data. | +| [EG-023](safety-regulatory-and-certification.md) | Safety, regulatory and certification | Full | Adopt only if the project is subject to a regulation or safety standard (for example radio, mains-connected, medical, or sold as a product). Otherwise record 'not applicable' in ADR-0001. | +| [EG-024](planning-delivery-and-improvement.md) | Planning, delivery and improvement | Full | A backlog, a short Definition of Done and a five-line risk list are enough. Hold a short retrospective after each release or milestone. Skip formal metrics unless they help you. | +| [EG-025](research-and-prototyping.md) | Research, prototyping and experiments | Standard | This is where personal projects live. Keep a `prototypes/` folder, one experiment record per spike, and a firm stop, extend or graduate decision. Never copy prototype code into the product: re-implement it against the graduation checklist. | +| [EG-026](competence-tooling-and-onboarding.md) | Competence, tooling and onboarding | Full | Treat the expected-depth table as your learning plan, especially git internals and the debug tools. Keep a short learning log and a `docs/how-to/` page for anything you had to relearn twice. A one-command environment setup is worth doing even solo, and it is what lets agents work reliably. | + +## What to scale down + +- **Roles collapse to you.** Owner, approver, release manager and security lead are all you. Keep the separation of activities: write, review, verify. +- **Approvals become self-review.** Review your own diff with the PR template checklist. Agent-authored changes are reviewed by you, line by line, with CI green, before merging. +- **RFCs become design notes.** Drop sponsors and comment periods. Write the note in `docs/rfc/` before a large change. +- **The conformance profile is your ADR-0001.** There is only one product, so record the profile you adopted and any tailoring there. All core rules still apply, and [core-rules.md](core-rules.md) lists them. +- **Registers and cycles are optional.** Ownership registers, review-by dates and competence matrices are worth having only when several people share the work. +- **Relax thresholds but keep ratchets.** PR size, coverage targets and patch times can be looser. Coverage must still not fall and warnings must stay at zero. +- **Heavy infrastructure only if you own it.** HIL queueing, on-call rotations, service levels and staged rollout apply only when you run that infrastructure. + +## What never scales down + +1. Everything is in version control, with Conventional Commits (VCS-03). +1. No secrets in the repository, logs or prompts, with scanning in CI (SEC-05). +1. CI runs the same commands you run locally, and no check is skipped to get a green build (PIP-01, PIP-08, TST-15). +1. Every behaviour you rely on has a test, and every bug gets a regression test (TST-07). +1. A human reviews agent output before it reaches `main` (AGT-03, AGT-04, AGT-07). +1. Agent boundaries and data classes for AI tools are written down and respected (AGT-05, AGT-08, SEC-10). +1. Dependencies are pinned and their licences known (TEC-04, TEC-05). +1. Significant decisions are recorded as ADRs, because agents and future you have no memory (ADR-03). +1. Documentation is updated in the same change as the behaviour (DOC-04). + +## Bootstrap a new project + +1. Create the repository. Copy the guidelines and the [templates](../templates/index.md) you need into its `docs/` folder, and add an [`AGENTS.md`](../templates/agents-md.md) and a `CLAUDE.md` that contains `@AGENTS.md`. +1. Choose your profile. Set `status: active` on the guidelines you adopt, and delete or mark `deprecated` the rest. +1. Fill in `AGENTS.md`: a project summary and the commands for build, test, lint and everything CI runs. +1. Add one command (for example `make ci`) that runs the format check, build, tests, lint and secret scan, and point CI at it. +1. Install a commit-msg hook that enforces Conventional Commits, and a secret scanner as a pre-commit hook. +1. Protect `main` where your host supports it (required PRs and required checks). Where it does not, adopt the habit of never committing to `main` directly. +1. Copy the [pull request template](../templates/pull-request-and-commits.md) to `.github/pull_request_template.md`, or your host's equivalent. +1. Write ADR-0001: the profile you adopted, the guidelines that do not apply (for example [EG-023](safety-regulatory-and-certification.md)), and your top three quality attributes ([EG-003](quality-policy-and-attributes.md)). +1. Add the first requirements to `docs/requirements/` and tag `v0.1.0`. + +## Working with AI agents on personal projects + +### Setup + +- Keep `AGENTS.md` at the repository root. For Claude Code, `CLAUDE.md` contains `@AGENTS.md` so that both tools read the same instructions. +- Keep [agent-rules.md](agent-rules.md) in the repository and tell the agent to read it at the start of a task. +- Keep a short approved-tools list: which AI tools may see which data class ([EG-022](security-and-data-protection.md), SEC-10). +- Point the agent at the [principles catalogue](principles-catalogue.md). When a rule is silent, the start-here principles decide. + +### Each task + +- Use the [task brief template](../templates/agent-task-brief.md): goal, requirement IDs, scope, constraints and definition of done. +- Ask the agent for its assumptions and open questions first (CP-20). For anything larger than a small fix, ask for a short plan before code. +- Keep tasks small and single-purpose (AGT-10). +- Require the agent to run the check command and report the results (AGT-09). +- Review the diff yourself. Does every changed line trace to the request (CP-16)? Would the new tests fail if the code were wrong (AGT-11)? + +### Between sessions + +- Agents have no memory. Put context in the repository (ADRs, requirements, how-to pages, `AGENTS.md`) and not in chat. +- When an agent gets something wrong twice, add the missing rule or fact to `AGENTS.md` or the guidelines. +- Version the prompts and skills you reuse (AGT-12). + +### Data + +- Never paste secrets, keys or other people's personal data into a prompt (SEC-05, AGT-08). +- Classify what you share using the scheme in [EG-022](security-and-data-protection.md), and only use tools you have approved for that class. diff --git a/content/guidelines/pipelines-and-quality-gates.md b/content/guidelines/pipelines-and-quality-gates.md new file mode 100644 index 0000000..c3e2d15 --- /dev/null +++ b/content/guidelines/pipelines-and-quality-gates.md @@ -0,0 +1,109 @@ +--- +id: EG-016 +title: "Pipelines and quality gates" +status: draft +owner: "Build and tooling lead" +version: 0.1.0 +part: "Code and build" +related: [EG-003, EG-011, EG-013, EG-014, EG-017, EG-021, EG-022] +review-by: 2027-09-30 +supersedes: +--- + +# EG-016: Pipelines and quality gates + +- **Owner:** Build and tooling lead +- **Applies to:** All build, test and release pipelines and their supporting infrastructure +- **Enforcement:** Branch protection, pipeline as code, CODEOWNERS +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-011](technology-and-third-party.md), [EG-013](coding-standard.md), [EG-014](version-control-and-versioning.md), [EG-017](testing-principles-and-strategy.md), [EG-021](release-update-and-production.md), [EG-022](security-and-data-protection.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Automate the enforcement of every other guideline in a pipeline that is fast, reproducible, trustworthy and owned. + +## Rules + +1. **PIP-01** *(core)* Pipelines MUST be defined as code in the repository. Logic MUST live in scripts and the build system, runnable locally with the same command. CI only orchestrates. +1. **PIP-02** *(core)* Toolchains MUST be pinned in versioned container images. Builds MUST be hermetic, with network access limited to declared package caches. +1. **PIP-03** Builds SHOULD be reproducible: the same commit and toolchain produce bit-identical artefacts (no embedded timestamps, `SOURCE_DATE_EPOCH` respected). Reproducibility MUST be verified periodically by rebuild and comparison. +1. **PIP-04** Stages MUST be ordered for fast feedback as in the stage table below. Stages 1 to 4 SHOULD complete within ten minutes. +1. **PIP-05** *(core)* Artefacts MUST be built once and promoted. The artefact that is tested MUST be the artefact that is released. It is stored immutably with metadata (commit, pipeline run, toolchain digest). +1. **PIP-06** Gates MUST be applied by branch type as in the gates table below. +1. **PIP-07** Mandatory gate content MUST include: warnings as errors, analyser thresholds, size and timing budget checks (AP-06), coverage that does not decrease, documentation build, requirement trace check, licence and SBOM check and secret scanning. +1. **PIP-08** *(core)* Gates MUST NOT be bypassed. An override needs a recorded exception (GOV-09) approved by the pipeline owner. +1. **PIP-09** A flaky test MUST be quarantined within one working day of detection, with an owner and a fix deadline (proposed ten working days). Quarantined tests are tracked and the flaky rate is reported. +1. **PIP-10** A broken `main` is the top priority. The policy is revert first, and `main` MUST be restored within one working day. +1. **PIP-11** The pipeline MUST have a named owner, a rotation for breakage response and service levels. Pipeline changes MUST go through PR with CODEOWNERS review. +1. **PIP-12** Each release build MUST produce supply chain outputs: SBOM, signed artefacts and provenance attestation, retained under the retention policy ([EG-021](release-update-and-production.md)). +1. **PIP-13** *(core)* Repository settings, branch protection and required checks MUST be held as code and applied by tooling (policy as code). +1. **PIP-14** Shared HIL rigs MUST be accessed through the pipeline with booking or queueing, health checks and recorded results ([EG-017](testing-principles-and-strategy.md)). +1. **PIP-15** Pipeline duration, queue time and failure reasons MUST be monitored, and used to keep feedback fast. +1. **PIP-16** Pipeline credentials MUST be least-privilege and short-lived. Release signing MUST use protected keys and follow the two-person rule ([EG-022](security-and-data-protection.md)). + +## Stages + +```mermaid +flowchart LR + A[Commit and format lint] --> B[Build all targets] + B --> C[Host unit tests] + C --> D[Static analysis] + D --> E[Integration and simulation] + E --> F[HIL tests] + F --> G[System and soak nightly] + G --> H[Package sign and evidence] +``` + +## Gates by branch type + +| Branch type | Required stages | +|-------------|-----------------| +| Feature branch | Commit lint, build, unit tests, static analysis | +| `main` | All of the above, plus integration, simulation and HIL | +| Release candidate | All stages, plus system and soak, trace report, SBOM, signing and the evidence pack | + +## Compliance + +| Rule | Checked by | +|------|------------| +| PIP-01, PIP-02 | Review of pipeline definitions, container digests in repository | +| PIP-03 | Scheduled rebuild-and-compare job | +| PIP-07, PIP-08 | Branch protection required checks | +| PIP-09, PIP-15 | Pipeline metrics report | +| PIP-13 | Policy as code drift check | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] The pipeline is defined as code, with logic in scripts that run locally by the same command (PIP-01) +- [ ] Toolchains are pinned in versioned container images and builds are hermetic (PIP-02) +- [ ] Reproducibility is verified periodically (PIP-03) +- [ ] Stages are ordered for fast feedback and stages 1 to 4 finish within ten minutes (PIP-04) +- [ ] Artefacts are built once, promoted and stored immutably with metadata (PIP-05) +- [ ] Gates are defined by branch type (PIP-06) +- [ ] Mandatory gate content is present: warnings, analysis, budgets, coverage, docs, trace, licence and SBOM, secrets (PIP-07) +- [ ] Gates cannot be bypassed without a recorded exception (PIP-08) +- [ ] Flaky tests are quarantined within one working day (PIP-09) +- [ ] The policy is revert first, and `main` is restored within one working day (PIP-10) +- [ ] The pipeline has an owner, a rotation and service levels (PIP-11) +- [ ] SBOM, signed artefacts and provenance are produced per release (PIP-12) +- [ ] Repository settings and required checks are held as code (PIP-13) +- [ ] Shared HIL rigs are accessed through the pipeline (PIP-14) +- [ ] Pipeline duration, queue time and failure reasons are monitored (PIP-15) +- [ ] Credentials are least-privilege and short-lived, and release signing uses the two-person rule (PIP-16) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Use one workflow that runs the same `make` targets as your machine: format check, build, tests, lint and secret scan. Pin the toolchain with a container or lockfile. Skip rotations, service levels and HIL queueing. + +### AI agents + +- Do not modify pipeline definitions, required checks or gates unless the task says so (AGT-05). +- Never skip, disable or loosen a check or test to make a pipeline pass (PIP-08, TST-15). +- Run the same commands CI runs before proposing a change (PIP-01). +- If a test is flaky, report it and do not delete it (PIP-09). +- If your change breaks `main`, revert first (PIP-10). diff --git a/content/guidelines/planning-delivery-and-improvement.md b/content/guidelines/planning-delivery-and-improvement.md new file mode 100644 index 0000000..dc826b2 --- /dev/null +++ b/content/guidelines/planning-delivery-and-improvement.md @@ -0,0 +1,111 @@ +--- +id: EG-024 +title: "Planning, delivery and improvement" +status: draft +owner: "Engineering manager" +version: 0.1.0 +part: "Delivery, research and people" +related: [EG-002, EG-003, EG-018, EG-020, EG-021] +review-by: 2027-09-30 +supersedes: +--- + +# EG-024: Planning, delivery and improvement + +- **Owner:** Engineering manager +- **Applies to:** All planning, delivery management and improvement activity +- **Enforcement:** Tracker configuration, review, retrospectives +- **Related:** [EG-002](values-and-ways-of-working.md), [EG-003](quality-policy-and-attributes.md), [EG-018](defects-and-technical-debt.md), [EG-020](observability-and-telemetry.md), [EG-021](release-update-and-production.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Plan honestly, deliver visibly and learn continuously: shared definitions of ready and done, forecasts that admit uncertainty, managed risks and dependencies, and a small set of metrics used for improvement. + +## Rules + +1. **PLN-01** Work MUST be planned in time-boxed iterations or flow with a visible backlog. Every item traces to a requirement, a defect or an approved improvement (REQ-13). +1. **PLN-02** Each work item type MUST have a Definition of Ready (below). +1. **PLN-03** Each work item type MUST have a Definition of Done (below), and the PR template checklist reflects it. +1. **PLN-04** Estimates SHOULD be relative or ranges, calibrated from history. Forecasts MUST be given as ranges with stated confidence and be updated regularly. Single-point commitments MUST NOT be made for uncertain work. +1. **PLN-05** A risk register MUST be kept (ID, description, cause, impact, probability, exposure, mitigation, owner, trigger, review date), reviewed at least fortnightly, with top risks visible to the sponsor. +1. **PLN-06** Dependencies across teams, sites, hardware and vendors MUST be recorded with an owner and a need-by date. +1. **PLN-07** Escalation thresholds MUST be defined for risk exposure, schedule slip and blocked time (proposed: blocked for more than two working days) and follow VAL-04. +1. **PLN-08** Capacity MUST be reserved for support, technical debt (DEF-10) and improvement. +1. **PLN-09** A small set of metrics MUST be agreed and tracked: lead time, release frequency, change failure rate, time to restore, escaped defects, review latency, build stability and flaky test rate. Metrics are used at team level for improvement and MUST NOT be used to rank individuals. +1. **PLN-10** Retrospectives MUST be held at a fixed cadence. Actions have an owner and date in the tracker and are reviewed at the next retrospective. +1. **PLN-11** Blameless postmortems MUST be held for severity 1 and 2 escapes, failed releases, pipeline outages and security incidents. They follow the template below and are published within five working days. +1. **PLN-12** Process and tooling improvements MUST be treated as first-class work in the backlog, tied to the metrics. +1. **PLN-13** Friction with a guideline MUST be raised as an issue or RFC, so that guidelines are kept useful (GOV-06). +1. **PLN-14** Status reporting SHOULD be derived from tracker and pipeline data, not compiled manually. + +## Definition of Ready + +| Work item type | Ready when | +|----------------|------------| +| Feature | Requirement linked and approved, acceptance criteria stated, dependencies identified, estimate given, need for an ADR or RFC decided | +| Defect | Reproducible or diagnostics attached, severity assigned, component identified | +| Technical debt or improvement | Impact described, scope bounded, owner named | +| Spike | Question stated, time-box set, exit criteria defined ([EG-025](research-and-prototyping.md)) | + +## Definition of Done + +| Work item type | Done when | +|----------------|-----------| +| Feature | Reviewed and merged, tests added and passing, requirement trace updated, documentation updated, ADR written if needed, logging and diagnostics added, budgets checked | +| Defect | Root cause recorded, regression test added, fix verified by someone else | +| Technical debt or improvement | Change merged, debt register and documentation updated | +| Spike | Findings recorded in an experiment record and a decision made | + +## Postmortem template + +```markdown +## Summary +## Impact +## Timeline +## Contributing factors +## What went well +## Actions (owner and date) +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| PLN-02, PLN-03 | Tracker workflow and PR template | +| PLN-05, PLN-06 | Risk register and dependency register reviews | +| PLN-09 | Metrics dashboard generated from tracker and pipeline data | +| PLN-10, PLN-11 | Action tracking in the tracker | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Work is planned in time-boxed iterations or flow with a visible backlog, and every item is traceable (PLN-01) +- [ ] A Definition of Ready exists per work item type (PLN-02) +- [ ] A Definition of Done exists per type and the PR template reflects it (PLN-03) +- [ ] Estimates are ranges and forecasts state confidence (PLN-04) +- [ ] A risk register is maintained and reviewed fortnightly (PLN-05) +- [ ] Dependencies are recorded with an owner and need-by date (PLN-06) +- [ ] Escalation thresholds are defined (PLN-07) +- [ ] Capacity is reserved for support, debt and improvement (PLN-08) +- [ ] A small metric set is tracked and not used to rank individuals (PLN-09) +- [ ] Retrospectives are held and actions are tracked (PLN-10) +- [ ] Blameless postmortems follow severity 1 and 2 escapes, failed releases, outages and security incidents (PLN-11) +- [ ] Improvements are first-class backlog items (PLN-12) +- [ ] Friction with a guideline is raised as an issue or RFC (PLN-13) +- [ ] Status is derived from tracker and pipeline data (PLN-14) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Full** profile (see [personal-projects.md](personal-projects.md)). A backlog, a short Definition of Done and a five-line risk list are enough. Hold a short retrospective after each release or milestone. Skip formal metrics unless they help you. + +### AI agents + +- Treat the Definition of Done for the work item type as the finish line: reviewed, tested, documented, traced, with diagnostics and budgets checked (PLN-03). +- State uncertainty as a range and not a single date (PLN-04). +- Report blockers and risks in the PR or task summary and do not work around them silently (PLN-05, PLN-06). +- Draft postmortems from the template when asked (PLN-11). diff --git a/content/guidelines/principles-catalogue.md b/content/guidelines/principles-catalogue.md new file mode 100644 index 0000000..1ae8456 --- /dev/null +++ b/content/guidelines/principles-catalogue.md @@ -0,0 +1,249 @@ +--- +title: "Principles catalogue" +status: draft +owner: "Architecture group" +last-reviewed: 2026-09-29 +--- + +# Principles catalogue + +## Purpose + +Principles are the reasons behind the rules. Rules say what to do and can be checked. Principles say what to aim for, and decide the cases that no rule covers. When a rule is silent or unclear, apply the relevant principle, say which one you applied, and record any significant decision as an ADR. + +This page gathers every principle set in one place. Each set is owned by the guideline named in its heading, and changes to a principle go through an RFC ([EG-008](decision-records.md)). + +## How to use principles + +1. Start with the [start-here list](#start-here), which holds the principles that matter most day to day and for AI agents. +1. In reviews, cite a principle or rule ID so that feedback is explainable (CPR-02). +1. Where principles conflict, use the precedence order in [EG-007](architecture-principles.md) (PRN-03). A product may tailor that order through its quality attribute profile, recorded in an ADR. +1. Where a principle can be checked by a tool, it should be (PRN-04). + +> **Note:** Principles are not rules. They are not audited item by item, and they do not override a core rule. A core rule that seems wrong is changed through an RFC and not worked around. + +## Start here + +Fourteen principles to learn first. They cover most day-to-day judgement and are the ones an AI agent should apply when a rule is silent. + +| ID | Principle | Statement | +|----|-----------|-----------| +| AP-02 | Layered, one-way dependencies | Hardware abstraction, platform services and application logic are separate layers, and dependencies point in one direction. | +| AP-05 | Simplicity | Choose the simplest design that meets the requirements. Complexity must earn its place. | +| AP-07 | Design for test | Logic is separable from hardware so that it runs on the host, and every component has seams for test doubles and HIL. | +| AP-13 | Prefer reversible decisions | Make cheap-to-reverse decisions quickly. Take irreversible ones deliberately, with evidence. | +| CP-02 | Simplicity, no speculation | Write the minimum code that solves the stated problem. | +| CP-06 | No silent failure | Errors are detected and either handled or propagated, never ignored. | +| CP-16 | Surgical changes | Change only what the task needs, and clean up only what your own change made redundant. | +| CP-20 | Surface assumptions | State assumptions and ask when something is unclear. Present alternatives rather than choosing silently. | +| TP-08 | Bugs start with a test | A defect is reproduced by a failing test before it is fixed. | +| SP-06 | Trust nothing at boundaries | Input from outside a trust boundary is untrusted and validated. | +| LP-01 | Build once, promote | The artefact that is tested is the artefact that is released. | +| AIP-01 | Humans stay accountable | The person who submits a change owns it and can explain it. | +| AIP-06 | Context lives in the repository | Agents have no memory between sessions, so context is kept in the repository. | +| WP-01 | Docs live with the code | Documentation is text in version control, next to what it describes. | + +## Guiding principles for consistency (GP) + +Set out in [EG-001](guidelines-governance.md). + +| ID | Principle | Statement | +|----|-----------|-----------| +| GP-01 | One way to do each thing | Where two approaches exist, the guideline picks one and records why. | +| GP-02 | If a tool can check it, a tool checks it | Reviewers spend attention on design, not formatting. | +| GP-03 | Conventions drive outcomes | A convention that does not feed an automated result (version, changelog, trace report, release) is etiquette and will decay. Prefer conventions that machines can read. | +| GP-04 | Everything is text in version control | Requirements, decisions, diagrams, pipelines, repository settings and guidelines are reviewed, versioned and diffable. | +| GP-05 | Record decisions once, at the right level | Context in an RFC, outcome in an ADR, rules in a guideline, enforcement in tooling. | +| GP-06 | Local equals CI | Any check that runs in the pipeline runs identically on a developer machine with one command. | +| GP-07 | Guidelines are short and owned | A few pages each, one named owner, a review date, and a stated way of checking compliance. | +| GP-08 | Understand the tool | Using a tool without knowing how it works (git, the linker, the RTOS, the debug probe) is a risk. Each guideline names the competence expected. | +| GP-09 | Explain the why | Rules carry a rationale where it is not obvious, so that people and agents can apply them sensibly at the edges. | +| GP-10 | Sequence adoption | Start with what other practices depend on: commit conventions, a single version source and a reproducible pipeline. | + +## Engineering values (EV) + +Set out in [EG-002](values-and-ways-of-working.md). + +| ID | Principle | Statement | +|----|-----------|-----------| +| EV-01 | Ownership | Every asset has a named owner accountable for its quality and future | +| EV-02 | Quality built in | Quality is produced by the way we work, not inspected in at the end | +| EV-03 | Written and open by default | Decisions, agreements and handovers are recorded where others can find them | +| EV-04 | Evidence over opinion | Claims about performance, defects and risk are backed by data | +| EV-05 | Blameless learning | We fix systems and processes, not people | +| EV-06 | Small steps | Small, frequent, reversible changes beat large, rare ones | +| EV-07 | Simplicity | The simplest solution that meets the need is preferred | +| EV-08 | Respect for others' time | Cross-site work is designed for time zones, and requests are clear and complete | + +## Requirements and decision principles (RP) + +Set out in [EG-006](requirements-and-traceability.md) and [EG-008](decision-records.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| RP-01 | Needs before solutions | Requirements say what is needed and why, not how it is built. | Solution-neutral wording, with the design recorded separately. | +| RP-02 | One requirement, one statement | Each requirement is atomic and unambiguous. | No compound requirements, and no words such as 'and/or' or 'appropriate'. | +| RP-03 | Verifiable or not a requirement | If a requirement cannot be verified, it is rewritten or removed. | Every requirement names a verification method. | +| RP-04 | Traceable both ways | Every requirement traces up to a need and down to design, code and test. | The pipeline generates the trace report and gaps are visible. | +| RP-05 | Measurable quality | Quality attributes are stated as measurable scenarios. | Environment, stimulus, response and measure, from the NFR catalogue. | +| RP-06 | Stable identity | Identifiers are permanent and never reused. | Retired requirements stay in the record. | +| RP-07 | Manage change, do not avoid it | Change is expected, assessed for impact and recorded. | A change impact check flags dependent tests and documents. | +| RP-08 | Decide with evidence and record why | Decisions weigh real options with evidence and are written down. | ADRs with options, consequences and verification. | + +## Architecture principles (AP) + +Set out in [EG-007](architecture-principles.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| AP-01 | Requirements first | Architecture exists to satisfy requirements and quality attributes, and is traceable to them. | Every component and interface traces to a requirement. Quality attributes are stated measurably. | +| AP-02 | Layered, one-way dependencies | Hardware abstraction, platform services and application logic are separate layers, and dependencies point in one direction. | No application code touches registers directly. Dependency direction is checked in CI. | +| AP-03 | Isolate what changes | Volatile things (silicon, vendor SDKs, protocols, cloud endpoints) sit behind stable interfaces that we own. | Vendor code is wrapped by adapters and can be swapped for tests or new hardware. | +| AP-04 | Interfaces are contracts | Every interface is explicit, owned, versioned and tested for compatibility. | Definitions in machine-readable form where possible. Breaking changes follow semantic versioning. | +| AP-05 | Simplicity | Choose the simplest design that meets the requirements. Complexity must earn its place. | Fewer components, fewer modes, fewer configuration options. | +| AP-06 | Budgets are architecture | Flash, RAM, CPU, power, latency and bandwidth are allocated to components and tracked from the start. | Budget table in the architecture document. Size and timing are tracked in CI and a regression fails the gate. | +| AP-07 | Design for test | Logic is separable from hardware so that it runs on the host, and every component has seams for test doubles and HIL. | Hardware access behind interfaces, dependency injection, no hidden global state. | +| AP-08 | Design for diagnosis | The system explains its own state and failures. | Logging, fault codes, crash capture and telemetry are architectural components ([EG-019](logging-debug-and-diagnostics.md), [EG-020](observability-and-telemetry.md)). | +| AP-09 | Secure by design | Security is a design property: least privilege, defence in depth, secure defaults, minimal attack surface. | Trust boundaries, threat model and key handling are designed before implementation. | +| AP-10 | Fail safe and recover | Every fault has defined behaviour, and there is a route back to a known good state. | Watchdog, fault containment, power-fail handling, rollback on a failed update. | +| AP-11 | Design for the whole lifecycle | Field update, compatibility over time, manufacture, service and end of life are designed in from the outset. | Update mechanism, version compatibility rules, provisioning flow, decommissioning. | +| AP-12 | Deterministic where it matters | Timing-critical behaviour is predictable and bounded. | Bounded resource use, no dynamic allocation in critical paths, analysable scheduling. | +| AP-13 | Prefer reversible decisions | Make cheap-to-reverse decisions quickly. Take irreversible ones deliberately, with evidence. | Flash layout, crypto scheme and wire formats need an RFC and an ADR. | +| AP-14 | Reuse before build, with control | Reuse proven components and buy where sensible, but own the boundary. | Third-party choices go through [EG-011](technology-and-third-party.md) and sit behind interfaces (AP-03). | +| AP-15 | Check architecture continuously | Architectural intent is enforced by automated checks as well as review. | Dependency rules, budget checks and interface compatibility tests run in the pipeline. | +| AP-16 | Structure follows ownership | Component boundaries match ownership, so that responsibilities are clear across sites. | Each component has an owner. Cross-site interfaces are contracts (AP-04). | +| AP-17 | Record the why | Decisions and their reasoning are kept where the code lives. | ADRs ([EG-008](decision-records.md)) and architecture documents in the repository ([EG-004](documentation-as-code.md)). | + +## Coding principles (CP) + +Set out in [EG-012](coding-principles.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| CP-01 | Clarity over cleverness | Code is read far more often than it is written. Write for the next reader. | Plain constructs and descriptive names. Anything clever is explained. | +| CP-02 | Simplicity, no speculation | Write the minimum code that solves the stated problem. | No unrequested features, no abstractions for single use, no configurability nobody asked for. | +| CP-03 | Correct first, then small and fast | Optimise against measured budgets, not intuition. | Measure before optimising. Record the reason for any non-obvious optimisation. | +| CP-04 | Make wrong code hard to write | Use types, `const`, enumerations and narrow interfaces to prevent misuse. | Distinct types over raw integers, no ambiguous boolean parameters, invalid states unrepresentable where practical. | +| CP-05 | Validate at boundaries, assert invariants | Check external input where it enters the system. Assert what must never be false. | Every interface, protocol parser and configuration loader validates. Assertion policy follows [EG-019](logging-debug-and-diagnostics.md). | +| CP-06 | No silent failure | Errors are detected and either handled or propagated, never ignored. | Return values checked, error codes defined, every failure path has defined behaviour. | +| CP-07 | Predictable resource use | Memory, stack and execution time are bounded and known. | Static allocation by default, bounded loops, stack usage analysed. | +| CP-08 | No undefined behaviour | Do not rely on behaviour the language leaves undefined or unspecified. | Restricted language subset, warnings as errors, static analysis, sanitisers in host tests. | +| CP-09 | Limit shared state | Minimise globals and shared mutable data. State has one owner. | Documented ownership and locking or message-passing rules, `const` by default. | +| CP-10 | High cohesion, low coupling | Small units with one clear responsibility and narrow interfaces. | Size guidance for functions and modules, dependency direction respected (AP-02). | +| CP-11 | Keep hardware separate from logic | Logic never touches registers or vendor APIs directly. | Hardware abstraction layer, logic testable on the host (AP-07). | +| CP-12 | Code and tests together | Code is written to be tested, and changes arrive with their tests. | Tests in the same PR, requirement IDs in test names, a bug fix starts with a failing test. | +| CP-13 | Comments explain why | Names and structure say what. Comments record intent, constraints and non-obvious reasons. | No comments that restate the code. Reference the requirement, ADR or erratum where relevant. | +| CP-14 | Consistency over preference | Follow the agreed style and existing conventions, even where you would choose differently. | Formatter and linter enforce style, so there are no style debates in review. | +| CP-15 | Warnings are defects | The build is free of compiler and analyser warnings. | Warnings as errors. Suppressions are inline, justified and reviewed. | +| CP-16 | Surgical changes | Change only what the task needs, and clean up only what your own change made redundant. | Small diffs. Every changed line traces to the request. Unrelated problems are reported, not fixed in passing. | +| CP-17 | Secure by habit | Treat all input as untrusted and avoid unsafe constructs. | Banned function list, no secrets in code, safe buffer handling, constant-time comparison for secrets. | +| CP-18 | Observable by construction | Code emits what is needed to understand its behaviour in the field. | Logging and fault reporting follow [EG-019](logging-debug-and-diagnostics.md). | +| CP-19 | Duplication over the wrong abstraction | Abstract when a pattern is proven, not before. | Rule of three, and refactor only with tests in place. | +| CP-20 | Surface assumptions | State assumptions and ask when something is unclear. Present alternatives rather than choosing silently. | Assumptions recorded in the PR description or ADR. Open questions raised before implementation, not after. | +| CP-21 | Deviations are explicit | When a principle or rule is broken, say so where the code is and record why. | Suppression comments carry the rule ID and justification. Deviation log maintained. | + +## Testing principles (TP) + +Set out in [EG-017](testing-principles-and-strategy.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| TP-01 | Cheapest level first | Test at the cheapest level that can catch the defect. Most tests run on the host. | Most tests run on the host, and hardware tests only where hardware is needed. | +| TP-02 | Trace to requirements | Every test verifies a requirement, and every requirement has verification. | Test names carry requirement IDs and the trace report shows gaps. | +| TP-03 | Deterministic and independent | Tests give the same result every run, in any order. | No dependence on time, order or shared state, and flaky tests are quarantined. | +| TP-04 | Behaviour over implementation | Tests check what the unit promises, not how it does it. | Assert on outputs and observable effects, not on internal calls. | +| TP-05 | Fail clearly | A failing test says what went wrong and where. | Assertion messages and logs show expected against actual. | +| TP-06 | Automate what repeats | Anything run more than twice is automated. | Regression, HIL and soak runs are scheduled in the pipeline. | +| TP-07 | Tests are code | Tests are reviewed, refactored and maintained to the same standard. | Test code follows the coding standard and is refactored. | +| TP-08 | Bugs start with a test | A defect is reproduced by a failing test before it is fixed. | A failing test comes first, then the fix, and the test stays. | +| TP-09 | Test what can go wrong | Negative, boundary and fault-injection cases are as important as the happy path. | Fault injection, boundary values and malformed input are included. | +| TP-10 | Hardware for what only hardware shows | Timing, power, peripherals and environment need real hardware. | Timing, power and peripheral behaviour are verified on rigs. | +| TP-11 | Controlled environments | The environment for every test run is known and recorded. | Toolchain, firmware version and rig configuration are recorded with every result. | + +## Security principles (SP) + +Set out in [EG-022](security-and-data-protection.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| SP-01 | Secure by design | Security is designed in before implementation, not added afterwards. | Threat model at design time, with security requirements traced. | +| SP-02 | Least privilege | Every component, person, process and credential has only the access it needs. | Memory protection, scoped tokens, role-based access. | +| SP-03 | Defence in depth | No single control is relied on, and any one layer may fail. | Layered checks, isolation, secure boot together with signed updates. | +| SP-04 | Minimise attack surface | Fewer interfaces, features and privileges mean fewer weaknesses. | Unused interfaces removed, debug ports locked in production. | +| SP-05 | Secure defaults | The default configuration is the safe one. | Risky features off by default, and no default credentials. | +| SP-06 | Trust nothing at boundaries | Input from outside a trust boundary is untrusted and validated. | Parsers validated and fuzzed, and lengths and ranges checked. | +| SP-07 | Protect secrets and keys | Secrets never appear in source, logs or prompts, and keys have a managed lifecycle. | Vault or HSM, rotation and revocation, secret scanning. | +| SP-08 | Use proven cryptography | Never invent cryptography. Use vetted algorithms and libraries. | Approved list, and crypto agility considered. | +| SP-09 | Fail securely | On failure the system stays safe and does not leak or open up. | Deny by default, and a safe state on fault. | +| SP-10 | Assume compromise | Design so that breaches are detected, contained and recoverable. | Audit events, revocation, recovery and the ability to update. | +| SP-11 | Security is continuous | Vulnerabilities are found, tracked and fixed for the whole life of the product. | SBOM monitoring and patch times. | +| SP-12 | Minimise and classify data | Collect and keep only what is needed, and handle it by its classification. | Classification scheme and retention limits. | + +## Diagnostics and operations principles (OP) + +Set out in [EG-019](logging-debug-and-diagnostics.md) and [EG-020](observability-and-telemetry.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| OP-01 | The system explains itself | Logs, fault codes and crash capture are built in from the start. | A common logging facility, a fault registry and crash dumps. | +| OP-02 | Measure before you optimise | Decisions about performance and reliability use data, not intuition. | Budgets, traces and telemetry. | +| OP-03 | Diagnostics are budgeted | The cost of diagnostics in memory, power and bandwidth is planned. | Log and telemetry budgets, rate limiting. | +| OP-04 | Fail visibly and recover | Faults are detected, recorded and recovered from, never silent. | Defined fault behaviour, watchdog, reset reason captured. | +| OP-05 | Every alert has an action | An alert has an owner, a runbook and a required action. | No alert without a response. | +| OP-06 | Same signals in lab and field | Lab and field emit the same diagnostics and telemetry. | Parsers and dashboards are exercised before release. | +| OP-07 | Reproducible from evidence | A field failure can be turned into a local reproduction and then a test. | Build IDs, symbols retained, a documented procedure. | +| OP-08 | Private by default | Logs and telemetry hold no secrets and no personal data unless approved. | Classification applied to every field. | +| OP-09 | Production is not debug | Debug and production builds differ, and production interfaces are locked. | Separate build configurations and release checks. | +| OP-10 | Field data feeds engineering | What we learn in the field changes requirements, tests and priorities. | A regular field review that feeds triage. | + +## Lifecycle and release principles (LP) + +Set out in [EG-014](version-control-and-versioning.md), [EG-016](pipelines-and-quality-gates.md) and [EG-021](release-update-and-production.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| LP-01 | Build once, promote | The artefact that is tested is the artefact that is released. | Immutable artefacts with metadata, and no rebuild for release. | +| LP-02 | Reproducible builds | The same commit and toolchain give the same result anywhere. | Pinned containerised toolchains. | +| LP-03 | Everything shipped is traceable | Any device can be traced to its version, hardware, components and evidence. | Version from tags, bill of materials, evidence pack. | +| LP-04 | Small, frequent, reversible releases | Smaller releases carry less risk and are easier to reverse. | Short-lived branches and staged rollout. | +| LP-05 | Updates are safe by design | An update is authenticated, atomic and recoverable. | A/B images, power-fail safety, rollback. | +| LP-06 | Roll out in stages and watch | Release to a few first and let real data decide. | Health gates and a halt switch. | +| LP-07 | Support what you ship | Every shipped version has a stated support period and end of life. | Supported version list and security update duration. | +| LP-08 | Evidence is a by-product | Proof of quality and compliance is produced by the process, not assembled afterwards. | Pipeline-generated evidence pack. | +| LP-09 | Compatibility is a promise | Versions state what they are compatible with, and that is tested. | Semantic versioning and a compatibility matrix. | +| LP-10 | Manufacturing is part of design | Provisioning, production test and yield are designed in. | Design for test and controlled key injection. | + +## Documentation principles (WP) + +Set out in [EG-004](documentation-as-code.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| WP-01 | Docs live with the code | Documentation is text in version control, next to what it describes. | Same repository, same PR, same review. | +| WP-02 | One source of truth | Define a fact once, link to it elsewhere, and generate what can be generated. | Generated references, and no hand-kept copies. | +| WP-03 | Audience and purpose first | State who a document is for and what question it answers. | Tutorial, how-to, reference and explanation kept apart. | +| WP-04 | Short, plain, precise | Write plainly, with the conclusion first and terms defined. | Short sentences, active voice, British English. | +| WP-05 | Diagrams are text | Diagrams are source that can be reviewed and diffed. | Mermaid or PlantUML. | +| WP-06 | Reviewed like code | Documents are linted, link-checked and reviewed. | Docs build in CI. | +| WP-07 | Explain why | Record the reasoning as well as the result. | ADRs and rationale fields. | +| WP-08 | Readable by people and agents | Structure, IDs and front matter make documents navigable by both. | Consistent headings and stable identifiers. | +| WP-09 | Keep it or delete it | A stale document is worse than none, so every document has an owner and a review date. | Front matter with owner and review date. | +| WP-10 | One vocabulary | Terms are defined once and used consistently. | A single glossary. | + +## AI agent principles (AIP) + +Set out in [EG-005](ai-agent-usage.md). + +| ID | Principle | Statement | Typical implications | +|----|-----------|-----------|----------------------| +| AIP-01 | Humans stay accountable | The person who submits a change owns it and can explain it. | No merge without a human who understands the change. | +| AIP-02 | Agents are untrusted contributors | Review agent output as you would any new contributor's, and more carefully where it is opaque. | Same PR, review and CI rules. | +| AIP-03 | Controls do not depend on the agent | Enforcement lives in tooling, not in prompts. | Hooks, gates, branch protection and CODEOWNERS. | +| AIP-04 | Explicit beats implicit | Agents have no tribal knowledge, so rules, commands and boundaries are written down. | AGENTS.md and numbered rules. | +| AIP-05 | Small, scoped tasks | One purpose per task, and small diffs. | Task briefs with a definition of done. | +| AIP-06 | Context lives in the repository | Agents have no memory between sessions, so context is kept in the repository. | ADRs, requirements, how-to pages and AGENTS.md. | +| AIP-07 | Evidence over assertion | Agents show test and analysis output and do not just claim success. | Results included in the PR. | +| AIP-08 | Surface assumptions | Agents state assumptions and ask when unclear. | See CP-20. | +| AIP-09 | Least authority, least data | Give agents only the permissions and data the task needs. | Data classes per tool and scoped credentials. | +| AIP-10 | Provenance is recorded | AI assistance and its licence implications are traceable. | Commit trailers and provenance checks. | +| AIP-11 | Learn from corrections | A mistake made twice becomes a written rule. | Update AGENTS.md or the guideline. | diff --git a/content/guidelines/pull-requests-and-review.md b/content/guidelines/pull-requests-and-review.md new file mode 100644 index 0000000..656d6d4 --- /dev/null +++ b/content/guidelines/pull-requests-and-review.md @@ -0,0 +1,121 @@ +--- +id: EG-015 +title: "Pull requests and code review" +status: draft +owner: "Firmware lead" +version: 0.1.0 +part: "Code and build" +related: [EG-005, EG-006, EG-014, EG-016] +review-by: 2027-09-30 +supersedes: +--- + +# EG-015: Pull requests and code review + +- **Owner:** Firmware lead +- **Applies to:** All changes to protected branches, by people and agents +- **Enforcement:** Branch protection, CODEOWNERS, PR template, CI +- **Related:** [EG-005](ai-agent-usage.md), [EG-006](requirements-and-traceability.md), [EG-014](version-control-and-versioning.md), [EG-016](pipelines-and-quality-gates.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Ensure that no change reaches a protected branch without independent review and evidence, and that review effort is spent on design and correctness rather than on what tools can check. + +## Rules + +1. **PRR-01** *(core)* All changes to protected branches MUST be made by PR. Direct pushes MUST NOT be allowed, including for owners. +1. **PRR-02** *(core)* The PR title MUST follow Conventional Commits and becomes the squash commit message. +1. **PRR-03** The PR description MUST follow the template: what and why, linked requirement, ticket or ADR, test evidence, risk and rollback notes, documentation updated, breaking change flag and AI assistance disclosure. +1. **PRR-04** A PR SHOULD have one purpose and stay under about 400 changed lines, excluding generated files. Larger changes MUST be split or accompanied by a walkthrough. +1. **PRR-05** *(core)* CODEOWNERS MUST define required reviewers for each path. At least one owner approves every PR, and safety, security and certified areas need two. +1. **PRR-06** *(core)* Reviewers MUST be someone other than the author who understands the area. An agent MUST NOT count as an approver (AGT-04). +1. **PRR-07** Required checks MUST pass before human review begins. Reviewers MUST NOT re-check what tools check. +1. **PRR-08** Reviewers SHOULD use the review focus by change type (below). +1. **PRR-09** The first review response SHOULD be given within one working day. PRs stale for more than three working days are escalated (VAL-04). +1. **PRR-10** Review comments MUST be labelled `blocking`, `suggestion`, `question` or `nit`, and SHOULD cite rule IDs. Comments address intent and correctness before style. +1. **PRR-11** All blocking comments MUST be resolved. New commits that change the approach MUST dismiss stale approvals. +1. **PRR-12** Draft PRs MAY be used for early feedback. Stacked PRs MAY be used when their order is documented. +1. **PRR-13** After approval, the author merges by squash by default. The branch is deleted after merge. +1. **PRR-14** PR links MUST be retained as review evidence and referenced from release traceability. +1. **PRR-15** Architecture review MUST NOT be a routine PR requirement. Architects are added as reviewers only for changes that meet RFC-12, and all other PRs are reviewed by the component owner or their delegates. + +## Review focus by change type + +| Change type | Reviewer focus | +|-------------|----------------| +| Firmware logic | Correctness against the requirement, error handling, resource use, concurrency, tests | +| Hardware access | Register use, timing, errata, isolation behind the abstraction layer | +| Build and pipeline | Reproducibility, gate integrity, secrets, blast radius | +| Interfaces | Compatibility, versioning, contract tests, documentation | +| Security-relevant code | Trust boundaries, input validation, key handling, threat model update | +| Documentation | Accuracy against code, structure, glossary use | +| Requirements | Testability, atomicity, allocation, change impact | + +## PR template + +```markdown +## What and why + +## Links +- Requirement: +- Ticket: +- ADR or RFC: + +## Evidence +- Tests run and results: +- Analysis and size or timing impact: + +## Risk and rollback + +## Checklist +- [ ] Documentation updated +- [ ] Breaking change flagged +- [ ] AI assistance disclosed (Assisted-by) +- [ ] Certified or regulated area touched (EG-023) +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| PRR-01, PRR-05, PRR-06, PRR-11 | Branch protection and CODEOWNERS | +| PRR-02, PRR-03 | CI: PR title and template checks | +| PRR-04 | CI: diff size warning | +| PRR-15 | CODEOWNERS scope review | +| PRR-09 | Review latency metric ([EG-024](planning-delivery-and-improvement.md)) | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] All changes to protected branches go through PR with no direct pushes (PRR-01) +- [ ] PR titles follow Conventional Commits (PRR-02) +- [ ] PR descriptions follow the template (PRR-03) +- [ ] PRs have one purpose and stay around 400 changed lines or fewer, excluding generated files (PRR-04) +- [ ] CODEOWNERS define required reviewers, with two for safety, security and certified areas (PRR-05) +- [ ] Reviewers are not the author and understand the area, and agents do not approve (PRR-06) +- [ ] Required checks pass before human review (PRR-07) +- [ ] Reviewers use the review focus for the change type (PRR-08) +- [ ] First response is within one working day and stale PRs are escalated (PRR-09) +- [ ] Comments are labelled and cite rule IDs (PRR-10) +- [ ] Blocking comments are resolved and stale approvals dismissed (PRR-11) +- [ ] Draft and stacked PRs are used where helpful, with documented order (PRR-12) +- [ ] PRs merge by squash and branches are deleted (PRR-13) +- [ ] PR links are kept as review evidence (PRR-14) +- [ ] Architecture review is not a routine PR requirement, and other PRs are reviewed by component owners (PRR-15) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Still use PRs, mainly for agent work and for reviewing your own diff. Self-review with the template checklist. Waive CODEOWNERS and turnaround rules, but keep 'CI must pass before merge' and 'small PRs'. + +### AI agents + +- Open a PR with a Conventional Commit title and the filled template: what and why, links, evidence, risk and checklist (PRR-02, PRR-03). +- Keep to one purpose per PR and roughly under 400 lines excluding generated files (PRR-04). +- Label your review comments blocking, suggestion, question or nit (PRR-10). +- Never approve or merge your own PR (PRR-06). +- Wait for required checks and fix failures instead of bypassing them (PRR-07). diff --git a/content/guidelines/quality-policy-and-attributes.md b/content/guidelines/quality-policy-and-attributes.md new file mode 100644 index 0000000..b27bcbc --- /dev/null +++ b/content/guidelines/quality-policy-and-attributes.md @@ -0,0 +1,97 @@ +--- +id: EG-003 +title: "Quality policy and quality attributes" +status: draft +owner: "Test and quality lead" +version: 0.1.0 +part: "Foundations" +related: [EG-006, EG-007, EG-017, EG-021] +review-by: 2027-09-30 +supersedes: +--- + +# EG-003: Quality policy and quality attributes + +- **Owner:** Test and quality lead +- **Applies to:** All products and all engineering activities +- **Enforcement:** Review, CI (budget and gate checks), release criteria +- **Related:** [EG-006](requirements-and-traceability.md), [EG-007](architecture-principles.md), [EG-017](testing-principles-and-strategy.md), [EG-021](release-update-and-production.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Define what quality means for our products so that requirements, architecture, tests and reviews aim at the same targets. Provide reusable non-functional requirement (NFR) templates so that quality attributes are made measurable rather than described in adjectives. + +## Rules + +1. **QUA-01** The team MUST maintain a quality policy of no more than one page, stating that products meet their requirements, are secure, can be diagnosed and updated in the field, and can be maintained for their supported life. +1. **QUA-02** The team MUST maintain a quality attribute model (below), adapted from ISO/IEC 25010, with definitions and example measures. +1. **QUA-03** Each product MUST have a quality attribute profile ranking its top attributes, approved by the architecture group and product owner. The profile guides trade-offs (see the precedence order in [EG-007](architecture-principles.md)). +1. **QUA-04** Quality attributes MUST be stated as measurable scenarios in requirements: environment, stimulus, response and measure. +1. **QUA-05** The team MUST maintain an NFR catalogue at `docs/requirements/nfr-catalogue.md` with reusable templates. Teams select and tailor entries rather than writing NFRs from scratch. +1. **QUA-06** Quality MUST be built in. There is no separate quality phase: the gates in [EG-016](pipelines-and-quality-gates.md) and the release criteria in [EG-021](release-update-and-production.md) are the quality controls. +1. **QUA-07** Each release MUST have quality evidence: gate results, requirement verification, budget margins, defect status and coverage. This is included in the release evidence pack ([EG-021](release-update-and-production.md)). +1. **QUA-08** Quality objectives and the attribute model SHOULD be reviewed at least annually, using field data ([EG-020](observability-and-telemetry.md)) and escaped defect analysis ([EG-018](defects-and-technical-debt.md)). + +## Quality attribute model + +| Attribute | Question it answers | Example measure | +|-----------|--------------------|-----------------| +| Functional correctness | Does it do what the requirements say? | Requirement verification pass rate | +| Performance and timing | Does it meet deadlines and throughput needs? | Worst-case execution time, latency percentiles | +| Reliability | Does it keep working over time and under faults? | Crash-free rate, soak hours without failure | +| Security | Is it protected against misuse and attack? | Open vulnerabilities by severity | +| Safety | Does it avoid harm when it fails? | Hazard mitigations verified | +| Resource efficiency | Does it stay within flash, RAM, CPU and power budgets? | Budget margin per component | +| Maintainability | Can it be changed safely and cheaply? | Change failure rate, review latency | +| Testability | Can behaviour be verified quickly and cheaply? | Share of logic testable on the host | +| Diagnosability | Can failures be understood in the field? | Share of field faults with a usable dump or log | +| Updatability | Can it be updated safely in the field? | Update success and rollback success rates | +| Interoperability | Does it work with other systems and revisions? | Compatibility matrix pass rate | +| Manufacturability | Can it be built, provisioned and tested at scale? | First-pass yield | + +## NFR scenario template + +```text +ID: NFR-PERF-012 +Attribute: Performance and timing +Environment: Normal operation, flash 80 percent full +Stimulus: A firmware image is received for update +Response: The image is validated and staged in the background +Measure: The 10 ms control task misses no deadlines across 1000 updates +Verification: Test (HIL, nightly) +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| QUA-04, QUA-05 | Requirement review checklist, NFR template check in CI | +| QUA-06 | Gate configuration review in [EG-016](pipelines-and-quality-gates.md) | +| QUA-07 | Release evidence pack completeness check | +| QUA-08 | Review-by date in front matter | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] A one-page quality policy is published (QUA-01) +- [ ] A quality attribute model exists with definitions and example measures (QUA-02) +- [ ] Each product has a ranked quality attribute profile, approved by the architecture lead and product owner (QUA-03) +- [ ] Quality attributes are stated as measurable scenarios in requirements (QUA-04) +- [ ] An NFR catalogue exists and requirements are selected from it (QUA-05) +- [ ] There is no separate quality phase, the gates are the quality controls (QUA-06) +- [ ] Each release has quality evidence in its evidence pack (QUA-07) +- [ ] The model and objectives are reviewed annually using field data (QUA-08) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Pick your top three quality attributes (for example correctness, simplicity, resource use) and write them in the README. Use the NFR scenario format only for requirements that matter. + +### AI agents + +- When a task involves a trade-off, use the project's ranked quality attributes and say which one you optimised (QUA-03). +- Write measurable acceptance criteria (environment, stimulus, response, measure) and not adjectives (QUA-04). diff --git a/content/guidelines/release-update-and-production.md b/content/guidelines/release-update-and-production.md new file mode 100644 index 0000000..85dee86 --- /dev/null +++ b/content/guidelines/release-update-and-production.md @@ -0,0 +1,98 @@ +--- +id: EG-021 +title: "Release, update and production" +status: draft +owner: "Release manager" +version: 0.1.0 +part: "Release, risk and compliance" +related: [EG-003, EG-014, EG-016, EG-017, EG-020, EG-022, EG-023, EG-010] +review-by: 2027-09-30 +supersedes: +--- + +# EG-021: Release, update and production + +- **Owner:** Release manager +- **Applies to:** All releases, field updates, production programming and supported versions +- **Enforcement:** Release pipeline, release criteria, sign-off +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-014](version-control-and-versioning.md), [EG-016](pipelines-and-quality-gates.md), [EG-017](testing-principles-and-strategy.md), [EG-020](observability-and-telemetry.md), [EG-022](security-and-data-protection.md), [EG-023](safety-regulatory-and-certification.md), [EG-010](components-reuse-and-variants.md) +- **Principles:** LP-01 to LP-10 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Know exactly what was shipped, release only what has passed the same gates every time, update devices safely and observably, and manufacture and support them for their full life. + +## Rules + +1. **REL-01** The release process MUST define stages (candidate, verification, approval, publication), roles (release manager, approvers) and a checklist held in the repository. +1. **REL-02** *(core)* A release candidate MUST be a tagged commit built once by the pipeline. Only that artefact is promoted (PIP-05). +1. **REL-03** Release criteria MUST include: all gates green, no open severity 1 or 2 defect without an approved waiver, complete requirement trace at the baseline, NFRs verified, budgets within limits, documentation built, SBOM and licence check clean, security review status, and regulatory evidence complete. +1. **REL-04** A release evidence pack MUST be generated automatically: trace report, test and analysis results, coverage, defect status, SBOM, signatures and approvals. It is retained for the supported life and any regulatory period. +1. **REL-05** *(core)* Release notes MUST be generated from conventional commits and closed defects, and MUST include breaking changes, upgrade notes and known issues. +1. **REL-06** A bill of materials MUST define each baseline: firmware version, hardware revisions, bootloader, toolchain digests, configuration schema, third-party versions and test suite version. +1. **REL-07** A compatibility matrix (firmware, hardware revision, bootloader, configuration, backend) MUST be maintained, with supported upgrade paths. +1. **REL-08** The update mechanism MUST be recorded in an ADR and be authenticated, integrity-checked, atomic (A/B or equivalent), power-fail safe and able to roll back. A downgrade protection policy MUST be defined. +1. **REL-09** Rollouts MUST be staged (canary, small cohort, wider) with health gates from telemetry (OBS-09) and the ability to pause or halt. +1. **REL-10** Every release MUST be tested for upgrade from all supported previous versions and for rollback (TST-13). +1. **REL-11** A hotfix process MUST be defined with expedited approval and the same mandatory gates, followed by a post-hoc review. +1. **REL-12** A support and end-of-life policy MUST list supported versions, security update duration and communication practice. +1. **REL-13** Production programming and provisioning MUST be controlled: the factory image is a released artefact, keys and certificates are injected through a controlled secure process ([EG-022](security-and-data-protection.md)), production test coverage is defined, and design-for-test features exist. +1. **REL-14** Manufacturing yield and failure data MUST be fed back to engineering. +1. **REL-15** Serial numbers MUST be traceable to firmware version, hardware revision and provisioning data. +1. **REL-16** Artefacts, symbols and toolchains MUST be retained so that any supported version can be rebuilt and diagnosed for its supported life. +1. **REL-17** Each release MUST have a post-release review of field results over a defined period, feeding [EG-024](planning-delivery-and-improvement.md). + +## Release flow + +```mermaid +flowchart LR + A[Tagged candidate] --> B[Gates and evidence] + B --> C[Approval] + C --> D[Staged rollout] + D --> E[Monitor field health] + E --> F[Post-release review] +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| REL-02, REL-04 | Release pipeline, evidence pack completeness check | +| REL-03 | Release checklist and sign-off record | +| REL-06, REL-07 | Bill of materials and matrix checks in CI | +| REL-09 | Rollout tooling and telemetry health gates | +| REL-13 | Production process audit | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Release stages, roles and a checklist are defined in the repository (REL-01) +- [ ] The candidate is a tagged commit built once, and that artefact is promoted (REL-02) +- [ ] Release criteria are met: gates, no open severity 1 or 2, trace, NFRs, budgets, docs, SBOM, security, regulatory (REL-03) +- [ ] The evidence pack is generated automatically and retained (REL-04) +- [ ] Release notes are generated from commits and defects (REL-05) +- [ ] A bill of materials defines each baseline (REL-06) +- [ ] A compatibility matrix and supported upgrade paths are maintained (REL-07) +- [ ] The update mechanism is recorded in an ADR: authenticated, atomic, power-fail safe, with rollback and a downgrade policy (REL-08) +- [ ] Rollouts are staged with health gates and the ability to halt (REL-09) +- [ ] Upgrade and rollback are tested from supported versions (REL-10) +- [ ] A hotfix process is defined (REL-11) +- [ ] A support and end-of-life policy is published (REL-12) +- [ ] Production programming and provisioning are controlled, with feedback, unit traceability and secure key injection (REL-13, REL-14, REL-15) +- [ ] Artefacts, symbols and toolchains are retained for the supported life (REL-16) +- [ ] A post-release review is held (REL-17) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Tag, build once, generate the changelog from commits and publish. Keep a one-line release checklist and write down which versions you support. Skip staged rollout and production provisioning unless you ship devices. + +### AI agents + +- Never create or move release tags, edit changelogs by hand or change version strings unless the task says so (VCS-10, VCS-11). +- Do not rebuild artefacts for release (REL-02). +- Put upgrade notes and breaking changes in commit footers so that release notes generate correctly (REL-05). diff --git a/content/guidelines/requirements-and-traceability.md b/content/guidelines/requirements-and-traceability.md new file mode 100644 index 0000000..790c970 --- /dev/null +++ b/content/guidelines/requirements-and-traceability.md @@ -0,0 +1,107 @@ +--- +id: EG-006 +title: "Requirements and traceability" +status: draft +owner: "Systems engineering lead" +version: 0.1.0 +part: "Requirements, architecture and design" +related: [EG-003, EG-008, EG-014, EG-017, EG-023] +review-by: 2027-09-30 +supersedes: +--- + +# EG-006: Requirements and traceability + +- **Owner:** Systems engineering lead +- **Applies to:** All product requirements at every level +- **Enforcement:** CI (trace report and gates), review +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-008](decision-records.md), [EG-014](version-control-and-versioning.md), [EG-017](testing-principles-and-strategy.md), [EG-023](safety-regulatory-and-certification.md) +- **Principles:** RP-01 to RP-08 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Ensure that everything we build traces to a need and everything we need is verified, with the evidence produced by the pipeline rather than assembled by hand. + +## Rules + +1. **REQ-01** Requirements MUST be decomposed through defined levels (stakeholder and system, subsystem, firmware and component). Each requirement is allocated to the next level down, and the allocation is recorded. +1. **REQ-02** A requirement MUST be atomic, unambiguous, feasible, necessary, consistent and verifiable. It MUST state one verification method: test, analysis, inspection or demonstration. +1. **REQ-03** *(core)* Requirements MUST be stored as text in the repository beside the design, with stable IDs of the form `REQ--`. IDs are never reused. +1. **REQ-04** Each requirement MUST carry a rationale, a parent, an owner and a status: `draft`, `approved`, `implemented`, `verified`, `retired`. Retired requirements are never deleted. +1. **REQ-05** Requirements MUST be baselined at each release by tag. Changes are made by PR, and history is kept in version control. +1. **REQ-06** *(core)* Requirement IDs MUST be referenced in commit footers, PR descriptions, test names and, where non-obvious, code comments and ADRs. +1. **REQ-07** Every approved requirement MUST be allocated to a design element, implemented and verified. Every test MUST trace to a requirement. Every significant code unit MUST trace to a requirement or an approved technical need. +1. **REQ-08** The pipeline MUST generate a trace report from requirement to design, code, test and result. Gaps produce warnings on `draft` requirements and failures on `approved` requirements at release. +1. **REQ-09** A change to a requirement MUST trigger a change impact check that flags dependent requirements, tests and documents. The PR MUST acknowledge each flagged item. +1. **REQ-10** A requirement MUST be reviewed by its author, an implementer and a verifier before it is approved. +1. **REQ-11** Non-functional requirements SHOULD be selected from the NFR catalogue ([EG-003](quality-policy-and-attributes.md)). +1. **REQ-12** Regulatory and security requirements MUST be captured as requirements with the same rules ([EG-022](security-and-data-protection.md), [EG-023](safety-regulatory-and-certification.md)). +1. **REQ-13** Work items MUST link to a requirement or a defect. Work with neither (tooling, refactoring) MUST state its justification. + +## Traceability chain + +```mermaid +flowchart LR + A[Stakeholder need] --> B[System requirement] + B --> C[Firmware requirement] + C --> D[Design and ADR] + D --> E[Code] + E --> F[Test] + F --> G[Test result] +``` + +## Requirement example + +```yaml +id: REQ-FW-0142 +title: Dual-bank image staging +statement: The firmware shall stage a received image in the inactive bank without interrupting normal operation. +type: functional +level: firmware +parent: REQ-SYS-0031 +verification: test +status: approved +owner: Update team +rationale: Avoids service interruption during updates. +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| REQ-03, REQ-04 | CI: requirement schema and ID uniqueness check | +| REQ-06 | CI: commit footer check, test naming check | +| REQ-07, REQ-08 | CI: trace report gate on release branches | +| REQ-09 | PR template item and change impact report | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Requirement levels and allocation rules are defined (REQ-01) +- [ ] Each requirement is atomic, unambiguous, feasible, necessary, consistent and verifiable, with one verification method (REQ-02) +- [ ] Requirements are stored as text in the repository with stable `REQ--` IDs (REQ-03) +- [ ] Each requirement has a rationale, parent, owner and status (REQ-04) +- [ ] Requirements are baselined by tag at each release (REQ-05) +- [ ] IDs are referenced in commit footers, PRs and test names (REQ-06) +- [ ] Every approved requirement is allocated, implemented and verified, and every test traces to a requirement (REQ-07) +- [ ] The pipeline generates a trace report with gap gates (REQ-08) +- [ ] A requirement change triggers a change impact check (REQ-09) +- [ ] Requirements are reviewed by author, implementer and verifier before approval (REQ-10) +- [ ] Non-functional requirements are selected from the NFR catalogue (REQ-11) +- [ ] Regulatory and security requirements are captured as requirements (REQ-12) +- [ ] Work items link to a requirement or defect, or state their justification (REQ-13) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Keep a short `docs/requirements/` list with IDs and a one-line verification each, and reference IDs with `Refs:` in commits. Skip formal levels and three-way review. For a small script, a 'what it must do' list in the README is enough. + +### AI agents + +- Before implementing, find the requirement ID or IDs the task serves and cite them in commits (`Refs: REQ-...`) and test names (REQ-06). +- If there is no requirement, say so and ask, or propose one for review (REQ-13). +- Never edit approved requirements without instruction, and flag the impact on tests and docs (REQ-09). diff --git a/content/guidelines/research-and-prototyping.md b/content/guidelines/research-and-prototyping.md new file mode 100644 index 0000000..10a8de7 --- /dev/null +++ b/content/guidelines/research-and-prototyping.md @@ -0,0 +1,99 @@ +--- +id: EG-025 +title: "Research, prototyping and experiments" +status: draft +owner: "Research lead" +version: 0.1.0 +part: "Delivery, research and people" +related: [EG-008, EG-011, EG-022, EG-024] +review-by: 2027-09-30 +supersedes: +--- + +# EG-025: Research, prototyping and experiments + +- **Owner:** Research lead +- **Applies to:** All exploratory work: spikes, prototypes, experiments and evaluations +- **Enforcement:** Repository structure, review, graduation checklist +- **Related:** [EG-008](decision-records.md), [EG-011](technology-and-third-party.md), [EG-022](security-and-data-protection.md), [EG-024](planning-delivery-and-improvement.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Give exploratory work room to move quickly without letting prototypes quietly become products, and make sure findings, including negative ones, are recorded and reproducible. + +## Rules + +1. **RSH-01** Work MUST be classified as exploratory or production in the work item. Exploratory work MUST have a question to answer and a time-box. +1. **RSH-02** Exploratory code MUST live in designated locations (`prototypes/`, a separate repository or `proto/` branches), be clearly marked non-production and be excluded from release builds and baselines. +1. **RSH-03** Exploratory work MAY be exempt from the coding standard details, requirement traceability and full gates. It MUST NOT be exempt from: no secrets in code, licence compliance ([EG-011](technology-and-third-party.md)), data classification ([EG-022](security-and-data-protection.md)), version control and a working build. +1. **RSH-04** Prototype code MUST NOT be promoted by copying it into the product. Graduation requires re-implementation or hardening against the production guidelines, following the graduation checklist (below). +1. **RSH-05** Each exploration MUST produce an experiment record in `docs/research/`: question, method, setup (versions, hardware, configuration), results, conclusion and next step. Negative results MUST be recorded. +1. **RSH-06** Experiments MUST be reproducible: scripts and configuration versioned, data locations recorded and the environment described. +1. **RSH-07** Each exploration MUST end with a decision: stop, extend (with reason) or graduate. Decisions affecting architecture group to an RFC or ADR ([EG-008](decision-records.md)). +1. **RSH-08** Maturity MUST be stated using the levels below, and the guidelines that apply increase with each level. +1. **RSH-09** Prototypes MUST be reviewed at the end of their time-box. Prototypes without an owner MUST be archived after 90 days. +1. **RSH-10** Evaluation boards, lab equipment and their configurations MUST be recorded with an owner. +1. **RSH-11** Findings SHOULD be shared with the team at the end of each exploration (demo or short write-up). +1. **RSH-12** AI agents MAY be used for prototyping under the same data rules. Prototype status does not relax AGT-05. + +## Maturity levels + +| Level | Name | Rules that apply | Exit needs | +|-------|------|------------------|------------| +| R0 | Idea | None | A question and a sponsor | +| R1 | Feasibility spike | RSH-03 minimum | Experiment record and decision | +| R2 | Prototype | Adds coding basics, docs in repository, review | Demonstrated against stated criteria | +| R3 | Pre-production candidate | Adds requirements, architecture and ADRs, coding standard, tests, security review | Graduation checklist passed | +| R4 | Product | All guidelines | Normal release process ([EG-021](release-update-and-production.md)) | + +## Graduation checklist + +1. Requirements written and traced ([EG-006](requirements-and-traceability.md)) +1. Architecture reviewed against principles, ADRs recorded ([EG-007](architecture-principles.md), [EG-008](decision-records.md)) +1. Code conforms to the coding standard and is reviewed ([EG-013](coding-standard.md), [EG-015](pull-requests-and-review.md)) +1. Tests at required levels, and pipeline gates passing ([EG-016](pipelines-and-quality-gates.md), [EG-017](testing-principles-and-strategy.md)) +1. Security review and licence check completed ([EG-022](security-and-data-protection.md), [EG-011](technology-and-third-party.md)) +1. Diagnostics and observability designed in ([EG-019](logging-debug-and-diagnostics.md), [EG-020](observability-and-telemetry.md)) +1. Documentation as code ([EG-004](documentation-as-code.md)) + +## Compliance + +| Rule | Checked by | +|------|------------| +| RSH-02 | CI: release builds exclude prototype paths | +| RSH-04, RSH-08 | Graduation review by architecture group | +| RSH-05 | Experiment record template check in CI | +| RSH-09 | Quarterly prototype inventory review | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Work is classified as exploratory or production, with a question and a time-box (RSH-01) +- [ ] Exploratory code lives in designated locations and is excluded from release builds (RSH-02) +- [ ] Exemptions are understood and the non-negotiables are kept: no secrets, licences, data rules, version control, a working build (RSH-03) +- [ ] Prototype code is never copied into the product and the graduation checklist is used (RSH-04) +- [ ] An experiment record captures question, method, setup, results, conclusion and next step (RSH-05) +- [ ] Experiments are reproducible (RSH-06) +- [ ] Each exploration ends with a decision to stop, extend or graduate (RSH-07) +- [ ] The maturity level is stated (RSH-08) +- [ ] Prototypes are reviewed at the end of the time-box and unowned ones archived after 90 days (RSH-09) +- [ ] Lab equipment and boards are recorded with an owner (RSH-10) +- [ ] Findings are shared with the team (RSH-11) +- [ ] Agent use in prototypes follows the same boundaries (RSH-12) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). This is where personal projects live. Keep a `prototypes/` folder, one experiment record per spike, and a firm stop, extend or graduate decision. Never copy prototype code into the product: re-implement it against the graduation checklist. + +### AI agents + +- State the question and time-box before starting a spike (RSH-01). +- Put spike code only under `prototypes/` or a `proto/` branch (RSH-02). +- Record results, including negative ones, in `docs/research/` using the experiment record template (RSH-05). +- Do not promote prototype code into the product by copying it, propose a graduation task instead (RSH-04). +- Prototype status does not relax agent boundaries (RSH-12). diff --git a/content/guidelines/safety-regulatory-and-certification.md b/content/guidelines/safety-regulatory-and-certification.md new file mode 100644 index 0000000..3c8821e --- /dev/null +++ b/content/guidelines/safety-regulatory-and-certification.md @@ -0,0 +1,86 @@ +--- +id: EG-023 +title: "Safety, regulatory and certification" +status: draft +owner: "Regulatory lead" +version: 0.1.0 +part: "Release, risk and compliance" +related: [EG-003, EG-006, EG-021, EG-022] +review-by: 2027-09-30 +supersedes: +--- + +# EG-023: Safety, regulatory and certification + +- **Owner:** Regulatory lead +- **Applies to:** All products and changes subject to safety, legal or certification requirements +- **Enforcement:** Change impact assessment, release criteria, audits +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-006](requirements-and-traceability.md), [EG-021](release-update-and-production.md), [EG-022](security-and-data-protection.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Make compliance a by-product of the way we work: obligations known and traced, evidence produced by the pipeline, and changes assessed for their effect on certified baselines. + +## Rules + +1. **REG-01** Applicable standards and regulations MUST be identified per product and market and recorded in a compliance register with an owner. This includes, where relevant, safety, legal metrology, radio, electromagnetic compatibility and cybersecurity obligations. +1. **REG-02** The compliance register MUST map each obligation to requirements, evidence, owner and status. +1. **REG-03** Regulatory obligations MUST be captured as requirements with verification methods and traced like any other ([EG-006](requirements-and-traceability.md)). +1. **REG-04** Evidence MUST be produced as a by-product of the process and stored in the release evidence pack (REL-04). It MUST NOT be assembled by hand at the end. +1. **REG-05** Every change touching a certified or regulated item MUST have a change impact assessment classifying it as no impact, notify or re-certify. Regulated components MUST be identified in CODEOWNERS, and the PR template asks the question. +1. **REG-06** The certified baseline (firmware and hardware versions) MUST be recorded. Changes outside it trigger assessment. Where applicable, regulated software SHOULD be separated architecturally from the rest. +1. **REG-07** Where a safety integrity level applies, the safety lifecycle of the chosen standard MUST be followed: hazard analysis, safety requirements, integrity level allocation, independence of verification and required coverage criteria. The chosen standard is recorded in an ADR. +1. **REG-08** Tools used for certified builds MUST be qualified or validated to the level required, and the toolchain frozen for the certified release. +1. **REG-09** Records MUST be retained and retrievable within an agreed time. Internal audits MUST be held at least annually against these guidelines and applicable standards, and findings tracked as defects. +1. **REG-10** Contact with notified bodies and test houses MUST be owned by a named person. Setup images and tools supplied for test are versioned and released like any other artefact. +1. **REG-11** Upcoming regulatory changes SHOULD be reviewed at least quarterly, with impact notes fed to planning. +1. **REG-12** Technical files MUST be maintained as documentation as code ([EG-004](documentation-as-code.md)) and versioned with the release. +1. **REG-13** Team members working on regulated areas MUST have the relevant compliance training recorded ([EG-026](competence-tooling-and-onboarding.md)). + +## Change impact classes + +| Class | Meaning | Action | +|-------|---------|--------| +| No impact | Outside certified scope, does not affect certified behaviour | Record rationale in the PR | +| Notify | Within scope but does not change certified behaviour | Notify the regulatory lead, record in evidence pack | +| Re-certify | Changes certified behaviour or scope | Plan re-assessment before release | + +## Compliance + +| Rule | Checked by | +|------|------------| +| REG-02, REG-03 | Compliance register audit, trace report | +| REG-04 | Release evidence pack completeness check | +| REG-05, REG-06 | CODEOWNERS on regulated paths, PR template, review | +| REG-09 | Annual internal audit | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Applicable standards and regulations are identified in a compliance register (REG-01) +- [ ] The register maps obligations to requirements, evidence, owner and status (REG-02) +- [ ] Regulatory obligations are captured as requirements and traced (REG-03) +- [ ] Evidence is produced by the pipeline into the evidence pack (REG-04) +- [ ] Changes to certified items have a change impact assessment (REG-05) +- [ ] The certified baseline is recorded and regulated software is separated where applicable (REG-06) +- [ ] The safety lifecycle is followed where an integrity level applies (REG-07) +- [ ] Tools for certified builds are qualified or validated and frozen (REG-08) +- [ ] Records are retained, audits held annually and findings tracked (REG-09) +- [ ] Test house and notified body contact is owned by a named person (REG-10) +- [ ] Regulatory changes are reviewed quarterly (REG-11) +- [ ] Technical files are maintained as docs as code (REG-12) +- [ ] Compliance training is recorded (REG-13) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Full** profile (see [personal-projects.md](personal-projects.md)). Adopt only if the project is subject to a regulation or safety standard (for example radio, mains-connected, medical, or sold as a product). Otherwise record 'not applicable' in ADR-0001. + +### AI agents + +- Do not modify code, data or configuration in certified or regulated areas unless the task says so, and flag the change impact class in the PR (REG-05). +- Do not change tools or toolchain versions for certified builds (REG-08). diff --git a/content/guidelines/security-and-data-protection.md b/content/guidelines/security-and-data-protection.md new file mode 100644 index 0000000..575227b --- /dev/null +++ b/content/guidelines/security-and-data-protection.md @@ -0,0 +1,106 @@ +--- +id: EG-022 +title: "Security and data protection" +status: draft +owner: "Security lead" +version: 0.1.0 +part: "Release, risk and compliance" +related: [EG-005, EG-011, EG-016, EG-019, EG-020, EG-021] +review-by: 2027-09-30 +supersedes: +--- + +# EG-022: Security and data protection + +- **Owner:** Security lead +- **Applies to:** All products, tooling, pipelines, data and AI tool use +- **Enforcement:** CI, review, security testing, audits +- **Related:** [EG-005](ai-agent-usage.md), [EG-011](technology-and-third-party.md), [EG-016](pipelines-and-quality-gates.md), [EG-019](logging-debug-and-diagnostics.md), [EG-020](observability-and-telemetry.md), [EG-021](release-update-and-production.md) +- **Principles:** SP-01 to SP-12 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Build security and privacy into products and into the way we build them, from threat model to key handling to vulnerability response. + +## Rules + +1. **SEC-01** Each product MUST have security requirements and a threat model, created at design time and updated on every architecture-significant change, using an agreed method and stored in the repository. +1. **SEC-02** The security architecture MUST address trust boundaries, secure boot with a chain of trust, authenticated and encrypted communication, secure update, protected storage, debug port protection, least privilege and memory protection where the hardware allows. +1. **SEC-03** *(core)* Only approved cryptographic algorithms and vetted libraries MUST be used, from a list owned by the security lead. Home-grown cryptography MUST NOT be used. +1. **SEC-04** Key management MUST be documented, with a key hierarchy, protected signing (HSM or protected service), controlled provisioning, and rotation and revocation plans. Private keys MUST NOT be in repositories or pipeline logs. +1. **SEC-05** *(core)* Secrets MUST NOT be in source. Secret scanning MUST run pre-commit and in CI. A leaked secret is an incident and MUST be rotated. +1. **SEC-06** Secure coding rules (CP-17), security-focused review of security-relevant code and static analysis with security rule sets MUST be applied. Parsers and protocol handlers SHOULD be fuzzed. +1. **SEC-07** Security testing (fuzzing, penetration testing before major releases, security regression tests) MUST be performed and its findings tracked as defects. +1. **SEC-08** *(core)* Vulnerability management MUST have an intake channel, a coordinated disclosure policy, severity assessment and patch times by severity (below). The SBOM MUST be monitored against vulnerability feeds. +1. **SEC-09** Supply chain controls MUST include signed artefacts, provenance, pinned dependencies ([EG-011](technology-and-third-party.md), [EG-016](pipelines-and-quality-gates.md)), least-privilege pipeline credentials and a two-person rule for release signing. +1. **SEC-10** *(core)* A data classification scheme MUST be in place (below), with handling rules for storage, logs, telemetry, sharing and AI tools. +1. **SEC-11** Products and telemetry MUST follow privacy by design: data minimisation, purpose limitation, defined retention and an impact assessment where personal data is processed (OBS-05, LOG-08). +1. **SEC-12** Access to repositories, pipelines, rigs and signing MUST be least-privilege, use multi-factor authentication and be reviewed quarterly. +1. **SEC-13** Debug and manufacturing interfaces MUST be disabled or protected in production (DBG-02, REL-13). +1. **SEC-14** A security incident procedure MUST define roles, communication and post-incident review (PLN-11). + +## Data classification + +| Class | Description | Handling | AI tools permitted | +|-------|-------------|----------|--------------------| +| Public | Published material | No restriction | Any approved tool | +| Internal | General engineering material | Company systems only | Approved tools only | +| Confidential | Source code, designs, customer or commercial information | Access controlled, encrypted in transit and at rest | Only tools approved for confidential data | +| Restricted | Keys, credentials, personal data, export-controlled material | Minimum access, no copies, audit trail | None unless explicitly approved | + +## Patch times by severity + +Proposed starting points, to be confirmed through an RFC. + +| Severity | Assessment | Fix or mitigation available within | +|----------|------------|-------------------------------------| +| Critical | Actively exploited or trivially exploitable | 14 days | +| High | Serious impact, reasonable exploitability | 30 days | +| Medium | Limited impact or difficult to exploit | 90 days | +| Low | Minimal impact | Next planned release | + +## Compliance + +| Rule | Checked by | +|------|------------| +| SEC-05 | CI and pre-commit secret scanning | +| SEC-06, SEC-07 | Analyser rule sets, fuzzing jobs, penetration test reports | +| SEC-08 | SBOM monitoring reports, vulnerability register | +| SEC-09, SEC-12 | Pipeline and access configuration audit | +| SEC-10 | Classification review of repositories, tools and telemetry | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Security requirements and a threat model exist and are updated on architecture change (SEC-01) +- [ ] The security architecture covers trust boundaries, secure boot, encrypted comms, secure update, protected storage and debug protection (SEC-02) +- [ ] Only approved cryptography and vetted libraries are used (SEC-03) +- [ ] Key hierarchy, protected signing, provisioning, rotation and revocation are documented (SEC-04) +- [ ] No secrets are in source, and scanning runs pre-commit and in CI (SEC-05) +- [ ] Secure coding rules and security-focused review apply, and parsers are fuzzed (SEC-06) +- [ ] Security testing is performed and findings are tracked (SEC-07) +- [ ] Vulnerability intake, disclosure policy, patch times and SBOM monitoring exist (SEC-08) +- [ ] Supply chain controls are in place (SEC-09) +- [ ] A data classification scheme has handling rules, including for AI tools (SEC-10) +- [ ] Privacy by design, retention and impact assessment are applied where needed (SEC-11) +- [ ] Access is least-privilege with multi-factor authentication and quarterly review (SEC-12) +- [ ] Debug and manufacturing interfaces are protected in production (SEC-13) +- [ ] A security incident procedure is defined (SEC-14) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Starter** profile (see [personal-projects.md](personal-projects.md)). Non-negotiable even solo: secret scanning, multi-factor authentication, dependency updates and a written list of what data may go to which AI tool. Do a one-page threat model only if the project is internet-facing or handles other people's data. + +### AI agents + +- Never write secrets, keys or tokens into code, config, logs, commits or prompts (SEC-05). +- If you find one, stop and report it. +- Never send Confidential or Restricted data to an unapproved tool (AGT-08, SEC-10). +- Do not use home-grown cryptography (SEC-03). +- Do not touch security-relevant code, authentication, key handling or debug protection without explicit instruction (AGT-05). +- Validate all input (CP-17). diff --git a/content/guidelines/technology-and-third-party.md b/content/guidelines/technology-and-third-party.md new file mode 100644 index 0000000..0594ef0 --- /dev/null +++ b/content/guidelines/technology-and-third-party.md @@ -0,0 +1,89 @@ +--- +id: EG-011 +title: "Technology selection and third-party components" +status: draft +owner: "Architecture group" +version: 0.1.0 +part: "Requirements, architecture and design" +related: [EG-005, EG-016, EG-022, EG-010] +review-by: 2027-09-30 +supersedes: +--- + +# EG-011: Technology selection and third-party components + +- **Owner:** Architecture group +- **Applies to:** All tools, platforms, libraries, vendor SDKs and open-source components +- **Enforcement:** RFC and ADR review, CI (manifest, licence and checksum checks) +- **Related:** [EG-005](ai-agent-usage.md), [EG-016](pipelines-and-quality-gates.md), [EG-022](security-and-data-protection.md), [EG-010](components-reuse-and-variants.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Keep technology choices consistent, deliberate and reversible, and keep every third-party component known, licensed, pinned and maintained. + +## Rules + +1. **TEC-01** The team MUST maintain a technology radar in the repository, classifying languages, tools, libraries, platforms and RTOSes as `adopt`, `trial`, `assess` or `hold`. Changes are made through an RFC. +1. **TEC-02** Adopting a new dependency or tool MUST be supported by an evaluation recorded in an ADR, covering fit, maturity, licence, security posture, support and longevity, footprint and cost of exit. +1. **TEC-03** *(core)* A licence policy MUST list approved licences and prohibited ones. Any other licence needs legal review before use. +1. **TEC-04** *(core)* All third-party components MUST be recorded in a manifest (name, version, source, licence, checksum, owner). An SBOM MUST be generated for each release. +1. **TEC-05** *(core)* Versions MUST be pinned and checksums verified. Floating versions and unpinned downloads in builds MUST NOT be used. +1. **TEC-06** Vendor SDKs and libraries MUST be wrapped behind interfaces we own (AP-03). Patches to vendored code MUST be kept as a patch set with rationale and upstream status. Silent forks MUST NOT exist. +1. **TEC-07** Dependency updates SHOULD be proposed automatically (for example Renovate or Dependabot). Security updates follow the patch times in [EG-022](security-and-data-protection.md), and other updates are reviewed at least quarterly. +1. **TEC-08** Each dependency MUST have an owner. Critical dependencies SHOULD have an exit strategy recorded in an ADR. +1. **TEC-09** AI-generated code MUST meet the same licence and provenance controls. Known third-party code MUST NOT be pasted in, and licence scanning SHOULD be used where tooling exists. +1. **TEC-10** Compiler, analyser and build tool versions MUST be pinned in the containerised toolchain ([EG-016](pipelines-and-quality-gates.md)) and listed in the radar. +1. **TEC-11** End-of-life dates for dependencies and tools MUST be monitored, and replacement planned before support ends. + +## Evaluation criteria + +| Criterion | Questions | +|-----------|-----------| +| Fit | Does it meet the requirements and quality attributes? | +| Maturity | Is it proven on similar targets, and how active is the project? | +| Licence | Is the licence approved, and are the obligations manageable? | +| Security | Track record, disclosure process, patch speed | +| Support | Vendor or community support, expected lifetime | +| Footprint | Flash, RAM and CPU cost against budgets | +| Exit | How hard is it to replace, and is it behind our interface? | + +## Compliance + +| Rule | Checked by | +|------|------------| +| TEC-03, TEC-04 | CI: licence scan and manifest completeness check | +| TEC-05 | CI: checksum verification, ban on unpinned dependencies | +| TEC-06 | Review, dependency direction check | +| TEC-07 | Automated update tooling reports | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] A technology radar exists with adopt, trial, assess and hold (TEC-01) +- [ ] New dependencies and tools have an evaluation recorded in an ADR (TEC-02) +- [ ] A licence policy lists approved and prohibited licences (TEC-03) +- [ ] A manifest of third-party components exists and an SBOM is produced per release (TEC-04) +- [ ] Versions are pinned and checksums verified (TEC-05) +- [ ] Vendor SDKs are wrapped behind our interfaces and patches are tracked (TEC-06) +- [ ] Dependency updates are proposed automatically, with security patch times (TEC-07) +- [ ] Each dependency has an owner, and critical ones have an exit strategy (TEC-08) +- [ ] AI-generated code is checked for licence and provenance (TEC-09) +- [ ] Toolchain versions are pinned in containers and listed in the radar (TEC-10) +- [ ] End-of-life dates are monitored (TEC-11) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Keep pinning, checksums, a dependency list with licences, and automated update PRs. The radar can be a table in the README. + +### AI agents + +- Do not add a dependency or tool without asking or drafting an ADR (TEC-02). +- Pin exact versions and update the manifest (TEC-04, TEC-05). +- Check licences against the approved list (TEC-03). +- Do not paste code of unknown provenance (TEC-09). +- Wrap vendor code behind an interface (TEC-06) and never edit vendored code directly. diff --git a/content/guidelines/testing-principles-and-strategy.md b/content/guidelines/testing-principles-and-strategy.md new file mode 100644 index 0000000..34ee409 --- /dev/null +++ b/content/guidelines/testing-principles-and-strategy.md @@ -0,0 +1,115 @@ +--- +id: EG-017 +title: "Testing principles and strategy" +status: draft +owner: "Test and quality lead" +version: 0.1.0 +part: "Verification" +related: [EG-003, EG-006, EG-007, EG-016, EG-018, EG-021] +review-by: 2027-09-30 +supersedes: +--- + +# EG-017: Testing principles and strategy + +- **Owner:** Test and quality lead +- **Applies to:** All verification activity, environments and test code +- **Enforcement:** CI, review, release criteria +- **Related:** [EG-003](quality-policy-and-attributes.md), [EG-006](requirements-and-traceability.md), [EG-007](architecture-principles.md), [EG-016](pipelines-and-quality-gates.md), [EG-018](defects-and-technical-debt.md), [EG-021](release-update-and-production.md) +- **Principles:** TP-01 to TP-11 (defined below) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Set the principles that guide how we test, require a test strategy for each product, and manage test environments (simulators and HIL rigs) as products in their own right. + +## Testing principles + +| ID | Principle | Statement | +|----|-----------|-----------| +| TP-01 | Cheapest level first | Test at the cheapest level that can catch the defect. Most tests run on the host. | +| TP-02 | Trace to requirements | Every test verifies a requirement, and every requirement has verification. | +| TP-03 | Deterministic and independent | Tests give the same result every run, in any order. | +| TP-04 | Behaviour over implementation | Tests check what the unit promises, not how it does it. | +| TP-05 | Fail clearly | A failing test says what went wrong and where. | +| TP-06 | Automate what repeats | Anything run more than twice is automated. | +| TP-07 | Tests are code | Tests are reviewed, refactored and maintained to the same standard. | +| TP-08 | Bugs start with a test | A defect is reproduced by a failing test before it is fixed. | +| TP-09 | Test what can go wrong | Negative, boundary and fault-injection cases are as important as the happy path. | +| TP-10 | Hardware for what only hardware shows | Timing, power, peripherals and environment need real hardware. | +| TP-11 | Controlled environments | The environment for every test run is known and recorded. | + +## Rules + +1. **TST-01** Each product MUST have a test strategy covering levels, entry and exit criteria, tools, environments, responsibilities and reporting. +1. **TST-02** Testing MUST be organised in levels (below), weighted towards host-based tests. +1. **TST-03** *(core)* Test names or metadata MUST include the requirement IDs they verify (REQ-06). The verification method recorded for each requirement MUST be executed. +1. **TST-04** Coverage MUST be measured on host tests with thresholds set per component criticality. Coverage MUST NOT decrease (ratchet). Requirement coverage is reported alongside code coverage, and stronger criteria (for example MC/DC) apply where a standard requires them. +1. **TST-05** Simulators, emulators and test doubles MUST be maintained and periodically validated against real hardware. +1. **TST-06** Negative tests and fault injection MUST be included for parsers, communication stacks, update, storage and security functions. +1. **TST-07** *(core)* Every fixed defect MUST have a regression test (DEF-04). The regression suite MUST run on every merge to `main`. +1. **TST-08** Test data and fixtures MUST be versioned with the tests. +1. **TST-09** HIL rigs and shared environments MUST be managed as products: named owner, configuration record, booking, calibration, health checks, spares and documented setup as code where possible. +1. **TST-10** Results MUST be archived per release and linked to requirements ([EG-021](release-update-and-production.md)). +1. **TST-11** Non-functional tests (soak, power profile, size and timing trends, environmental) SHOULD run continuously or nightly, not only before release. +1. **TST-12** Verification of high-risk requirements MUST be performed or reviewed by someone other than the author. +1. **TST-13** Every release MUST be tested for upgrade from all supported previous versions and for rollback ([EG-021](release-update-and-production.md)). +1. **TST-14** Exploratory testing SHOULD be planned and time-boxed, and findings recorded as defects or new tests. +1. **TST-15** Flaky tests are handled under PIP-09. Skipping or deleting a failing test MUST NOT be used to make a gate pass. + +## Test levels + +| Level | Runs on | Purpose | +|-------|---------|---------| +| Unit | Host | Logic of one unit, fast, deterministic | +| Component and integration | Host or simulator | Interactions between units and with doubles | +| HIL integration | Hardware rig | Real peripherals, timing, drivers | +| System | Hardware rig or product | End-to-end behaviour against system requirements | +| Non-functional | Rig or lab | Performance, power, soak, environmental, resilience | +| Security | Rig or lab | Fuzzing, penetration and security regression tests | +| Update and compatibility | Rig | Upgrade, downgrade and rollback across supported versions | +| Production test | Factory | Manufacturing defects and provisioning correctness | + +## Compliance + +| Rule | Checked by | +|------|------------| +| TST-03 | CI: test metadata check and trace report | +| TST-04 | CI: coverage gate and ratchet | +| TST-07, TST-13 | Pipeline configuration, release criteria | +| TST-09 | Rig register audit | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] A test strategy exists per product with levels, entry and exit criteria, tools and environments (TST-01) +- [ ] Testing is weighted towards host-level tests (TST-02) +- [ ] Test names or metadata carry requirement IDs (TST-03) +- [ ] Coverage is measured with thresholds and a ratchet, and requirement coverage is reported (TST-04) +- [ ] Simulators and test doubles are maintained and validated against hardware (TST-05) +- [ ] Negative tests and fault injection cover parsers, comms, update, storage and security (TST-06) +- [ ] Every fixed defect has a regression test and the suite runs on every merge to `main` (TST-07) +- [ ] Test data and fixtures are versioned (TST-08) +- [ ] Rigs are managed as products (TST-09) +- [ ] Results are archived per release and linked to requirements (TST-10) +- [ ] Non-functional tests run nightly or continuously (TST-11) +- [ ] Verification of high-risk requirements is independent of the author (TST-12) +- [ ] Upgrade and rollback are tested from all supported versions (TST-13) +- [ ] Exploratory testing is planned and time-boxed, with findings recorded (TST-14) +- [ ] No failing test is skipped or deleted to pass a gate (TST-15) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Standard** profile (see [personal-projects.md](personal-projects.md)). Write unit tests on the host for logic, a regression test for every bug, and a smoke test for the whole thing. A coverage ratchet is cheap and effective. Skip rig management unless you own a rig. + +### AI agents + +- Write a failing test that reproduces a bug before fixing it (TP-08). +- Add tests in the same change as the code, named with requirement IDs (TST-03). +- Test behaviour and not implementation, and include negative and boundary cases (TP-04, TP-09). +- Never skip, delete or weaken a test to make a gate pass (TST-15). +- Check that generated tests verify the requirement and would fail if the code were wrong (AGT-11). diff --git a/content/guidelines/values-and-ways-of-working.md b/content/guidelines/values-and-ways-of-working.md new file mode 100644 index 0000000..8115b40 --- /dev/null +++ b/content/guidelines/values-and-ways-of-working.md @@ -0,0 +1,106 @@ +--- +id: EG-002 +title: "Engineering values and ways of working" +status: draft +owner: "Engineering manager" +version: 0.1.0 +part: "Foundations" +related: [EG-008, EG-024, EG-026] +review-by: 2027-09-30 +supersedes: +--- + +# EG-002: Engineering values and ways of working + +- **Owner:** Engineering manager +- **Applies to:** All team members and sites +- **Enforcement:** Review, retrospectives, ownership register checks +- **Related:** [EG-008](decision-records.md), [EG-024](planning-delivery-and-improvement.md), [EG-026](competence-tooling-and-onboarding.md) +- **Principles:** EV-01 to EV-08 (see [principles.md](principles-catalogue.md)) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +State the values that tie the other guidelines together, and set out how the team makes decisions, resolves disagreements, and works across sites and time zones. + +## Values + +| ID | Value | Meaning in practice | +|----|-------|---------------------| +| EV-01 | Ownership | Every asset has a named owner accountable for its quality and future | +| EV-02 | Quality built in | Quality is produced by the way we work, not inspected in at the end | +| EV-03 | Written and open by default | Decisions, agreements and handovers are recorded where others can find them | +| EV-04 | Evidence over opinion | Claims about performance, defects and risk are backed by data | +| EV-05 | Blameless learning | We fix systems and processes, not people | +| EV-06 | Small steps | Small, frequent, reversible changes beat large, rare ones | +| EV-07 | Simplicity | The simplest solution that meets the need is preferred | +| EV-08 | Respect for others' time | Cross-site work is designed for time zones, and requests are clear and complete | + +## Rules + +1. **VAL-01** The values above SHOULD be used as tie-breakers when guidelines are silent and principles do not settle a question. +1. **VAL-02** Every component, guideline, pipeline, test rig and tool MUST have a named owner in CODEOWNERS or the ownership register. Unowned assets are reported monthly. +1. **VAL-03** Decision rights MUST be recorded in the decision rights table (below). The accountable person decides after consulting those affected. +1. **VAL-04** Disagreements escalate in steps: discuss directly, write it up as an RFC or ADR, sponsor decides, then architecture group or engineering manager. Each step has a time limit of five working days. +1. **VAL-05** After a decision, the team MUST disagree and commit. Reopening a decision needs new evidence, recorded in a new ADR. +1. **VAL-06** Decisions, agreements and handovers MUST be written down in the repository or tracker. Outcomes of meetings MUST be recorded within one working day. +1. **VAL-07** Every component MUST have one owning site and a named deputy at a second site, so that no knowledge or authority is site-exclusive. +1. **VAL-08** Cross-site working agreements MUST define core overlap hours, expected response time (acknowledge within one working day), and a handover note for work that passes between sites. +1. **VAL-09** All sites use the same tools, conventions and pipelines. A site-specific variant needs an ADR. +1. **VAL-10** Work status MUST be visible in the tracker and pipeline, so that status is read, not requested. +1. **VAL-11** Changes SHOULD be small enough to be reviewed within one working day. +1. **VAL-12** Raising a risk, a disagreement or a mistake MUST be treated as expected behaviour and never penalised. + +## Decision rights + +| Decision | Decides | Must consult | +|----------|---------|--------------| +| Component design within its boundary | Component owner | Architecture group | +| Interface between components or sites | Architecture group | Both component owners | +| Change to a guideline | Guideline owner, through an RFC | Affected teams | +| Adoption of a tool or dependency | Architecture group, through an RFC | Build and tooling lead, security lead | +| Release go or no-go | Release manager | Test and quality lead, security lead | +| Exception to a security rule | Security lead | Architecture group | +| Priority between work items | Product owner | Engineering manager | + +## Handover note + +A handover note between sites states what was done, and what is in progress and where. It also says what is blocked and by whom, what needs a decision, and where the current state lives (branch, PR, ticket). + +## Compliance + +| Rule | Checked by | +|------|------------| +| VAL-02 | Ownership register and CODEOWNERS check in CI | +| VAL-04, VAL-05 | RFC and ADR history review | +| VAL-06, VAL-10 | Retrospective review, tracker audit | +| VAL-07, VAL-08 | Ownership register, quarterly cross-site review | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Values are published and used as tie-breakers (VAL-01) +- [ ] Every component, pipeline, rig and tool has a named owner in CODEOWNERS or the ownership register (VAL-02) +- [ ] The decision rights table is current (VAL-03) +- [ ] Escalation steps and time limits are agreed, and decisions are committed to once made (VAL-04, VAL-05) +- [ ] Decisions, agreements and meeting outcomes are written down within one working day (VAL-06) +- [ ] Each component has an owning site and a deputy at another site (VAL-07) +- [ ] Cross-site agreements cover overlap hours, response time and the handover note (VAL-08) +- [ ] All sites use the same tools and conventions, and variants have an ADR (VAL-09) +- [ ] Work status is visible in the tracker and pipeline (VAL-10) +- [ ] Changes are small enough to review within one working day (VAL-11) +- [ ] Raising a risk, disagreement or mistake is treated as expected behaviour (VAL-12) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Full** profile (see [personal-projects.md](personal-projects.md)). Keep written-by-default, small steps, simplicity and evidence over opinion. Decision rights collapse to you. Skip the cross-site rules. + +### AI agents + +- Prefer small, reviewable changes (VAL-11). +- Record decisions and assumptions in the PR or an ADR, not only in chat (VAL-06). +- Raise risks and doubts early and do not hide uncertainty (VAL-12). diff --git a/content/guidelines/version-control-and-versioning.md b/content/guidelines/version-control-and-versioning.md new file mode 100644 index 0000000..84f6957 --- /dev/null +++ b/content/guidelines/version-control-and-versioning.md @@ -0,0 +1,135 @@ +--- +id: EG-014 +title: "Version control, commits and versioning" +status: draft +owner: "Build and tooling lead" +version: 0.1.0 +part: "Code and build" +related: [EG-006, EG-008, EG-015, EG-016, EG-021, EG-026, EG-010] +review-by: 2027-09-30 +supersedes: +--- + +# EG-014: Version control, commits and versioning + +- **Owner:** Build and tooling lead +- **Applies to:** All repositories, branches, commits, tags and versioned artefacts +- **Enforcement:** Hooks, CI, branch protection +- **Related:** [EG-006](requirements-and-traceability.md), [EG-008](decision-records.md), [EG-015](pull-requests-and-review.md), [EG-016](pipelines-and-quality-gates.md), [EG-021](release-update-and-production.md), [EG-026](competence-tooling-and-onboarding.md), [EG-010](components-reuse-and-variants.md) + +> **Note:** Rules marked *(core)* apply to every product and cannot be tailored. Other rules, and all numeric values, are proposed defaults that a product may tailor in its conformance profile ([EG-001](guidelines-governance.md)). + +## Purpose + +Keep history clean, meaningful and machine-readable, so that version numbers, changelogs, release notes and trace reports are generated from it rather than written by hand. + +## Rules + +1. **VCS-01** *(core)* Git MUST be the single source of truth. The repository structure MUST be documented, and the choice of monorepo or multi-repo (with its multi-repo tool) recorded in an ADR. Large binaries MUST use LFS or an artefact store. Secrets and build outputs MUST NOT be committed. +1. **VCS-02** The branching model MUST be trunk-based: a protected `main` and short-lived branches (target under three days) named `feat/`, `fix/`, `docs/` or `chore/` with the ticket ID. Release branches are used only for maintenance of supported versions. +1. **VCS-03** *(core)* Commit messages MUST follow Conventional Commits, using the allowed types (below) and scopes mapped to components in a scope file. +1. **VCS-04** *(core)* A breaking change MUST be marked with `!` and a `BREAKING CHANGE:` footer. +1. **VCS-05** *(core)* Footers MUST include `Refs:` for requirement or ticket IDs. They SHOULD include `ADR:` where relevant, and MUST include `Assisted-by:` where AI assisted (AGT-06). +1. **VCS-06** Commits SHOULD be atomic and buildable, so that bisection works. +1. **VCS-07** *(core)* Commit format MUST be enforced by a commit-msg hook (installed by one command) and by a CI lint. +1. **VCS-08** *(core)* `main` MUST have linear history through squash or rebase merging. Force-push to shared branches MUST NOT be used. History rewriting is allowed only on your own unmerged branches. +1. **VCS-09** *(core)* Firmware versions MUST follow semantic versioning as defined below. +1. **VCS-10** *(core)* The version MUST have a single source, derived from the Git tag by the build and injected into the artefact. Version strings MUST NOT be edited by hand. Builds MUST embed the commit hash and a dirty flag, and the device MUST report its version. +1. **VCS-11** *(core)* Release tags MUST be annotated, signed, immutable, and named `vX.Y.Z` (pre-releases `vX.Y.Z-rc.N`). Tags trigger the release pipeline. +1. **VCS-12** Version bump, changelog and release notes MUST be generated from commits and closed defects by tooling (for example release-please or semantic-release). +1. **VCS-13** Fixes MUST be made on `main` first and back-ported to supported release branches (`release/X.Y`) by `git cherry-pick -x`. Supported versions are listed in the release policy ([EG-021](release-update-and-production.md)). +1. **VCS-14** Team members MUST understand the tool they use daily. Git competence expectations are in [EG-026](competence-tooling-and-onboarding.md). +1. **VCS-15** The commit subject MUST be in the imperative mood, under 72 characters, with no full stop (`fix(power): prevent deep sleep during an active session`, not `fixed power issue`). The body SHOULD say why the change was needed and what behaviour changed, not how the code does it. + +## Commit types + +| Type | Meaning | Version effect | +|------|---------|----------------| +| `feat` | New capability | MINOR | +| `fix` | Bug fix | PATCH | +| `perf` | Performance improvement | PATCH | +| `refactor` | Restructure without behaviour change | None | +| `docs` | Documentation only | None | +| `test` | Tests only | None | +| `build` | Build system or dependencies | None | +| `ci` | Pipeline changes | None | +| `chore` | Maintenance | None | +| `revert` | Reverts a commit | As reverted | +| any type with `!` | Breaking change | MAJOR | + +```text +feat(bootloader)!: add dual-bank image layout + +Requires update agent 2.0 or later to stage images. + +BREAKING CHANGE: flash map changed, single-bank images no longer boot. +Refs: REQ-SYS-0142, PROJ-1187 +ADR: ADR-0023 +Assisted-by: example-agent +``` + +## Semantic versioning for firmware + +| Level | Increment when | +|-------|----------------| +| MAJOR | Incompatible change to update protocol, flash layout, bootloader contract, configuration schema, wire protocol or public API | +| MINOR | Backwards-compatible new feature, configuration option or protocol extension | +| PATCH | Backwards-compatible fix or performance improvement | + +Hardware revision compatibility is recorded in the compatibility matrix ([EG-021](release-update-and-production.md)) and not encoded in the version number. + +## From commit to release + +```mermaid +flowchart LR + A[Conventional commits] --> B[Pipeline on main] + B --> C[Version bump and changelog] + C --> D[Signed tag] + D --> E[Release build and evidence] + E --> F[Artefacts and release notes] +``` + +## Compliance + +| Rule | Checked by | +|------|------------| +| VCS-03, VCS-04, VCS-05, VCS-07, VCS-15 | commit-msg hook and CI commit lint | +| VCS-02, VCS-08 | Branch protection and repository settings as code ([EG-016](pipelines-and-quality-gates.md)) | +| VCS-10, VCS-11, VCS-12 | Build system and release pipeline | +| VCS-13 | Cherry-pick footer check on release branches | + +## Checklist + +Tick an item when it is true for your project. Rule IDs point to the detail above. + +- [ ] Git is the single source of truth, the structure is documented, and no secrets or build outputs are committed (VCS-01) +- [ ] Branching is trunk-based with short-lived, correctly named branches (VCS-02) +- [ ] Commits follow Conventional Commits with scopes mapped to components (VCS-03) +- [ ] Breaking changes use `!` and a `BREAKING CHANGE:` footer (VCS-04) +- [ ] Footers carry `Refs:`, `ADR:` and `Assisted-by:` where relevant (VCS-05) +- [ ] Commits are atomic and buildable (VCS-06) +- [ ] A commit-msg hook and a CI lint enforce the format (VCS-07) +- [ ] `main` has linear history and shared branches are never force-pushed (VCS-08) +- [ ] Semantic versioning is defined for the product (VCS-09) +- [ ] The version comes from the tag, is injected by the build and is never hand-edited (VCS-10) +- [ ] Release tags are annotated, signed and immutable (VCS-11) +- [ ] Version bump, changelog and release notes are generated from commits (VCS-12) +- [ ] Back-ports use `cherry-pick -x` onto supported release branches (VCS-13) +- [ ] The team understands git well enough to use it safely (VCS-14) +- [ ] Commit subjects are imperative and short, and bodies say why (VCS-15) + +## Applying to personal projects and AI agents + +### Personal projects + +Starts at the **Starter** profile (see [personal-projects.md](personal-projects.md)). Use Conventional Commits from day one, because they drive your version and changelog. Signed tags are recommended but optional. Use release branches only if you support old versions. + +### AI agents + +- Write Conventional Commits with type, scope and the footers `Refs:` and `Assisted-by:` (VCS-03, VCS-05). +- Mark breaking changes with `!` and `BREAKING CHANGE:` (VCS-04). +- Keep commits atomic and buildable (VCS-06). +- Work on a short-lived branch and never push to `main` directly (VCS-02). +- Never force-push shared branches or rewrite history you did not create (VCS-08). +- Never edit version strings by hand (VCS-10). +- Never commit secrets, build outputs or large binaries (VCS-01). diff --git a/content/guides/agile-requirements-and-user-stories.md b/content/guides/agile-requirements-and-user-stories.md new file mode 100644 index 0000000..7c7bc8c --- /dev/null +++ b/content/guides/agile-requirements-and-user-stories.md @@ -0,0 +1,186 @@ +--- +title: Agile requirements and user stories +--- + +# Agile requirements and user stories + +This guideline establishes the engineering standard for defining, sizing, and verifying requirements across agile product delivery teams. It defines the Connextra user story structure, the INVEST validation quality model, Behaviour-Driven Development (BDD) / Gherkin acceptance criteria, and outcome-oriented Sprint Goals. + +## 1. Context and Alignment in the Engineering Lifecycle + +Requirements represent the contract between product management, systems architecture, and delivery engineers. Ambiguous or poorly decomposed requirements are the leading cause of late-cycle defects, scope creep, and delivery friction. + +```mermaid +flowchart LR + A["Product Strategy / Epics"] --> B["Sprint Goals (Outcome)"] + B --> C["User Stories (INVEST)"] + C --> D["Acceptance Criteria (Gherkin / BDD)"] + D --> E["Definition of Ready"] + E --> F["Implementation & TDD"] + F --> G["Definition of Done"] +``` + +[[CAPTION:Figure]] Requirements traceability flow from epic outcomes down to automated acceptance tests. + +This guideline aligns directly with: +- **[Definition of Ready](../templates/definition-of-ready.md)**: Defines the criteria a user story must satisfy before being committed into a sprint backlog. +- **[Definition of Done](../templates/definition-of-done.md)**: Defines the engineering checklist required before a completed story can be accepted into a releasable increment. +- **[Writing a software design specification](software-design-specification.md)**: Provides the technical design specification for complex stories requiring architectural elaboration. + +## 2. User Story Authoring Standard + +A user story is an invitation to a conversation, capturing the intent of a feature from the perspective of an entity that derives value. + +### 2.1 The Connextra Structure + +All user stories must be written using the standard three-part Connextra syntax: + +```text +As a +I want +So that +``` + +[[CAPTION:Figure]] Standard three-part Connextra user story syntax. + +#### Persona Specificity Rules +- **Prohibited**: Generic personas such as `As a user` or `As a developer`. +- **Mandatory**: Concrete personas representing real stakeholders or system boundaries: + - *Consumer IoT*: `As a homeowner with solar battery storage...` + - *Firmware / Hardware*: `As a field service technician connecting over BLE...` + - *Platform / Operations*: `As a site reliability engineer monitoring edge broker latency...` + - *Security*: `As a compliance auditor verifying cryptographic key rotation...` + +### 2.2 The INVEST Validation Model + +Every user story committed to a sprint must satisfy the six INVEST criteria: + +| Criterion | Definition | Verification Check | +| :--- | :--- | :--- | +| **I: Independent** | The story can be developed, tested, and released in any sequence without hard dependencies on other in-flight stories. | Are blocking dependencies resolved or decomposed into standalone deliverables? | +| **N: Negotiable** | The story captures the goal, leaving implementation mechanics open to engineering refinement. | Does the story avoid dictating rigid implementation details that prevent technical trade-offs? | +| **V: Valuable** | Delivers discernible value to a customer, stakeholder, or downstream system upon completion. | Can the product owner or customer articulate the benefit of this increment alone? | +| **E: Estimable** | The team understands the scope and technical complexity sufficiently to assign relative effort. | Has ambiguity been clarified through spikes or technical designs ([Writing a software design specification](software-design-specification.md))? | +| **S: Small** | Sized to fit comfortably within a single sprint, ideally taking 2 to 4 engineering days. | Can the story be completed, tested, and code-reviewed well before the sprint boundary? | +| **T: Testable** | Accompanied by unambiguous acceptance criteria that can be validated objectively. | Can QA or automated test harnesses verify whether the criteria pass or fail? | + +[[CAPTION:Table]] INVEST criteria for user story quality assurance. + +## 3. Acceptance Criteria and Behaviour-Driven Development (BDD) + +Acceptance criteria define the boundary of a story, establishing the conditions that must be true for the product owner and QA to accept the work. + +### 3.1 Scenario-Based Criteria (Gherkin Syntax) + +For functional flows, user interaction, and integration contracts, acceptance criteria must use the declarative Gherkin syntax (`Given-When-Then`): + +- **Given**: The initial static context, system state, or precondition before the action occurs. +- **When**: The trigger, user interaction, external event, or command under test. +- **Then**: The observable outcome, state change, or output assertion. +- **And / But**: Additional preconditions or postconditions refining the scenario. + +#### Example 1: IoT Smart Gateway Firmware Over-the-Air (FOTA) Update + +```gherkin +Feature: Dual-bank firmware image verification and activation + In order to maintain security and operational reliability + As an IoT edge gateway running Zephyr RTOS + I want to verify and activate newly downloaded firmware images safely + + Scenario: Successful firmware image signature verification + Given an IoT gateway with active firmware version "2.4.1" on partition Bank A + And a downloaded firmware payload "firmware-v2.5.0.bin" staged in Bank B + When the bootloader validates the staged image with public key "pk_prod_secp256r1" + Then the cryptographic ECDSA signature check returns SUCCESS + And the bootloader marks Bank B as "pending_test_boot" + And the system triggers a warm hardware reset into Bank B + + Scenario: Corrupted image fails signature verification + Given an IoT gateway with active firmware version "2.4.1" on partition Bank A + And a downloaded firmware payload with an invalid signature in Bank B + When the bootloader validates the staged image with public key "pk_prod_secp256r1" + Then the cryptographic signature check returns FAILURE + And the bootloader erases Bank B to avoid execution of untrusted code + And the gateway logs a security fault code 0xE004 to local flash + And the system continues running the verified image in Bank A +``` + +[[CAPTION:Figure]] Gherkin acceptance test scenarios for an embedded IoT firmware update workflow. + +#### Example 2: Telemetry Ingestion API Rate Limiting + +```gherkin +Feature: API Gateway Ingestion Rate Limiting + In order to protect back-end telemetry pipelines from denial-of-service spikes + As an API Gateway edge proxy + I want to throttle device requests exceeding contracted tier quotas + + Scenario: Device exceeds per-minute request rate limit + Given an authenticated IoT device "dev-88492" on the Standard tier (limit: 120 req/min) + And the device has already submitted 120 requests in the current 60-second window + When the device submits a POST request to "/v1/telemetry" + Then the API Gateway rejects the request with HTTP status 429 + And the response header contains "Retry-After: 15" + And the response body conforms to RFC 9457 Problem Details schema +``` + +[[CAPTION:Figure]] Gherkin acceptance test scenario for an API rate-limiting contract. + +### 3.2 Rule-Based Checklist Criteria + +Non-scenario requirements (for example, static code metrics, accessibility, or security standards) may be expressed as explicit checklists: + +- [ ] Code changes must satisfy MISRA C:2012 guidelines ([MISRA C](../standards/misra-c.md)) with zero mandatory rule violations. +- [ ] Ingestion endpoint p99 latency must remain under 150 ms under a synthetic load of 2,000 requests/sec. +- [ ] All public API parameters must be documented with OpenAPI 3.1 schemas ([API design and error handling](../standards/api-design.md)). + +## 4. Sprint Goal Alignment Standard + +Sprint backlogs must not be arbitrary collections of unrelated tasks. Every sprint must formulate a single, cohesive Sprint Goal that guides team focus and trade-offs when unexpected hurdles arise during the sprint. + +### 4.1 Sprint Goal Template + +Teams must express their Sprint Goal using the following outcome-focused structure: + +```text +Our focus is on: +We believe it delivers: +To: +This will be confirmed when: +``` + +[[CAPTION:Figure]] Outcome-focused Sprint Goal template. + +#### Concrete Sprint Goal Example + +> **Our focus is on**: Implementing zero-touch device provisioning over Bluetooth Low Energy.
+> **We believe it delivers**: A 70% reduction in field installation time and eliminates manual credential typing.
+> **To**: Field deployment technicians and smart home installers.
+> **This will be confirmed when**: A factory-fresh gateway successfully pairs, receives encrypted Wi-Fi credentials over BLE, and establishes its first MQTT connection to the cloud staging cluster in under 45 seconds. + +## 5. Story Decomposition and Splitting Patterns + +When a story is too large to satisfy the Small criterion in INVEST, teams must split it vertically rather than horizontally. Horizontal splitting (for example, "Story 1: Create DB tables", "Story 2: Build API", "Story 3: Build UI") results in orphaned, non-valuable increments. Vertical slicing delivers thin end-to-end functionality in each increment: + +```mermaid +flowchart TD + subgraph Bad["Anti-Pattern: Horizontal Layer Splitting"] + H1["Story 1: Database Schema (No standalone value)"] + H2["Story 2: Back-End Controller (Cannot test end-to-end)"] + H3["Story 3: Front-End UI (Delivered weeks later)"] + end + subgraph Good["Best Practice: Vertical Slicing"] + V1["Slice 1: Happy-path end-to-end flow for single device type"] + V2["Slice 2: Error handling, retries, and edge cases"] + V3["Slice 3: Support for secondary device variants"] + end +``` + +[[CAPTION:Figure]] Horizontal layer splitting anti-pattern versus vertical end-to-end slicing. + +### 5.1 Common Vertical Splitting Techniques + +- **Workflow Steps**: Split by sequential user actions (for example, first implement device discovery, then implement device registration). +- **Business Rule Variations**: Start with standard default rules, then add complex discount calculations or tiered rates in subsequent stories. +- **Data Variations**: Support a single sensor data format (for example, temperature only) before extending support to humidity and air quality metrics. +- **Platform or Device Slices**: Implement support for BLE commissioning before implementing alternative NFC pairing. diff --git a/content/guides/architecture-review-board.md b/content/guides/architecture-review-board.md new file mode 100644 index 0000000..274d5be --- /dev/null +++ b/content/guides/architecture-review-board.md @@ -0,0 +1,502 @@ +--- +title: Running an architecture review board +--- + +# Running an architecture review board + +## Purpose + +The Architecture Review Board (ARB) governs architecturally significant decisions. These are the ones that are expensive to unwind once shipped, because they cross a boundary between teams or set the shape of a system for years to come. + +Most of this guide is written with a multi-component programme in mind. One common split is a device or edge component, a platform or head-end service, a data layer and a downstream business system. The model applies to any set of components owned by different teams that need to interoperate. The same board also has a job to do on decisions contained entirely within one system, covered in Section 3. + +The board owns two things: the seams between systems, and the significant calls inside any one of them. It is not a general design review for each component, a gate for every architectural decision regardless of size, or a substitute for each team's own internal review process. + +### Scaling from single team to cross-functional + +The same mechanics apply whether there's one team or several. A single team running its own architecture review can use the identical process. A proposal document and a draft ADR are reviewed against the tests in Section 3, decided by a small group and recorded permanently. The difference is only in scale. + +Where a single team is the whole picture, the board might just be the tech lead and one or two senior engineers. Most decisions then clear the bar without a formal session at all, since there's no other team to coordinate with. The rest of this guide describes the cross-functional version, with separate architect seats per component. It exists because coordinating several teams needs a slightly heavier process, to make sure nobody's missing from the room. Read the membership and cadence sections as the fuller version and trim them down if there's only one team in scope. + +## What goes to the board + +### Two tests, not one + +Two separate questions decide whether something belongs at the ARB. Either is sufficient on its own. + +**The cross-system test.** Anyone proposing a change should be able to answer this before submitting anything: + +> If I change this, does someone outside my team need to change something too? + +**The significance test.** Even where the answer above is no, a second question applies: + +> Would this be expensive or embarrassing to unwind in a year's time? + +A single component can fail this test on its own, with no other team involved. Examples are adopting a new database engine, moving to a different pattern such as event sourcing, or changing its security posture. These go to the ARB too, for the same reason cross-system changes do. Nobody else has to act on them immediately, but getting them wrong is costly, and other teams will likely copy whatever precedent gets set here. + +A third question, applied by the chair at intake regardless of which test triggered the submission, narrows things further: + +> Is this trivially reversible? + +If yes, the full process probably isn't warranted even if one of the two tests above technically fires. + +```mermaid +flowchart TD + A[Proposed change] --> B{Does someone outside\nmy team need to change\nsomething too?} + A --> C{Would this be expensive\nor embarrassing to unwind\nin a year's time?} + B -->|Yes| D{Trivially reversible?} + C -->|Yes| D + B -->|No| E + C -->|No| E[Stays with the\nowning team] + D -->|No| F[Goes to the ARB] + D -->|Yes| E +``` + +### In scope: cross-system + +| Category | Example | +|---|---| +| Shared data model / contract changes | Adding, changing, or deprecating a field or resource in an interface consumed by another component | +| Shared data contracts | Encoding or schema changes affecting more than one consuming system | +| Cross-system security decisions | Trust boundary changes, key or certificate management, mitigations that span subsystems | +| Cross-cutting non-functional requirements | Latency, availability, or throughput budgets that span multiple components | +| Integration pattern changes | Introducing or changing sync vs async communication, new message topics or queues, new API versions between systems | +| End-to-end business flows | Any flow where one component's enforcement or assumptions has to match another's | + +### In scope: architecturally significant, single system + +| Category | Example | +|---|---| +| Technology or platform adoption | Choosing or replacing a database engine, message broker, or core language/runtime for a component | +| Significant pattern adoption | Moving a component to event sourcing, CQRS, or a materially different structural pattern | +| Security architecture | Changes to authentication, authorisation, or cryptographic approach contained within one system | +| Precedent-setting choices | Anything likely to be copied by other teams once it exists, whether or not it was intended as a template | + +### Out of scope + +| Category | Example | +|---|---| +| Internal implementation detail | A component's internal caching strategy, provided it doesn't affect SLAs others depend on and isn't a significant pattern shift | +| Component-local decisions | Choice of library or framework within a single component, where the choice is routine rather than significant | +| UI-only changes | Front-end changes with no data model impact | +| Internal refactoring | Refactoring that doesn't touch the shared contract or shift the component's architecture | + +### Handling drift + +The board may keep reviewing implementation detail within one subsystem, rather than the boundary or the significant call itself. That's a reason to tighten scope, not a reason to give the board more authority. It can also point to a Team Topologies mismatch: a collaboration mode used between teams where an X-as-a-Service relationship would do, or the other way round. Worth raising on its own terms rather than folding into the ARB's remit. + +## Membership and roles + +### Core voting members + +| Role | Represents | Vote | +|---|---|---| +| Chair | Overall architecture coherence, casting vote on deadlock | Yes | +| Architect, Component A | Constraints and design of Component A | Yes | +| Architect, Component B | Constraints and design of Component B | Yes | +| Architect, Component C | Constraints and design of Component C | Yes | +| Architect, Component D | Constraints and design of Component D | Yes | +| Security lead | Cross-cutting threat model and security posture | Yes | + +Adjust the number of component architect seats to the actual number of subsystems in scope. Keep voting membership to 6-8 people where possible. Anyone beyond that attends for specific agenda items as a contributor, not as a standing member. + +### Non-voting standing attendee: product manager + +The PM attends every session but doesn't vote on technical trade-offs. Their role is different from the architects': + +- flagging delivery infeasibility, where a technically sound decision can't land in the agreed timeline +- flagging commercial or regulatory constraints already committed to +- surfacing cross-programme dependencies the architects may not be tracking + +The PM's role is to raise delivery impact, not to arbitrate technical correctness. If a concern can't be resolved in the session, it escalates above the ARB to whoever owns the delivery-versus-quality trade-off. It does not stall the architectural decision itself. + +### Contributors and senior developers + +Invited per agenda item where relevant: + +- delivery leads for affected teams +- senior developers or tech leads from the team whose change is on the agenda +- test or QA representatives where a change affects cross-system test strategy +- external stakeholders where a decision touches a third-party integration + +Whether developers sit as contributors or as the board itself depends on scale. In a single-team setup (Section 1.1), developers and senior engineers usually are the board. There's no separate architect layer to defer to, and adding one just to mirror a cross-functional structure would slow things down for no benefit. + +In the cross-functional case, keep voting with the component architects. But bring in the senior developer or tech lead from the affected team whenever their change is on the agenda. They usually wrote the proposal document and drafted the ADR. They're often better placed than the architect to answer questions about feasibility, migration mechanics, or what actually happens to the deployed fleet during rollout. Not giving them a seat in the room when their own change is being discussed tends to produce decisions that look fine on paper and fall apart in delivery. + +Where this gets it wrong is treating a senior developer as a junior version of an architect, there to observe rather than contribute. If their input would change the decision, they should be in the room for that item, not relayed through their architect secondhand. + +### Decision rights + +Ambiguity here is the most common way an ARB fails, so state the model explicitly and revisit it if it stops working. + +**Recommended: chair-decides-with-board-input.** The board discusses, each voting member states their position and any dissent, and the chair makes the final call informed by that discussion. Dissent gets recorded in the ADR rather than smoothed over. This tends to move faster than pure consensus across several subsystems with genuinely competing constraints, while still keeping disagreement in the record. + +**Alternative: pure consensus.** Slower, and prone to deadlock, but can suit a smaller or newer board that hasn't built trust yet. If used, define an explicit deadlock-breaking mechanism up front, such as escalation to the programme sponsor. + +Whichever is chosen, name it in this document. "The board decides" without a mechanism behind it isn't a decision process. + +## Intake and decision process + +### Standard flow + +Every submission needs two things: a proposal document, and a draft ADR. The ADR isn't something written up after the board has decided; it's drafted in Proposed status and reviewed alongside the proposal itself. + +The label on the proposal document doesn't matter much. A design doc, an RFC, an interface control document redline or another format all work. It needs to cover the problem, the options considered, the impact and a recommendation, in a form the board can read in a few minutes. + +```mermaid +flowchart LR + A[Proposer submits\nproposal doc + draft ADR\nStatus: Proposed] --> B[Pre-read circulated\n48h before session] + B --> C{Board discusses\nand decides} + C -->|Approve| D[ADR: Accepted] + C -->|Approve with\nconditions| D + C -->|Reject| E[ADR: Rejected] + C -->|Defer| F[Back for more analysis] + F --> A + D --> G[Merged to architecture repo\nlinked from proposal\nregisters updated if relevant] +``` + +Verification checkpoints along this flow: + +- the proposal states which test triggered it (Section 2.1) and the impact on each affected subsystem where relevant +- the draft ADR's Context and Decision match the proposal's recommendation +- nothing gets discussed cold at the meeting +- the ADR is merged, linked, and cross-referenced from any architecture model it affects, and added to the relevant register (Section 6) if it establishes or changes one + +### 4.2 Fast-track path + +For genuinely urgent decisions that can't wait for the next standing session: + +1. A smaller quorum, the chair plus the two most affected architects, reviews a short proposal and draft ADR asynchronously +2. The decision is provisional until ratified at the next standing session +3. The draft ADR is updated at the point of the fast-track decision, not deferred until ratification + +Fast-track exists for urgency, not for avoiding scrutiny. If something is routinely fast-tracked because "there wasn't time," that's a planning problem worth raising on its own, not a reason to widen the fast-track path. + +### 4.3 Two-stage touchpoint for complex migrations + +Some decisions, breaking changes to a shared data model being the clearest example, are better split into two touchpoints rather than resolved in one sitting. + +```mermaid +flowchart TD + S1[Stage 1: architectural decision\napprove the change and\nrequired/optional classification] --> M[Delivery works out\nmigration timeline] + M --> S2[Stage 2: migration and\nrollout sign-off] + S2 --> A[ADR moves to Accepted\nconsequences finalised] +``` + +Resolving both in one session tends to rush the migration plan. Or it blocks a sound architectural call on logistics that were never going to be ready in time. Single-system significant decisions (Section 2.3) rarely need this split, since there's no other team's rollout to coordinate with. A large internal migration, such as replacing a database engine, may still warrant it. + +## 5. Worked process: breaking changes to a shared data model + +Breaking changes to interfaces or data models used by several systems are one of the highest-stakes recurring cases for most programmes. They are hardest where one side of the interface is slow or hard to update, as embedded devices and third-party systems are. + +### 5.1 What counts as breaking + +Adding a new required field or resource is unambiguously breaking. Every consuming system now has to send or expect it, and anything already deployed without it no longer complies with the updated definition. The same goes for removing a field, changing its type or meaning, or changing encoding in a way existing consumers can't parse. + +Adding an optional field is generally non-breaking and doesn't need this full process, though it's still worth a short ADR. + +### 5.2 Required artefacts + +For this category, the proposal document is the redlined interface document itself rather than a separate free-form write-up. The board sees the actual change in context, and the accompanying draft ADR captures the decision. + +| Artefact | Purpose | +|---|---| +| Updated interface document, redlined | The actual diff between current and proposed definitions, shown in context rather than described in prose | +| Draft ADR, status Proposed | Why the change is needed, the change itself and its required-versus-optional classification, and the headline migration impact including which systems implement by when | +| Backward compatibility / migration plan | Deployed-base impact, how many instances are on the old version, dual-version handling during transition, rollout sequencing, and the interface version bump (minor or major) | +| Updated interface or encoding contract | Format diff reviewed alongside the redlined interface document, since an identifier change is only half the contract | +| Updated architecture model | Only if the change alters what flows across a component boundary in a way the existing model doesn't already show | + +Once the rollout is non-trivial, keep the migration plan as a separate linked document rather than in the ADR's Consequences section. The ADR itself then stays proportionate (Section 7). The draft ADR isn't a separate document from the final one. It's the same record, moving from Proposed to Accepted, Rejected or Superseded as it goes through the process. + +### 5.3 Process sequence + +1. Proposer submits the redlined interface document and a draft ADR in Proposed status at Stage 1, with the migration plan attached or linked if complexity warrants it +2. Board reviews the redline and the ADR's Context and Decision, and approves or rejects the change and its required/optional classification +3. Once approved, delivery works out the migration timeline with the affected teams, if not already resolved +4. The migration plan comes back to the board at Stage 2 for a lighter sign-off +5. The ADR moves to Accepted once both stages are complete, with Consequences finalised and the migration plan linked + +## 6. Artefacts and registers + +### 6.1 Produced by the ARB + +- **ADR log**: the core, permanent decision record +- **Cross-system risk register**: where cross-cutting risks, including any threat-modelling findings, that span subsystems are tracked +- **Living context/container diagram**: a C4-style view at levels 1-2, showing subsystem boundaries and their contracts, kept current as decisions land +- **Technology and pattern register**: a running list of approved technologies, frameworks, and architectural patterns, each linked to the ADR that approved it. This is what stops a settled choice, a database engine, a messaging pattern, an authentication approach, being re-argued from scratch every time someone new joins or starts a piece of work. Entries get added whenever Section 2.3 fires, and reviewed periodically rather than treated as permanent; a pattern approved two years ago may no longer be the right default +- **Architecture principles register**: a short set of standing tenets, for example "prefer async over sync unless x," that sit above individual ADRs and let the board judge new proposals against something established rather than reasoning every case from first principles +- **Reference architecture / solution blueprint catalogue**: worked, end-to-end patterns for recurring problem shapes, such as how the organisation does event-driven integration or ingress security, more prescriptive than a radar entry and meant to be copied rather than just consulted +- **API and integration style guide**: naming, versioning, error handling, and encoding conventions, so each new interface doesn't independently re-decide things the register already settled once +- **Approved vendor or product list**: narrower than the technology register, focused on external suppliers and licensed products, often carrying procurement or security sign-off that a general technology entry doesn't need +- **Deprecation or sunset register**: the mirror image of the technology register, listing what's being phased out and by when, so new work doesn't get built on something already scheduled for retirement +- **Quality attribute / NFR catalogue**: standard latency, availability, and throughput targets that get reused across programmes instead of re-derived from scratch each time a new component is scoped + +### 6.2 Consumed by the ARB + +- Proposal documents submitted by proposers, each with an accompanying draft ADR +- Threat model outputs, where they touch shared components +- Delivery and dependency status from the PM + +### 6.3 Not owned by the ARB + +Detailed design documents, per-component specifications, and test plans stay with the owning team. The board reviewing these in detail is a scope-creep signal (Section 2.5). + +### 6.4 Who owns these registers: this ARB, or R&D as a whole + +Where this ARB is the only one, covering the whole engineering organisation, it owns all of the above directly and there's nothing further to decide. + +Where several ARBs exist, one per programme or product line, each with its own component architects, ownership shouldn't sit with any single one of them. A database engine or messaging pattern approved by one programme's board and silently re-decided by another's produces exactly the fragmentation these registers were meant to prevent. + +```mermaid +flowchart TD + subgraph RD[R&D / engineering leadership] + REG[Shared registers:\ntechnology, principles,\nreference architectures,\ndeprecations, NFR catalog] + end + A1[Programme A ARB] -->|proposes new entries| REG + A2[Programme B ARB] -->|proposes new entries| REG + A3[Programme C ARB] -->|proposes new entries| REG + REG -->|shared view| A1 + REG -->|shared view| A2 + REG -->|shared view| A3 +``` + +In that case the registers belong at the R&D or engineering-leadership level, above any individual ARB. Each programme's ARB still raises new entries through its own intake process (Section 4.1) when Section 2.3 applies. But the entry is escalated up rather than recorded only locally. What each ARB consults day to day is a view onto the shared set, not a document it maintains in isolation. + +This mirrors the scaling logic in Section 1.1. As the number of boards grows, some things that were fine to own locally need to move up a level. These registers are usually the first place that becomes obvious. + +### 6.5 Dispensations + +Sometimes a team needs to depart from an approved entry in one of the registers above, a technology choice, a pattern, a principle, for a specific and time-bound reason. Rather than quietly deviating or blocking delivery on a full register change, request a dispensation. + +A dispensation request states: + +- which register entry or principle it departs from +- the specific reason, scoped to the system or decision at hand, not a general exemption for the team +- an expiry date; dispensations are never granted indefinitely +- what needs to happen to close the gap and return to the approved standard + +The board approves or rejects a dispensation the same way as any other submission (Section 4.1). It is logged as a lightweight entry in the ADR log, not a full proposal and draft ADR. Expiry triggers the follow-up, not anyone remembering to check back in. When a dispensation lapses, the team has either moved to the approved standard or comes back to the board for renewal with an updated reason. If the exception is starting to look like the more common case, the standard itself is revisited. + +### 6.6 Structuring the technology register: a radar rather than a list + +A flat approved/not-approved list is the simplest way to run the technology register, but it loses information a more established format captures well. Thoughtworks publishes a Technology Radar twice a year. It sorts items into four quadrants (techniques, platforms, tools, and languages and frameworks) and four rings that reflect confidence rather than chronology. The rings are Adopt (seriously consider using), Trial (ready for use but less proven), Assess (worth watching, not yet ready to trial) and Hold (proceed with caution, or actively avoid). Thoughtworks also publish a "build your own radar" tool specifically for organisations wanting to run the same format internally. + +Borrowing the ring structure gives the technology register something a flat list can't. It can show a technology moving from Assess through Trial to Adopt as confidence grows, rather than forcing an in-or-out decision before anyone has used it. A component team piloting something new can sit in Trial without that being read as an organisation-wide mandate. Moving something to Hold is a clear, recorded signal to stop starting new work with it, without forcing an immediate rip-out of what's already there. + +Two adaptations are worth making if adopting this format internally rather than just reading the public one: + +- **Every entry needs a linked ADR.** Thoughtworks' public radar reflects their own consultants' collective experience across the industry; an internal one should reflect this organisation's own decisions. An entry with no ADR behind it is an opinion, not a governance record, and undermines the point of the register. +- **Review on a fixed cadence, not continuously.** A twice-yearly cycle, or aligned to whatever release cadence the organisation already runs on, gives a fixed point to move entries between rings and retire ones that no longer apply, rather than the register drifting out of date between ad hoc edits. + +## 7. The ADR: format and proportionality + +An ADR's length should match the decision, not a fixed template. A three-sentence ADR for a simple, uncontested change is correct, not incomplete. Optional sections like Alternatives Considered or a separate Rationale only need filling in where the decision actually warrants that level of explanation. + +### 7.1 Minimal example + +```markdown +# ADR-0042: Add `example_field` as a required field in [Interface X] + +- **Status**: Accepted +- **Date**: YYYY-MM-DD +- **Author**: [Name] +- **Context link**: [Proposal document reference, if applicable] + +## Context + +[Consuming system] needs this value on every message to support +[capability]; making it optional means it can't rely on its presence. + +## Decision + +Add as a required field in [Interface X] v3.4.0. Instances on v3.3.0 +are treated as non-compliant until upgraded. + +## Consequences + +[Consuming system] has to handle a mixed-version deployed base during +rollout (see migration plan, linked). Owning team implements by +[milestone]. +``` + +Rationale and Alternatives Considered are left out here because the decision didn't need them. Include either section only when there's something worth recording. + +### 7.2 When to expand + +Expand the ADR only where the decision itself is genuinely more complex. Examples are restructuring how a business flow's data moves across three systems, or adopting a significant single-system pattern (see Section 2.3). Add a Rationale section where the trade-offs need spelling out. Add an Alternatives Considered section where other options were seriously weighed. Both stay optional; don't add them by habit. + +### 7.3 When an ADR is required at all + +Write a draft ADR for every submission to the ARB, regardless of length and regardless of whether the optional sections get used. Trying to judge "was this significant enough" case by case tends to become its own recurring argument, and a blanket rule avoids that. It also pays off during compliance or audit review. "When and why was this field made required?" is a common question, and a short ADR answers it far better than a long discussion thread. + +## 8. Cadence and escalation + +### 8.1 Cadence + +| Session | Frequency | Duration | Purpose | +|---|---|---|---| +| Standing session | Fortnightly | 60-90 minutes, hard timebox | Proposal and draft ADR review, decisions, backlog triage | +| Fast-track | Ad hoc | As needed | Urgent cross-system decisions, ratified at next standing session | +| Backlog triage | First 10 minutes of standing session | 10 minutes | Sort new submissions before working through existing ones | + +### 8.2 Escalation + +| Situation | Escalates to | +|---|---| +| PM raises a delivery-infeasibility concern that can't be resolved in-session | Programme board or sponsor | +| Decision-rights deadlock under a consensus model | Programme sponsor | +| Repeated scope drift into implementation detail | Chair, for charter review, not board authority expansion | + +## 9. Anti-patterns + +Rubber-stamping, where nothing is ever rejected or sent back, means the board isn't doing its job. Becoming a bottleneck, where teams route around it because it's too slow, means scope has crept too far into implementation detail. Decisions without ownership, where an ADR has no named owner for follow-through, undermine the whole point of recording them. A Team Topologies mismatch is when the board keeps making calls that should sit inside one team's boundary. It means the interaction mode between teams needs revisiting, not that the board needs more authority. And gold-plating, writing long ADRs or filling in optional sections out of habit for trivial decisions, defeats the proportionality this guide is built around. + +## Appendix A: Draft ADR template + +```markdown +# ADR-XXXX: [Short, meaningful title for the decision] + +- **Status**: Proposed | Accepted | Rejected | Superseded +- **Date**: YYYY-MM-DD +- **Author**: [Name] +- **Context link**: [Link to proposal document or ticket, if applicable] +- **Supersedes**: [ADR-XXXX] (optional) +- **Superseded by**: [ADR-YYYY] (optional) + +## Context + +What led to this decision. What problem or scenario required a choice. + +## Decision + +State the decision clearly and directly. + +## Consequences + +What changes as a result. Any risks, costs, or follow-up actions. + +## Rationale (optional) + +Why this decision, and what the trade-offs were. Include only where the +decision needs explaining beyond Context and Decision. + +## Alternatives considered (optional) + +Other options and why they were rejected. Include only where other +options were seriously weighed. +``` + +## Appendix B: Open decisions for this charter + +These should be settled before adoption, since they change how the rest of the document is applied: + +1. Decision-rights model: chair-decides-with-board-input, or pure consensus with a named deadlock-breaker (Section 3.4) +2. For threat-modelling-driven mitigations spanning subsystems, does the board approve the mitigation direction only, or also review the resulting contract change before it ships (Section 4.3) +3. Number and names of core component architect seats, matched to the actual subsystems in the programme (Section 3.1) +4. Whether the migration plan for breaking data model changes is always a separate linked artefact, or folded into the ADR's Consequences for lower-complexity cases (Section 5.2) +5. Which proposal document format, design doc, RFC, ICD redline, or something else, becomes the organisation's default for the general case in Section 4.1 +6. Which of the registers in Section 6.1 sit with this ARB versus a broader R&D/engineering-leadership body, based on how many ARBs exist across the organisation (Section 6.4), and who owns each one day to day either way + +## Appendix C: Example architecture principles + +A starting set for the principles register (Section 6.1). Trim, replace, or extend to match the organisation's own priorities; these are illustrative, not prescriptive. + +1. **Prefer boring technology.** Default to proven, well-understood tools unless there's a documented reason not to. Novelty is a cost to be justified, not a default to be assumed. +2. **Data ownership is single and explicit.** Each piece of data has exactly one system of record. Other systems hold copies or projections, never a second write path. +3. **Interfaces are versioned, not mutated.** A breaking change gets a new version rather than an in-place change that quietly breaks existing consumers. +4. **Security is designed in, not bolted on.** Authentication, authorisation, and encryption decisions are made at design time, not retrofitted after an incident. +5. **Favour asynchronous integration by default.** Synchronous coupling between systems is the exception, justified case by case, not the default choice. +6. **Every architecturally significant decision is recorded.** If it isn't in an ADR, it didn't happen, and can't be relied on as precedent. +7. **Design for observability from day one.** A component isn't done until its logs, metrics, and traces are wired into the shared observability tooling. +8. **Reversibility beats prediction.** Where the future is genuinely uncertain, prefer the option that's cheaper to undo over the option that looks theoretically optimal. + +## Appendix D: Example pattern catalogue + +A starting set for the reference architecture / pattern catalogue (Section 6.1), again illustrative rather than exhaustive. + +| Pattern | Use it for | Avoid when | +|---|---|---| +| Event-driven integration | Cross-system notifications where the receiving system doesn't need an immediate response | The caller needs a synchronous result before it can proceed | +| API gateway / edge aggregation | A single entry point handling auth, rate limiting, and routing for external consumers | Internal-only traffic where the extra hop adds latency for no benefit | +| Strangler fig migration | Replacing a component incrementally, routing a growing share of traffic to the new version over time | A clean-slate rebuild with no need to run old and new side by side | +| CQRS (command/query separation) | Read and write load profiles that diverge significantly enough to justify separate models | Simple CRUD workloads, where the added complexity isn't earned | +| Circuit breaker | Wrapping calls to an unreliable dependency so failures fail fast rather than cascade | Calls to dependencies with their own strong reliability guarantees, where the extra layer adds no protection | +| Anti-corruption layer | A translation boundary protecting a component's internal model from a legacy or external system's data shape | Two systems that already share a common, well-governed data model | +| Outbox pattern | Ensuring an event is only published once a database write has committed, avoiding dual-write inconsistency | Systems with no event publication step, where the problem doesn't arise | + +## Appendix E: Example C4 diagrams, Level 1 and Level 2 + +A worked example using a small IoT system: a home weather station reporting to a cloud service. It can be viewed on a web dashboard and a mobile app, and is integrated with Home Assistant for automations. Level 1 shows the system in its environment. Level 2 opens it up to show its containers, including the two paths into Home Assistant. A local network path keeps working without internet access, and a cloud path is used for remote access and longer history. Both diagrams use the C4-PlantUML library from PlantUML's standard library (`!include `), so nothing needs to be downloaded. + +### Level 1: System context + +```plantuml +@startuml +!include + +Person(homeowner, "Homeowner", "Views current and historical weather data, configures alerts") +System(weatherPlatform, "Weather Station Platform", "Collects, stores, and serves weather data from a home weather station; core sensing and local automation continue to work without internet access") +System_Ext(homeAssistant, "Home Assistant", "Self-hosted home automation platform") + +Rel(homeowner, weatherPlatform, "Views weather data via web and mobile") +Rel(homeowner, homeAssistant, "Uses for home automation") +Rel(weatherPlatform, homeAssistant, "Publishes weather data to, locally or via the cloud") + +SHOW_LEGEND() +@enduml +``` + +### Level 2: Containers + +```plantuml +@startuml +!include + +Person(homeowner, "Homeowner") +System_Ext(homeAssistant, "Home Assistant", "Self-hosted home automation platform") + +System_Boundary(weatherPlatform, "Weather Station Platform") { + Container(station, "Weather Station", "Embedded device with sensors", "Measures temperature, humidity, wind, and rainfall, and sends readings periodically") + Container(cloud, "Cloud Service", "Cloud application", "Receives and validates readings, and exposes data via an API") + ContainerDb(storage, "Data Storage", "Time-series database", "Stores historical weather readings") + Container(web, "Web Interface", "Single-page web app", "Dashboard for viewing current and historical weather data") + Container(app, "Mobile App", "iOS/Android app", "Displays weather data and push notifications") + Container(haIntegration, "Home Assistant Integration", "Custom component / bridge", "Exposes weather data to Home Assistant, over the local network or via the cloud service") +} + +Rel(station, cloud, "Sends sensor readings when online", "MQTT/HTTPS") +Rel(station, haIntegration, "Publishes readings on the local network", "Local MQTT/HTTP, offline-capable") +Rel(cloud, storage, "Reads and writes readings", "DB driver") +Rel(web, cloud, "Queries weather data", "HTTPS/JSON") +Rel(app, cloud, "Queries weather data, receives alerts", "HTTPS/JSON") +Rel(haIntegration, cloud, "Syncs extended history and enables remote access when online", "HTTPS/JSON") +Rel(homeowner, web, "Views dashboard via") +Rel(homeowner, app, "Views weather, receives notifications via") +Rel(homeAssistant, haIntegration, "Polls data from, locally or via the cloud") + +SHOW_LEGEND() +@enduml +``` + +## References + +Works referenced or drawn on in this guide: + +- Richards, M. and Ford, N. *Fundamentals of Software Architecture*. O'Reilly. Covers architecture decision records and governance structures directly. +- Richards, M., Ford, N., Rehmer, D. and Sass, Z. *Software Architecture: The Hard Parts*. O'Reilly. Trade-off analysis across team and component boundaries. +- Erder, M., Pureur, P. and Woods, E. *Continuous Architecture in Practice*. Addison-Wesley. Lightweight governance and fitness functions in an agile delivery context. +- Clements, P. et al. *Documenting Software Architectures: Views and Beyond*. Addison-Wesley (SEI series). Source for formal architecture evaluation methods such as ATAM. +- Bass, L., Clements, P. and Kazman, R. *Software Architecture in Practice*. Addison-Wesley (SEI series). The original source for ATAM and related architecture evaluation methods. +- Hohpe, G. *The Software Architect Elevator*. O'Reilly. On the organisational and political mechanics of making architecture governance actually work. +- Skelton, M. and Pais, M. *Team Topologies*. IT Revolution Press. On interaction modes between teams and what it means when a review board keeps making calls that should sit inside one team's boundary. +- Brown, S. *Software Architecture for Developers, Volume 1 and Volume 2*. Leanpub. Source material for the C4 model and its practical use in architecture documentation. +- Starke, G., Simons, M., Zörner, S. and Müller, R.D. *arc42 by Example*. A second documentation framework alongside C4, useful for structuring architecture documentation more broadly. +- Keeling, M. *Design It! From Programmer to Software Architect*. Pragmatic Bookshelf. Covers lightweight architecture decision-making practice directly. +- Millett, S. and Tune, N. *Patterns, Principles, and Practices of Domain-Driven Design*. Wrox. On boundary and pattern decisions that are architecturally significant even within a single system. +- DeMarco, T. and Lister, T. *Waltzing with Bears: Managing Risk on Software Projects*. Dorset House. On risk management practice for software projects. +- Medjaoui, M., Wilde, E., Mitra, R. and Amundsen, M. *Continuous API Management*. O'Reilly. On managing API and integration decisions as they evolve. +- Monson-Haefel, R. (ed.) *97 Things Every Software Architect Should Know*. O'Reilly. Short, aphoristic collective wisdom from practising architects. +- Thoughtworks. *Technology Radar*. Published twice yearly at thoughtworks.com/radar. Source for the ring and quadrant structure discussed as an option for the technology register in Section 6.6. +- The Open Group. *TOGAF Standard*. Source for the Architecture Board, Architecture Compliance review, and dispensation (time-bound waiver) concepts referenced in this guide's governance model. +- Nygard, M. "Documenting Architecture Decisions" (original ADR format, 2011). The source of the ADR format this guide builds on. diff --git a/content/guides/c4-model/01-c4-model-foundations.md b/content/guides/c4-model/01-c4-model-foundations.md new file mode 100644 index 0000000..55eaceb --- /dev/null +++ b/content/guides/c4-model/01-c4-model-foundations.md @@ -0,0 +1,152 @@ +--- +title: "The C4 Model: Origins, Philosophy, and Core Mechanics" +--- + +# The C4 Model: Origins, Philosophy, and Core Mechanics + +## 1. The Architecture Communication Crisis: Why Traditional Diagrams Fail + +Visual communication is essential to software and systems engineering. When technical teams describe a new system, they instinctively sketch boxes and lines on a whiteboard. Yet, across the industry, software architecture documentation frequently suffers from what architect Simon Brown diagnosed as the **architecture communication crisis**: + +1. **The "Box-and-Line" Dilemma**: Ad-hoc architectural sketches often feature arbitrary shapes, ambiguous clouds, and unlabelled lines. A line connecting two rectangles might represent a function call, an asynchronous message, an electrical SPI bus, or a vague business relationship. +1. **The UML Fallacy**: In the 1990s and 2000s, the Object Management Group (`OMG`) developed the Unified Modelling Language (`UML`). While UML attempted to standardise software modelling with 14 distinct diagram types, it proved overly academic and heavy for modern fast-paced engineering teams. Developers rarely agreed on the subtle distinctions between aggregation and composition, and comprehensive UML class diagrams quickly drifted out of synchronisation with the codebase. +1. **Mismatched Levels of Abstraction**: A single whiteboard diagram often attempts to show everything at once: high-level business users, cloud servers, individual C header files, and database tables. Mixing architectural abstraction levels overwhelms readers and conceals critical design decisions. + +```mermaid +graph TD + subgraph The Traditional Whiteboard Problem + W1["Vague Shapes
(Is it a process, a server, or a class?)"] + W2["Unlabelled Lines
(What protocol? Synchronous or asynchronous?)"] + W3["Mixed Abstractions
(Databases next to C structs next to users)"] + end + + subgraph The C4 Solution + C1["Explicit Abstractions
Person, System, Container, Component"] + C2["Unambiguous Relationships
Every line has an action and a protocol"] + C3["Hierarchical Zoom Levels
Google Maps metaphor for software systems"] + end + + W1 -.-> C1 + W2 -.-> C2 + W3 -.-> C3 +``` +[[CAPTION:Figure]] How the C4 model resolves the classic architectural communication crisis. + +## 2. Origins of the C4 Model: Simon Brown's Breakthrough + +The **C4 model** was created by independent software architect **Simon Brown** between 2006 and 2018. Brown observed that while agile software methodologies had revolutionised coding, testing, and delivery, teams had largely abandoned architectural documentation because existing tools were too cumbersome. + +Brown created C4 to provide a **lightweight, developer-friendly mental model** for describing software architecture. Rather than introducing a complex new notation, C4 focuses on a simple hierarchy of abstractions: + +- **Context** (Level 1) +- **Containers** (Level 2) +- **Components** (Level 3) +- **Code** (Level 4) + +Crucially, C4 is **notation-independent**. You can render a C4 model using PlantUML, Mermaid, Structurizr, Visio, or a dry-erase whiteboard. The value lies in the clarity of the underlying abstractions and the discipline of explicit communication. + +## 3. The Central Metaphor: Maps and Hierarchical Zoom Levels + +The defining breakthrough of the C4 model is its analogy to **digital maps** (such as Google Maps or OpenStreetMap). + +When navigating a digital map, you do not view individual street addresses from the continent level. You start at an altitude that shows oceans, countries, and major cities, and zoom in progressively: + +```mermaid +flowchart TB + subgraph Google Maps Zoom + M1["Continent / Country View"] + M2["City / District View"] + M3["Street / Neighbourhood View"] + M4["Building Floorplan Blueprint"] + end + + subgraph C4 Architecture Zoom + L1["Level 1: System Context
(Systems and User Personas)"] + L2["Level 2: Containers
(Applications, Databases, and Services)"] + L3["Level 3: Components
(Internal Subsystems and RTOS Tasks)"] + L4["Level 4: Code / Deployment
(Classes, State Machines, PCB Silicon)"] + end + + M1 <--> L1 + M2 <--> L2 + M3 <--> L3 + M4 <--> L4 +``` +[[CAPTION:Figure]] The Google Maps hierarchical zoom metaphor applied to the C4 architecture model. + +### Audience-Targeted Communication + +Different stakeholders require different altitudes of information. A Level 3 Component diagram will confuse an executive sponsor. A Level 1 System Context diagram gives a firmware engineer too little detail to implement an RTOS task: + +| C4 Zoom Level | Target Audience | Key Questions Answered | +| :--- | :--- | :--- | +| **Level 1: System Context** | Product owners, executives, non-technical stakeholders, external integration partners. | What is the product? Who uses it? Which external systems does it integrate with? What is out of scope? | +| **Level 2: Containers** | Software architects, engineering leads, DevOps, security teams, systems engineers. | What are the deployable software units? Where does data live? Which protocols link edge devices to the cloud? | +| **Level 3: Components** | Firmware developers, cloud software engineers, QA automation engineers. | How is a container structured internally? What are the key tasks and modules? Who is responsible for data serialisation? | +| **Level 4: Code & Deployment** | Silicon designers, driver implementers, power optimisation engineers. | How does the state machine transition? How are memory buffers partitioned? How are silicon packages wired? | +[[CAPTION:Table]] Stakeholder mapping across the four C4 abstraction levels. + +## 4. Why the C4 Model Works: Core Principles + +The success of the C4 model in modern engineering organisations stems from four foundational principles: + +### A. Abstraction Over Notation + +C4 does not mandate specific colours, line weights, or geometric shapes. Instead, it formalises **four core software building blocks**: + +1. **Person**: A human actor who interacts with the software system (for example, a homeowner or field technician). +1. **Software System**: The highest level of software abstraction; something that delivers value to users, whether developed in-house or provided externally. +1. **Container**: A separately runnable, deployable execution unit or data store (for example, a firmware binary on an MCU, a cloud microservice, a database, or a mobile app). +1. **Component**: A structural grouping of related functionality encapsulated behind a clean interface within a container (for example, an RTOS task, a sensor driver module, or a REST controller). + +### B. The Golden Rule of Explicit Relationships + +In C4, every connecting arrow must be unambiguous. An arrow without a label or protocol is considered an architectural defect: + +```mermaid +graph LR + subgraph Bad Practice + A1["Sensor Device"] --- A2["Cloud Platform"] + end + subgraph Good Practice + B1["Sensor Device"] -->|"Publishes environmental telemetry
[MQTT over TLS 1.3 / Port 8883]"| B2["Cloud Platform"] + end +``` +[[CAPTION:Figure]] The C4 rule of explicit relationships: every connection states intent and transport technology. + +Every relationship label must answer two questions: +1. **What is happening?** (Intent: *"Publishes environmental telemetry"*, *"Reads calibration constants"*) +1. **How does it travel?** (Technology: `[MQTT over TLS 1.3]`, `[I2C Bus @ 400 kHz]`, `[HTTPS / REST]`) + +### C. Static Structure vs Runtime Behaviour + +C4 diagrams primarily model **static structure**—the enduring software constructs that exist in the codebase and infrastructure. + +To show runtime behaviour, timing or life cycle transitions, C4 adds two kinds of diagram to the static views. **Dynamic Diagrams** are sequence diagrams that number the messages in order. **Deployment Diagrams** map software containers onto physical infrastructure and hardware. + +## 5. Adapting C4 for Embedded & Connected Systems + +In pure enterprise software, containers execute on virtualised cloud servers and interact over HTTP. In **connected IoT and embedded systems**, software containers execute on physical silicon packages, communicate across electrical traces, and interact with the physical environment. + +Historically, hardware engineers documented systems using electrical schematics and pinout tables, while cloud software engineers used API specifications and cloud architecture diagrams. This created an engineering disconnect: + +```mermaid +graph LR + subgraph The Traditional Disconnect + HW["Hardware Engineers
(Schematics, PCBs, Pinouts)"] + FW["Firmware Developers
(C/C++, Registers, RTOS Tasks)"] + CL["Cloud Architects
(Microservices, Queues, DBs)"] + end + + HW -. "Speaks electrical language" .-> FW + FW -. "Speaks byte buffers" .-> CL +``` +[[CAPTION:Figure]] The engineering communication chasm across connected IoT teams. + +The C4 model bridges this chasm by establishing a **shared visual lingua franca**: + +1. **Hardware as Containers**: Physical sensors, secure elements, and flash memory chips are modelled as hardware containers alongside firmware software containers. +1. **Buses as Protocols**: Peripheral buses (`I2C`, `SPI`, `UART`, `CAN`) are treated with the same architectural rigour as network protocols (`MQTT`, `CoAP`, `HTTPS`). +1. **Concurrency as Components**: RTOS background tasks, interrupt service routines (`ISRs`), and event queues inside the microcontroller firmware are cleanly mapped to Level 3 Components. + +By standardising on C4, cross-functional engineering teams can reason about end-to-end latency, security threat models, power consumption budgets, and firmware update mechanisms within a single cohesive framework. diff --git a/content/guides/c4-model/02-diagrams-as-code-tooling.md b/content/guides/c4-model/02-diagrams-as-code-tooling.md new file mode 100644 index 0000000..a169af4 --- /dev/null +++ b/content/guides/c4-model/02-diagrams-as-code-tooling.md @@ -0,0 +1,385 @@ +--- +title: "Diagrams as Code: Tooling, DSLs, and Repository Integration" +--- + +# Diagrams as Code: Tooling, DSLs, and Repository Integration + +## 1. The Diagrams-as-Code Paradigm + +Historically, engineering teams documented software architectures using visual drawing tools such as Microsoft Visio, Lucidchart, draw.io, or PowerPoint. While graphical editors offer an intuitive drag-and-drop experience, they introduce severe long-term maintenance liabilities in production software environments: + +1. **Opaque Binary Files and Unmergeable XML**: Graphical diagram files cannot be easily diffed in pull requests. Two engineers editing the same architecture diagram often encounter irreconcilable merge conflicts. +1. **Architectural Drift**: Because visual drawings live in external web portals or disconnected file shares, developers modify code without updating the architecture diagrams. Within months, documentation becomes obsolete. +1. **Manual Inconsistency**: In a traditional drawing tool, if an MQTT port changes from 1883 to 8883, every single slide and diagram containing that connection must be individually located and manually redrawn. + +```mermaid +graph LR + subgraph Traditional Diagramming + GUI["Web / GUI Drawing Tools
(Visio, draw.io, Slides)"] + Bin["Binary / Complex XML Blobs
(Unmergeable in Git)"] + Drift["Architectural Drift
(Drawings rot as code changes)"] + GUI --> Bin --> Drift + end + + subgraph Diagrams-as-Code + Text["Human-Readable Text Files
(.md, .puml, .dsl)"] + Git["Git Version Control
(Peer review, diffs, PR checks)"] + Build["Automated CI/CD Compilation
(Deterministic SVG, PDF, Web)"] + Text --> Git --> Build + end +``` +[[CAPTION:Figure]] The architectural evolution from manual visual drawing to version-controlled Diagrams-as-Code. + +The **Diagrams-as-Code** philosophy treats architecture documentation with the same engineering rigor as application source code: + +- Architecture diagrams are written in clean, human-readable text. +- Diagrams live inside the same Git repository as the code they describe. +- Every architectural change is reviewed in pull requests alongside code commits. +- Continuous Integration (`CI`) pipelines automatically validate, lint, and compile diagrams into high-resolution graphics. + +This repository provides comprehensive support for three complementary Diagrams-as-Code technologies: **PlantUML**, **Mermaid**, and **Structurizr DSL**. + +## 2. PlantUML & the C4-PlantUML Standard + +[PlantUML](https://plantuml.com/) is a mature, open-source diagramming engine that converts plain text definitions into vector graphics (`SVG`) and raster images (`PNG`) using Graphviz. + +### A. How C4-PlantUML Works + +Standard PlantUML focuses on traditional UML class, sequence, and component diagrams. To support C4 modelling, the open-source community created **C4-PlantUML**—a comprehensive library of pre-processor macros that provide native C4 visual styles, shapes, and connection semantics. + +In this repository, C4-PlantUML templates are vendored under `templates/`: +- `templates/C4_Context.puml`: Level 1 System Context definitions. +- `templates/C4_Container.puml`: Level 2 Container definitions. +- `templates/C4_Component.puml`: Level 3 Component definitions. +- `templates/C4_Deployment.puml`: Level 4 Physical Deployment definitions. + +### B. Core C4-PlantUML Macro Reference + +The primary macros provided by the library include: + +| Macro | Arguments | Purpose | +| :--- | :--- | :--- | +| `Person(alias, label, desc)` | `alias, "Display Name", "Description"` | Defines a human actor. | +| `System(alias, label, desc)` | `alias, "Display Name", "Description"` | Defines an internal software system. | +| `System_Ext(alias, label, desc)` | `alias, "Display Name", "Description"` | Defines an external, third-party software system. | +| `Container(alias, label, tech, desc)` | `alias, "Display Name", "Technology", "Description"` | Defines a software container (for example, firmware, microservice, or web application). | +| `ContainerDb(alias, label, tech, desc)` | `alias, "Display Name", "Technology", "Description"` | Defines a data store or persistent storage partition. | +| `Component(alias, label, tech, desc)` | `alias, "Display Name", "Technology", "Description"` | Defines a structural software component within a container. | +| `Rel(from, to, label, tech)` | `from, to, "Action / Intent", "Protocol / Transport"` | Connects two elements with an explicit directional relationship. | +| `System_Boundary(alias, label) { ... }` | `alias, "Boundary Name"` | Groups containers logically within a system boundary. | +| `Container_Boundary(alias, label) { ... }` | `alias, "Boundary Name"` | Groups components logically within a container boundary. | +| `Deployment_Node(alias, label, type) { ... }` | `alias, "Node Name", "Hardware / Infrastructure Type"` | Defines a physical enclosure, PCB, or cloud hosting environment. | +[[CAPTION:Table]] Core C4-PlantUML macro reference. + +### C. PlantUML Examples Across the Top 3 C4 Levels + +Below are concise, copy-pasteable PlantUML examples demonstrating how to author each of the top three C4 levels: + +#### Level 1: System Context in PlantUML +```plantuml +@startuml +!include + +Person(homeowner, "Homeowner", "Monitors home comfort and indoor air quality") +System(sensorSystem, "Smart Environmental Monitor", "Measures temperature, humidity, and air quality") +System_Ext(cloudPlatform, "Cloud Platform", "Ingests telemetry and issues push alerts") +System_Ext(localHub, "Local Smart Hub", "Automates home HVAC and ventilation") + +Rel(homeowner, sensorSystem, "Observes LED indicator", "Physical / Visual") +Rel(sensorSystem, cloudPlatform, "Publishes telemetry", "MQTT over TLS 1.3") +Rel(sensorSystem, localHub, "Broadcasts sensor events", "Matter over Wi-Fi") +@enduml +``` +[[CAPTION:Figure]] PlantUML Level 1 System Context diagram. + +#### Level 2: Containers in PlantUML +```plantuml +@startuml +!include + +System_Boundary(sensorDevice, "Smart Environmental Sensor") { + Container(fw, "Sensor Application Firmware", "C / FreeRTOS", "Manages measurement, power gating, and networking") + Container(sensor_chip, "BME688 Sensor Subsystem", "MEMS Silicon", "Digital gas, temperature, and humidity sensing") + ContainerDb(flash, "Non-Volatile Storage", "SPI NOR Flash", "Stores credentials, certificates, and staged firmware") +} + +Container_Ext(broker, "Cloud Ingestion Broker", "MQTT Broker", "Ingests device telemetry over mTLS") + +Rel(fw, sensor_chip, "Triggers sample acquisition", "I2C Bus @ 400 kHz") +Rel(fw, flash, "Reads configuration & stores logs", "SPI Bus @ 32 MHz") +Rel(fw, broker, "Publishes environmental payload", "MQTT over TLS 1.3 / Port 8883") +@enduml +``` +[[CAPTION:Figure]] PlantUML Level 2 Container diagram. + +#### Level 3: Components in PlantUML +```plantuml +@startuml +!include + +Container_Boundary(fw_container, "Sensor Microcontroller Firmware") { + Component(acq, "Sensor Acquisition Manager", "RTOS Task", "Schedules readings and filters raw sensor data") + Component(power, "Power Orchestrator", "State Controller", "Coordinates deep-sleep intervals and power rails") + Component(mqtt, "MQTT Client", "coreMQTT Module", "Serialises telemetry and publishes over TLS") + Component(hal, "Sensor I2C HAL", "Driver Layer", "Direct hardware register read/write operations") +} + +Container_Ext(sensor_hw, "BME688 Hardware IC", "Hardware", "Physical sensor registers") +Container_Ext(cloud_gw, "Cloud Gateway", "MQTT Broker", "Inbound telemetry port") + +Rel(power, acq, "Wakes for scheduled measurement", "RTOS Event Flag") +Rel(acq, hal, "Reads compensated values", "C Function Call") +Rel(hal, sensor_hw, "Executes register transfer", "I2C Bus") +Rel(acq, mqtt, "Enqueues telemetry payload", "RTOS Queue") +Rel(mqtt, cloud_gw, "Transmits packet", "TLS Socket") +@enduml +``` +[[CAPTION:Figure]] PlantUML Level 3 Component diagram. + +### D. Repository Tooling and Automation + +This repository provides seamless tooling to build, preview, and test PlantUML diagrams: + +1. **Native Windows Batch Wrapper (`tools/bin/plantuml.cmd`)**: Allows executing `plantuml` from any command prompt on Windows without configuring manual Java classpath commands. +1. **Documentation Site Rendering (`mkdocs.yml`)**: The `plantuml_markdown` extension parses ```plantuml fences in markdown files and compiles them into crisp vector SVGs at documentation build time. +1. **Document Generator (`createdocs.py`)**: When building formal DOCX or PDF engineering manuals, `createdocs.py` automatically executes `templates/plantuml.jar` to render and inline all diagrams into the final document. + +## 3. Mermaid: Lightweight Markdown-Native Diagramming + +[Mermaid](https://mermaid.js.org/) is a JavaScript-based diagramming library designed to render directly inside web browsers. + +### A. Strengths of Mermaid + +1. **Native GitHub and Web Support**: GitHub, GitLab, and MkDocs render Mermaid diagrams natively without requiring any local compiler, Java runtime, or Graphviz installation. +1. **Zero Toolchain Overhead**: Anyone reading documentation in a web browser or reviewing a pull request sees interactive diagrams rendered instantly. +1. **Broad Diagram Type Support**: Mermaid supports flowcharts (`flowchart TD`), sequence diagrams (`sequenceDiagram`), state machines (`stateDiagram-v2`), and Gantt timelines. + +### B. Mermaid Examples Across the Top 3 C4 Levels + +While Mermaid does not feature built-in C4 macros like PlantUML, you can cleanly model C4 abstractions using Mermaid's `flowchart` and `subgraph` syntax across all three levels: + +#### Level 1: System Context in Mermaid +```mermaid +flowchart LR + User["Homeowner
[Person]"] + Device["Smart Environmental Sensor
[Software System]
Monitors ambient conditions"] + Cloud["Cloud Platform
[Software System]
Stores telemetry & delivers alerts"] + Hub["Local Smart Hub
[External System]
Home Assistant / Matter"] + + User -->|"Observes LED indicator
[Visual]"| Device + Device -->|"Publishes periodic telemetry
[MQTT over TLS 1.3]"| Cloud + Device -->|"Emits local sensor events
[Matter over Wi-Fi]"| Hub +``` +[[CAPTION:Figure]] Mermaid Level 1 System Context diagram. + +#### Level 2: Containers in Mermaid +```mermaid +flowchart TD + subgraph SensorBoundary [Smart Environmental Sensor] + MCU["Application MCU
[Container: FreeRTOS Firmware]"] + SensorIC["BME688 Sensor
[Container: MEMS Silicon]"] + Flash["NOR Flash
[Container: SPI LittleFS Storage]"] + end + + subgraph CloudBoundary [Cloud Platform] + Broker["Cloud Ingestion Broker
[Container: MQTT Gateway]"] + Database["Telemetry Database
[Container: TimescaleDB]"] + end + + SensorIC -->|"Reads climate metrics
[I2C Bus @ 400 kHz]"| MCU + MCU -->|"Persists credentials & logs
[SPI Bus @ 32 MHz]"| Flash + MCU -->|"Publishes telemetry
[MQTT / TLS 1.3]"| Broker + Broker -->|"Inserts batch metrics
[SQL / Native TCP]"| Database +``` +[[CAPTION:Figure]] Mermaid Level 2 Container diagram. + +#### Level 3: Components in Mermaid +```mermaid +flowchart TB + subgraph FirmwareContainer [Container: Microcontroller Application Firmware] + Acq["Sensor Acquisition Manager
[Component: RTOS Task]"] + Power["Power & Sleep Orchestrator
[Component: State Machine]"] + MQTT["MQTT Client
[Component: coreMQTT Subsystem]"] + HAL["Sensor HAL & Driver
[Component: I2C Driver]"] + end + + SensorChip[("BME688 Sensor IC")] + CloudGW[("Cloud Ingestion Broker")] + + Power -->|"Wakes on timer tick
[RTOS Event Flag]"| Acq + Acq -->|"Reads measurement registers
[C Function API]"| HAL + HAL -->|"I2C register transaction
[I2C @ 400 kHz]"| SensorChip + Acq -->|"Queues telemetry frame
[RTOS Message Queue]"| MQTT + MQTT -->|"Publishes TLS packet
[mTLS / Port 8883]"| CloudGW +``` +[[CAPTION:Figure]] Mermaid Level 3 Component diagram. + +### C. Limitations of Mermaid + +- **Lack of C4 Semantic Enforcement**: Mermaid treats all nodes as generic flowchart vertices. It does not enforce person shapes, database cylinders, or formal C4 container styling unless manually configured with custom CSS classes. +- **Automated Layout Constraints**: Mermaid computes diagram layout dynamically in the browser. In complex diagrams with hundreds of nodes, layout engines (such as dagre) can produce overlapping links. + +## 4. Structurizr DSL: Model-First Architecture as Code + +While PlantUML and Mermaid are **diagram-first** tools, the **Structurizr DSL** represents a paradigm shift: **Model-First Architecture as Code**. + +```mermaid +graph TD + subgraph Diagram-First Approach [PlantUML / Mermaid] + D1["System Context Diagram File"] + D2["Container Diagram File"] + D3["Component Diagram File"] + Note1["Updating a component name requires
manually editing all 3 files."] + end + + subgraph Model-First Approach [Structurizr DSL] + M["Unified Architectural Model
(Entities, Metadata, and Relationships defined once)"] + V1["View 1: System Context"] + V2["View 2: Containers"] + V3["View 3: Components"] + V4["View 4: Deployment"] + M --> V1 + M --> V2 + M --> V3 + M --> V4 + Note2["Updating an entity automatically
propagates to all declared views."] + end +``` +[[CAPTION:Figure]] Diagram-First vs Model-First architectural modelling. + +### A. The Core Principle: Separation of Model from Presentation + +In a diagram-first tool, each diagram file is an isolated graphic. If you change a database technology from SQLite to LittleFS, or update an MQTT endpoint, you must locate and modify every individual diagram file. + +In the **Structurizr DSL**, you define the architecture **once** in a single conceptual model: +1. **The Model (`model`)**: Declares people, software systems, containers, components, deployment nodes, and their relationships. +1. **The Views (`views`)**: Declares which subsets of the model to visualise (for example, a Context view showing only external systems, a Container view, or a Component view showing internal modules). +1. **The Styles (`styles`)**: Decouples visual presentation (colours, shapes, icons) from the architectural topology. + +### B. Structurizr DSL Grammar Across the Top 3 Levels + +A complete Structurizr workspace file demonstrates the top three C4 levels defined within a single model and projected as individual views: + +```structurizr +workspace "Smart Sensor System" "Model-first architecture example covering top 3 levels" { + + model { + # --------------------------------------------------------------------- + # Level 1: People and Software Systems + # --------------------------------------------------------------------- + user = person "Homeowner" "Monitors indoor air quality" + cloud = softwareSystem "Cloud Platform" "IoT ingestion and analytics backend" + + sensor = softwareSystem "Smart Environmental Sensor" "IoT edge device" { + # ----------------------------------------------------------------- + # Level 2: Containers inside the Software System + # ----------------------------------------------------------------- + sensorChip = container "Sensor Hardware IC" "Bosch BME688" "Sensor" + flashStorage = container "NVS Flash Storage" "SPI NOR Flash" "Database" + + sensorFw = container "Application Firmware" "C / FreeRTOS" "Firmware" { + # ------------------------------------------------------------- + # Level 3: Components inside the Container + # ------------------------------------------------------------- + acqMgr = component "Sensor Acquisition Manager" "Periodic RTOS Task" + powerOrch = component "Power Orchestrator" "State Machine Controller" + mqttClient = component "MQTT Telemetry Client" "coreMQTT Subsystem" + i2cDriver = component "Sensor I2C Driver" "Peripheral HAL Driver" + } + } + + # Relationships across all levels + user -> sensor "Observes status LED" "Physical / Visual" + sensor -> cloud "Publishes telemetry" "MQTT over TLS 1.3" + + sensorFw -> sensorChip "Triggers measurement" "I2C @ 400 kHz" + sensorFw -> flashStorage "Reads credentials" "SPI @ 32 MHz" + + powerOrch -> acqMgr "Signals measurement wake" "RTOS Event Flag" + acqMgr -> i2cDriver "Reads data registers" "C Function API" + i2cDriver -> sensorChip "Executes I2C transfer" "I2C Bus" + acqMgr -> mqttClient "Enqueues telemetry payload" "RTOS Queue" + mqttClient -> cloud "Transmits packet" "TLS Socket" + } + + views { + # Level 1 View: System Context + systemContext sensor "Level1_Context" "System Context showing user, sensor, and cloud platform" { + include * + autoLayout lr + } + + # Level 2 View: Containers + container sensor "Level2_Containers" "Container diagram of the Smart Environmental Sensor" { + include * + autoLayout tb + } + + # Level 3 View: Components + component sensorFw "Level3_Components" "Component diagram of the application firmware" { + include * + autoLayout tb + } + + styles { + element "Person" { + shape Person + background #08427b + color #ffffff + } + element "Software System" { + background #1168bd + color #ffffff + } + element "Container" { + background #438dd5 + color #ffffff + } + element "Component" { + shape Component + background #5C6BC0 + color #ffffff + } + element "Database" { + shape Cylinder + background #2E7D32 + color #ffffff + } + } + } +} +``` +[[CAPTION:Figure]] Complete Structurizr DSL specification demonstrating Level 1, Level 2, and Level 3 declarations and views. + +### C. Structurizr tooling with py-doc + +The workspace above is in this repository as [`workspace.dsl`](https://github.com/markbac/markbac.github.io/raw/main/content/guides/c4-model/files/workspace.dsl). [py-doc](https://github.com/markbac/py-doc) works with Structurizr workspaces through its `c4` commands: + +1. **Check the model**, as a CI dry run: + ```bash + py-doc c4 check content/guides/c4-model/files/workspace.dsl --strict + ``` +1. **Explore it interactively** in Structurizr Local, to zoom from System Context down to Component views and adjust layouts: + ```bash + py-doc c4 up content/guides/c4-model/files/workspace.dsl + ``` +1. **Export the views** as images or diagram source (PlantUML, C4-PlantUML, Mermaid and others) for documents and build pipelines: + ```bash + py-doc c4 export -f svg mermaid -o build/c4 + ``` + +## 5. Tooling Selection Matrix + +Use the matrix below to select the optimal tool for your project and audience: + +| Evaluation Dimension | Mermaid | PlantUML (C4-PlantUML) | Structurizr DSL | +| :--- | :--- | :--- | :--- | +| **Underlying Paradigm** | Diagram-First | Diagram-First | Model-First | +| **Primary File Format** | Markdown fences (```mermaid) | Text files (`.puml`) or fences | Text file (`workspace.dsl`) | +| **Native GitHub Web Rendering** | Native (client-side in browser) | Requires CI pre-rendering or proxy | Requires export to PNG/SVG/Mermaid | +| **C4 Semantic Support** | Generic flowcharts (requires custom classes) | Native macros (`Person`, `Container`, `Rel`) | First-class native C4 object model | +| **Multi-View Consistency** | Manual (must update every diagram) | Manual (must update every diagram) | Automatic (updates in model flow to all views) | +| **Interactive Navigation** | Static diagrams | Static diagrams | Interactive web zoom (Structurizr Lite) | +| **Local Dependencies** | None (renders in browser) | Java and Graphviz | Docker / Java / Python wrapper | +| **Best Suited For** | READMEs, issue discussions, quick PR diagrams | Formal PDF/DOCX reports, detailed engineering docs | Enterprise and multi-team IoT architectures | +[[CAPTION:Table]] Tooling comparison: Mermaid vs PlantUML vs Structurizr DSL. diff --git a/content/guides/c4-model/03-system-context.md b/content/guides/c4-model/03-system-context.md new file mode 100644 index 0000000..f3c0eeb --- /dev/null +++ b/content/guides/c4-model/03-system-context.md @@ -0,0 +1,98 @@ +--- +title: System Context Diagrams (Level 1) +--- + +# System Context Diagrams (Level 1) + +## 1. Defining the Product Ecosystem Scope + +The **System Context diagram** represents the highest level of zoom in the C4 model. It establishes what is inside our product scope and how it connects to people, local home infrastructure, cloud backends, and external third-party services. + +In our reference system—the **Smart Home Environmental Monitor**: +- The **Physical Sensor Device** measures ambient temperature, relative humidity, volatile organic compounds (`VOCs`), and CO2 levels in the home. +- The **Smart Home Cloud Platform** ingests real-time metrics, computes historical analytics, manages firmware update rollouts, and pushes alert notifications. +- **External Systems** include local smart home hubs (for example, Home Assistant, Apple Home), third-party weather forecast APIs, and telecommunication carriers. + +```mermaid +flowchart TB + subgraph Users and Human Actors + Homeowner["Homeowner / Resident
[Person]
Monitors home air quality, receives push notifications, and configures comfort thresholds."] + Installer["Field Technician / Installer
[Person]
Commissions devices, inspects battery health, and performs zero-point sensor calibrations."] + end + + subgraph Scope: Smart Home IoT Solution + Sensor["Smart Home Environmental Sensor
[Physical IoT Product]
Battery-powered smart sensor capturing temperature, humidity, VOCs, and CO2."] + CloudPlatform["Smart Home Cloud Platform
[Software System]
Collects telemetry, evaluates alert thresholds, coordinates FOTA updates, and provides customer APIs."] + end + + subgraph External Systems and Infrastructure + LocalHub["Local Smart Home Hub
[External System: Home Assistant / Apple Home]
Local automation controller automating ventilation and heating based on sensor triggers."] + WeatherAPI["Public Weather Forecast API
[External System]
Provides outdoor ambient conditions for indoor/outdoor air quality comparison."] + NotificationService["Apple APNs & Google FCM
[External Infrastructure]
Delivers push notifications to user smartphones."] + end + + Sensor -- "Publishes periodic telemetry and alerts
[MQTT over TLS / Wi-Fi]" --> CloudPlatform + Sensor -- "Local real-time event broadcasts
[Matter over Thread / BLE]" --> LocalHub + CloudPlatform -- "Queries local outdoor air quality
[HTTPS / REST / JSON]" --> WeatherAPI + CloudPlatform -- "Dispatches push alert payloads
[HTTPS / Apple APNs & FCM]" --> NotificationService + NotificationService -- "Displays threshold alert banner" --> Homeowner + Homeowner -- "Views charts and changes setpoints
[HTTPS / Mobile App]" --> CloudPlatform + Installer -- "Diagnoses device and provisions Wi-Fi
[Bluetooth Low Energy (BLE)]" --> Sensor + LocalHub -- "Controls ventilation based on air quality" --> Homeowner +``` +[[CAPTION:Figure]] System Context diagram (Level 1) for the Smart Home Environmental Sensor ecosystem. + +## 2. Matching PlantUML C4 Context Diagram + +Below is the matching **PlantUML C4 Context diagram** using the standard `C4_Context.puml` library. The build toolchain automatically resolves the include from the repository's `templates/` folder: + +````plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title System Context: Smart Home Environmental Sensor Solution + +Person(homeowner, "Homeowner / Resident", "Monitors indoor air quality, views trends, and receives comfort alerts.") +Person_Ext(installer, "Technician / Installer", "Performs initial Wi-Fi provisioning, battery checks, and sensor calibration.") + +Enterprise_Boundary(c0, "Smart Home IoT Ecosystem") { + System(sensor, "Smart Environmental Sensor", "Battery-powered indoor air quality monitor measuring temperature, humidity, VOCs, and CO2.") + System(cloud, "Smart Home Cloud Platform", "Ingests time-series telemetry, evaluates rules, manages FOTA updates, and exposes REST APIs.") +} + +System_Ext(hub, "Local Smart Home Hub", "Home Assistant / Matter controller automating HVAC and ventilation locally.") +System_Ext(weather, "Public Weather Service", "External meteorological service providing outdoor baseline air quality and temperature.") +System_Ext(push, "Push Notification Service", "Apple APNs and Google Firebase Cloud Messaging for urgent threshold alerts.") + +Rel(sensor, cloud, "Publishes telemetry & alerts", "MQTT over TLS / Wi-Fi") +Rel(sensor, hub, "Broadcasts local sensor events", "Matter over Thread / BLE") +Rel(installer, sensor, "Provisions Wi-Fi & calibrates", "Bluetooth Low Energy") +Rel(cloud, weather, "Queries outdoor metrics", "HTTPS / REST / JSON") +Rel(cloud, push, "Dispatches alert events", "HTTPS / JSON") +Rel(push, homeowner, "Delivers threshold alerts", "Push Notification") +Rel(homeowner, cloud, "Views analytics & changes settings", "HTTPS / Mobile App") + +@enduml +```` +[[CAPTION:Figure]] PlantUML C4 Context diagram for the Smart Home IoT ecosystem. + +## 3. Key Relationships and Protocol Annotations + +On an IoT Context diagram, always annotate connecting arrows with **both the data intent and the transport layer**: + +| Interaction Path | Purpose / Description | Protocol & Physical Carrier | +| :--- | :--- | :--- | +| **Sensor $\rightarrow$ Cloud** | Periodic telemetry frames, battery status, and threshold alerts | `[MQTT over TLS / Wi-Fi @ 2.4 GHz]` | +| **Sensor $\rightarrow$ Local Hub** | Real-time sensor state broadcasts for instant local HVAC automations | `[Matter over Thread / 802.15.4]` | +| **Technician $\rightarrow$ Sensor** | Initial credential onboarding, diagnostics, and factory calibration | `[Bluetooth Low Energy (BLE) 5.0]` | +| **Cloud $\rightarrow$ Weather Service** | Ingesting external weather data for predictive heating models | `[HTTPS / REST / JSON]` | +| **Mobile App $\rightarrow$ Cloud** | User dashboard telemetry views and configuration updates | `[HTTPS / GraphQL & WebSockets]` | +[[CAPTION:Table]] Interaction protocols for the Smart Home Sensor context diagram. + +## 4. Tips & Tricks for Level 1 IoT Diagrams + +1. **Model Both the Device and the Cloud**: A common mistake is drawing *only* the physical hardware or *only* the cloud platform. A true IoT product system context must encompass the hardware edge, the local network, and the cloud services as unified actors. +1. **Differentiate Human Personas**: Separate day-to-day residents from installation technicians. Technicians typically possess administrative privileges, local physical access (BLE/buttons), and different tools. +1. **Capture Offline Capabilities**: In smart homes, cloud connectivity may fail while local automations must continue operating. Showing the local path (`Sensor -> Local Hub`) immediately communicates resilience to stakeholders. diff --git a/content/guides/c4-model/04-container-diagrams.md b/content/guides/c4-model/04-container-diagrams.md new file mode 100644 index 0000000..c609f1d --- /dev/null +++ b/content/guides/c4-model/04-container-diagrams.md @@ -0,0 +1,121 @@ +--- +title: Container Diagrams (Level 2) +--- + +# Container Diagrams (Level 2) + +## 1. End-to-End IoT System Architecture + +A **Container diagram** zooms into the systems defined at Level 1 to show the high-level technical architecture. In an Internet of Things (IoT) solution, it captures the whole journey of the data. That runs from the silicon and firmware at the edge, through local networks and cloud microservices, to the user's phone. + +```mermaid +flowchart TB + subgraph Device [Physical Smart Home Sensor Device] + subgraph Hardware & Compute + MCU["Sensor Microcontroller (ESP32-S3)
[Container: Dual-Core Xtensa @ 240MHz]
Executes sensor sampling, Wi-Fi networking, and high-level firmware application."] + SensorIC["Environmental Sensor IC (BME688)
[Container: Dedicated Sensor Silicon]
Measures temperature, pressure, humidity, and gas VOC levels."] + end + subgraph Storage & Security + FlashMem["SPI NOR Flash (4 MB)
[Container: Partitioned Storage]
Dual-bank FOTA partitions (slot0/slot1) and LittleFS configuration storage."] + eFuseStore["Cryptographic eFuses
[Container: On-Chip Hardware Key Storage]
Holds AES device identity keys and secure boot signature digests."] + end + end + + subgraph Cloud [Smart Home Cloud Platform] + MQTTBroker["IoT MQTT Broker Cluster
[Container: EMQX / Mosquitto]
Terminates TLS 1.3 connections and routes incoming telemetry topics."] + IngestionWorker["Telemetry Ingestion Worker
[Container: Go Microservice]
Validates telemetry schemas, enriches records, and writes time-series samples."] + DeviceRegistry["Device Registry & Shadow Service
[Container: Python / FastAPI Microservice]
Tracks device onboarding, desired/reported state shadows, and FOTA targets."] + RulesEngine["Alert & Rule Engine
[Container: Node.js Microservice]
Evaluates air quality thresholds and triggers push notifications."] + TimescaleDB["Time-Series Database
[Container: TimescaleDB / PostgreSQL]
Stores timestamped temperature, humidity, and VOC data points."] + AppAPI["Customer API Gateway
[Container: GraphQL / Node.js Service]
Provides authenticated query endpoints for mobile applications."] + end + + subgraph Client [User Mobile Environment] + MobileApp["Smart Home Mobile Application
[Container: Flutter / iOS & Android]
Displays real-time sensor dials, historical air quality charts, and alert settings."] + end + + SensorIC -- "Raw sensor readings
[I2C @ 400 kHz]" --> MCU + MCU -- "Read/Write configs and FOTA slots
[Quad-SPI @ 40 MHz]" --> FlashMem + MCU -- "Validate boot signatures
[Hardware Memory Bus]" --> eFuseStore + + MCU -- "Publishes telemetry & heartbeats
[MQTT over TLS / Wi-Fi]" --> MQTTBroker + MQTTBroker -- "Streams raw telemetry events
[Kafka / Internal Event Queue]" --> IngestionWorker + IngestionWorker -- "Writes structured metrics
[SQL / Batch Insert]" --> TimescaleDB + IngestionWorker -- "Publishes metric updates
[Internal Event Bus]" --> RulesEngine + + AppAPI -- "Reads historical telemetry
[SQL Query]" --> TimescaleDB + AppAPI -- "Reads/writes device metadata
[gRPC]" --> DeviceRegistry + MobileApp -- "Queries dashboard & sensor history
[HTTPS / GraphQL & WebSockets]" --> AppAPI +``` +[[CAPTION:Figure]] End-to-end Container diagram (Level 2) showing the physical device, cloud microservices, and mobile client app. + +## 2. Matching PlantUML C4 Container Diagram + +The following **PlantUML C4 Container diagram** represents the identical architecture using standard C4 elements and custom tags: + +````plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Container Diagram: Smart Home Environmental Sensor Solution + +System_Boundary(device, "Physical Sensor Device") { + Container(mcu, "Sensor Microcontroller", "ESP32-S3 Dual-Core @ 240MHz", "Executes sensor sampling, Wi-Fi networking, and high-level firmware application.") + Container(sensor_ic, "Environmental Sensor IC", "Bosch BME688 Silicon", "Measures ambient temperature, humidity, atmospheric pressure, and VOC gas index.") + ContainerDb(flash, "SPI NOR Flash Memory", "4 MB SPI Partition", "Stores dual-bank FOTA firmware images (slot0/slot1) and LittleFS configuration store.") + ContainerDb(efuse, "Hardware eFuses", "On-Chip OTP Memory", "Stores factory cryptographic keys, device MAC, and Secure Boot hashes.") +} + +System_Boundary(cloud, "Smart Home Cloud Platform") { + Container(mqtt, "IoT MQTT Broker Cluster", "EMQX / Mosquitto", "Terminates mTLS sessions and ingests real-time sensor telemetry topics.") + Container(ingest, "Telemetry Ingestion Worker", "Go Microservice", "Validates binary/JSON telemetry payloads and writes time-series records.") + Container(registry, "Device Registry & Shadow", "Python / FastAPI", "Manages device activation, configuration shadows, and FOTA campaigns.") + Container(rules, "Alert & Rules Engine", "Node.js Microservice", "Evaluates indoor air quality thresholds and fires threshold alerts.") + ContainerDb(tsdb, "Time-Series Database", "TimescaleDB / PostgreSQL", "Stores compressed historical environmental time-series data.") + Container(api, "Customer API Gateway", "Node.js / GraphQL", "Exposes customer-facing endpoints for mobile dashboards and charts.") +} + +Person(user, "Homeowner / Resident", "Monitors air quality via smartphone.") +Container(mobile, "Smart Home Mobile App", "Flutter / iOS & Android", "Renders interactive dashboard dials, historical graphs, and air alerts.") + +Rel(sensor_ic, mcu, "Samples raw telemetry", "I2C @ 400 kHz") +Rel(mcu, flash, "Reads/writes config & FOTA blocks", "Quad-SPI @ 40 MHz") +Rel(mcu, efuse, "Validates secure boot signature", "Internal Bus") + +Rel(mcu, mqtt, "Publishes telemetry and alert frames", "MQTT over TLS / Wi-Fi") +Rel(mqtt, ingest, "Dispatches raw telemetry stream", "Internal Queue") +Rel(ingest, tsdb, "Inserts telemetry records", "SQL / TimescaleDB") +Rel(ingest, rules, "Sends metric events", "Internal Bus") + +Rel(api, tsdb, "Queries telemetry trends", "SQL Query") +Rel(api, registry, "Queries device shadow state", "Internal gRPC") +Rel(mobile, api, "Queries dashboards & sends settings", "HTTPS / GraphQL") +Rel(user, mobile, "Interacts with mobile UI", "Touch") + +@enduml +```` +[[CAPTION:Figure]] PlantUML C4 Container diagram for the Smart Home IoT ecosystem. + +## 3. Container Classifications + +A clean Level 2 diagram categorises containers by their functional layer: + +| Container | Category | Technology | Responsibility | +| :--- | :--- | :--- | :--- | +| **Sensor Microcontroller** | Edge Compute | ESP32-S3 (Xtensa Dual-Core) | Runs FreeRTOS, sensor drivers, Wi-Fi stack, and application tasks. | +| **Environmental Sensor IC** | Edge Sensor | Bosch BME688 / I2C | Gathers raw analog/digital temperature, humidity, and gas resistance. | +| **Flash Memory Partition** | Edge Storage | SPI NOR Flash (LittleFS) | Retains calibration curves, Wi-Fi credentials, and FOTA staging slots. | +| **IoT MQTT Broker** | Cloud Ingestion | EMQX Cluster / Mosquitto | Handles hundreds of thousands of concurrent TLS socket connections. | +| **Ingestion Worker** | Cloud Processing | Go / Goroutines | High-throughput stream processing validating incoming telemetry. | +| **TimescaleDB** | Cloud Storage | TimescaleDB (PostgreSQL) | Hypertables optimised for high-rate append-only time-series data. | +| **Customer API Gateway** | Cloud API | GraphQL / Node.js / Express | Serves user dashboard requests, user authentication, and mobile tokens. | +| **Mobile Application** | Client UI | Flutter / Dart | Native iOS and Android application displaying real-time graphs and air alerts. | +[[CAPTION:Table]] Container definitions and technologies for the Smart Home IoT solution. + +## 4. Tips & Tricks for Level 2 Diagrams + +1. **Avoid Over-Splitting Microcontroller Silicon**: Treat the microcontroller running the primary firmware as one execution container. Do not create separate containers for internal RAM, internal registers, or internal timers. +1. **Show Storage Containers Clearly**: Any component that persists data across reboots (such as external SPI flash, LittleFS partitions, or cloud databases) should be represented using container database shapes (`ContainerDb`). +1. **Document Data Protocol Formats**: State both the transport and the serialisation format on communication lines (for example, `[MQTT over TLS / JSON payload]` or `[HTTPS / GraphQL]`). diff --git a/content/guides/c4-model/05-component-diagrams.md b/content/guides/c4-model/05-component-diagrams.md new file mode 100644 index 0000000..4795fb3 --- /dev/null +++ b/content/guides/c4-model/05-component-diagrams.md @@ -0,0 +1,196 @@ +--- +title: Component Diagrams (Level 3) +--- + +# Component Diagrams (Level 3) + +## 1. Decomposing Containers into Components + +A **Component diagram** zooms into an individual container (from Level 2) to reveal its internal modular software architecture. + +In connected systems, components exist both at the physical edge and in the cloud. In this chapter, we explore two complementary Level 3 Component views: +1. **The High-Level Firmware Component Architecture** inside the physical Sensor Microcontroller container. +2. **The Cloud Telemetry Ingestion Worker** inside the cloud microservices layer. + +## 2. High-Level Firmware Component Architecture + +Within the microcontroller firmware (executing on an RTOS like FreeRTOS or Zephyr), components represent **cohesive functional modules, subsystems, or background tasks** connected by message queues and function interfaces: + +```mermaid +flowchart TB + subgraph Firmware [Container: Sensor Microcontroller High-Level Firmware] + subgraph Sensing & Application Layer + SensorMgr["Sensor Acquisition Manager
[Component: Periodic RTOS Task]
Samples the environmental sensor, averages readings, and compensates for thermal drift."] + PowerOrch["Power & Sleep Orchestrator
[Component: State Controller Task]
Coordinates sensor heater duty-cycles, radio sleep states, and battery preservation."] + end + + subgraph Communications & Cloud Integration Layer + NetMgr["Wi-Fi & Network Manager
[Component: Event-Driven Task]
Maintains WPA2/WPA3 associations, performs exponential backoff, and hosts BLE onboarding."] + MQTTClient["MQTT Telemetry Client
[Component: Client Subsystem]
Formats JSON/CBOR telemetry frames, handles QoS 1 deliveries, and manages subscriptions."] + FotaAgent["FOTA Update Agent
[Component: Background Task]
Streams firmware blocks over HTTPS/MQTT, writes to flash staging slot, and verifies image hashes."] + end + + subgraph System Services & Hardware Abstraction + ConfigStore["Local Configuration Store
[Component: LittleFS Storage Subsystem]
Thread-safe access to Wi-Fi credentials, alert thresholds, and calibration offsets."] + SensorDriver["BME688 Sensor Driver & HAL
[Component: I2C Driver Layer]
Configures gas sensor heating profiles and reads raw ADC sample registers."] + end + end + + %% External interfaces + SensorIC[("Environmental Sensor IC")] + NORFlash[("SPI NOR Flash")] + CloudBroker[("Cloud MQTT Broker")] + + SensorDriver -- "I2C register transfers" --> SensorIC + ConfigStore -- "Block read/write" --> NORFlash + FotaAgent -- "Write downloaded image" --> NORFlash + MQTTClient -- "TLS Socket Send/Receive" --> CloudBroker + + %% Internal interactions + SensorMgr -- "Read temperature, humidity, VOCs" --> SensorDriver + SensorMgr -- "Dispatch metric bundle
[RTOS Message Queue]" --> MQTTClient + SensorMgr -- "Read calibration data" --> ConfigStore + + PowerOrch -- "Schedule measurement ticks" --> SensorMgr + PowerOrch -- "Trigger radio sleep mode" --> NetMgr + + NetMgr -- "Network available notification" --> MQTTClient + MQTTClient -- "Forward FOTA payload notification" --> FotaAgent + MQTTClient -- "Update remote threshold settings" --> ConfigStore +``` +[[CAPTION:Figure]] High-level firmware Component diagram (Level 3) for the Smart Home Sensor microcontroller. + +## 3. Matching PlantUML C4 Firmware Component Diagram + +Below is the matching **PlantUML C4 Component diagram** detailing the firmware subsystems using `C4_Component.puml`: + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Component Diagram: Sensor Microcontroller Firmware + +Container_Boundary(fw, "Sensor Microcontroller Firmware (FreeRTOS)") { + Component(sens_mgr, "Sensor Acquisition Manager", "C Task", "Coordinates sensor sampling schedule, averages data, and calculates air comfort scores.") + Component(power, "Power & Sleep Orchestrator", "C State Machine", "Manages ultra-low-power sleep states, timer wakeups, and peripheral power rails.") + Component(net, "Wi-Fi & Network Manager", "C Task", "Maintains Wi-Fi association, manages backoff reconnects, and handles BLE provisioning.") + Component(mqtt, "MQTT Telemetry Client", "C Subsystem", "Serialises telemetry to JSON/CBOR, publishes metrics, and processes inbound cloud commands.") + Component(fota, "FOTA Update Agent", "C Task", "Streams firmware update blocks from cloud, validates cryptographic hashes, and marks boots.") + Component(cfg, "Local Configuration Store", "LittleFS Manager", "Provides persistent key-value storage for Wi-Fi credentials and threshold configs.") + Component(drv, "BME688 Driver & I2C HAL", "Driver Layer", "Drives I2C peripheral hardware and controls sensor heating plates.") +} + +ContainerDb_Ext(flash, "SPI NOR Flash", "External Memory", "Stores LittleFS partition and staging FOTA slot.") +Container_Ext(sensor_ic, "BME688 Sensor", "Hardware IC", "Physical ambient sensor hardware.") +Container_Ext(broker, "Cloud MQTT Broker", "Cloud Gateway", "Ingests telemetry and issues commands.") + +Rel(sens_mgr, drv, "Triggers measurement & reads registers", "C Function API") +Rel(drv, sensor_ic, "Reads gas & climate registers", "I2C Bus @ 400 kHz") +Rel(sens_mgr, mqtt, "Queues telemetry packet", "RTOS Message Queue") +Rel(sens_mgr, cfg, "Fetches calibration curve", "C API") + +Rel(power, sens_mgr, "Signals sample intervals", "RTOS Event Flag") +Rel(power, net, "Signals sleep/wake windows", "RTOS Event Flag") + +Rel(net, mqtt, "Notifies network connectivity status", "Event Callback") +Rel(mqtt, broker, "Publishes telemetry & receives topics", "TLS Socket") +Rel(mqtt, fota, "Signals new firmware available", "RTOS Queue") +Rel(mqtt, cfg, "Persists updated setpoints", "C API") + +Rel(fota, flash, "Writes downloaded image blocks", "SPI Block Driver") +Rel(cfg, flash, "Reads/writes configuration records", "LittleFS API") + +@enduml +``` +[[CAPTION:Figure]] PlantUML C4 Component diagram for the sensor firmware. + +## 4. Cloud Telemetry Ingestion Worker Component Architecture + +Zooming into the cloud tier, the **Telemetry Ingestion Worker** container consumes raw MQTT streams, validates payload signatures, applies calibration curves, and batches samples into the time-series store: + +```mermaid +flowchart TB + subgraph WorkerContainer["Container: Telemetry Ingestion Worker (Go Service)"] + Consumer["MQTT Stream Consumer
[Component: Worker Goroutine]
Subscribes to broker queue and deserialises protobuf/JSON frames."] + Validator["Schema & Signature Validator
[Component: Go Package]
Verifies HMAC-SHA256 device signatures and schema compatibility."] + Calibrator["Metric Normaliser & Calibrator
[Component: Pipeline Stage]
Applies sensor altitude offsets and converts raw VOC resistance to IAQ index."] + BatchWriter["Time-Series Batch Buffer
[Component: Ring Buffer]
Aggregates measurements and executes bulk COPY inserts into database."] + ShadowSyncer["Device Shadow Syncer
[Component: gRPC Client]
Updates device connectivity status, last-seen timestamps, and battery levels."] + end + + BrokerQueue[("Cloud MQTT Broker / Message Queue")] + TimescaleDB[("TimescaleDB Telemetry Store")] + RegistrySvc[("Device Registry Service")] + + BrokerQueue -->|"Delivers raw telemetry stream
[Kafka / AMQP Protocol]"| Consumer + Consumer -->|"Passes unvalidated frame
[Go Channel]"| Validator + Validator -->|"Passes verified payload
[Go Channel]"| Calibrator + Calibrator -->|"Appends calibrated record
[In-Memory Buffer]"| BatchWriter + Calibrator -->|"Dispatches device heartbeat
[Asynchronous Channel]"| ShadowSyncer + + BatchWriter -->|"Executes batch SQL insert
[SQL / pgxpool Native TCP]"| TimescaleDB + ShadowSyncer -->|"Updates reported shadow state
[gRPC over HTTP/2]"| RegistrySvc +``` +[[CAPTION:Figure]] High-level cloud component diagram (Level 3) for the Telemetry Ingestion Worker microservice. + +## 5. Matching PlantUML C4 Cloud Component Diagram + +Below is the matching **PlantUML C4 Component diagram** for the cloud Telemetry Ingestion Worker service using `C4_Component.puml`: + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Component Diagram: Cloud Telemetry Ingestion Worker + +Container_Boundary(worker, "Telemetry Ingestion Worker (Go Microservice)") { + Component(consumer, "MQTT Stream Consumer", "Go Goroutine Pool", "Subscribes to internal broker queues and unmarshals telemetry payloads.") + Component(validator, "Schema & Security Validator", "Go Package", "Validates payload structure and authenticates cryptographic device tokens.") + Component(calibrator, "Metric Normaliser & Calibrator", "Pipeline Module", "Applies altitude corrections and converts raw VOC gas sensor readings.") + Component(batcher, "Time-Series Batch Writer", "Buffer Pool", "Buffers samples in memory and executes bulk SQL copy operations.") + Component(syncer, "Device Shadow Syncer", "gRPC Client", "Synchronises device battery, firmware version, and last-seen state.") +} + +Container_Ext(broker, "Cloud Ingestion Broker", "EMQX / AWS IoT", "Dispatches device messages to worker queue.") +ContainerDb_Ext(tsdb, "Telemetry Time-Series Store", "TimescaleDB", "Stores time-series metrics.") +Container_Ext(registry, "Device Registry Service", "Node.js Microservice", "Maintains device shadow digital twins.") + +Rel(broker, consumer, "Streams incoming telemetry", "AMQP / Kafka Queue") +Rel(consumer, validator, "Passes raw payload", "Go Channel") +Rel(validator, calibrator, "Passes validated data", "Go Channel") +Rel(calibrator, batcher, "Appends processed sample", "In-Memory Ring Buffer") +Rel(calibrator, syncer, "Signals device heartbeat", "Go Channel") + +Rel(batcher, tsdb, "Executes multi-row batch insert", "SQL / Native TCP") +Rel(syncer, registry, "Updates reported shadow state", "gRPC over HTTP/2") + +@enduml +``` +[[CAPTION:Figure]] PlantUML C4 Component diagram for the cloud Telemetry Ingestion Worker microservice. + +## 6. Component Concurrency & Execution Mapping + +In connected architectures, defining how components map to execution units prevents concurrency bottlenecks and race conditions: + +| Component | Execution Context | Priority / Scaling | Concurrency Mechanism | +| :--- | :--- | :--- | :--- | +| **Sensor Acquisition Manager** (Firmware) | Periodic RTOS Task | High | Wakes on timer tick; blocks while awaiting I2C completion. | +| **Power & Sleep Orchestrator** (Firmware) | Event-Driven Task | Highest | Coordinates sleep entry when all task queues are empty. | +| **Wi-Fi & Network Manager** (Firmware) | System Event Task | Normal | Handles network state transitions and IP DHCP renewal. | +| **MQTT Telemetry Client** (Firmware) | Networking Task | Normal | Consumes from telemetry queue; blocks on network sockets. | +| **FOTA Update Agent** (Firmware) | Background Task | Lowest | Executes cooperatively; throttled to avoid starving sensing. | +| **BME688 Driver & HAL** (Firmware) | Synchronous Library | Caller Context | Mutex-protected to prevent concurrent I2C bus collisions. | +| **MQTT Stream Consumer** (Cloud) | Goroutine Pool (50 workers) | Autoscaling Pod | Worker pool reading concurrently from partitioned queue. | +| **Time-Series Batch Writer** (Cloud) | Scheduled Flush Goroutine | High Throughput | Flushes ring buffer on 1-second interval or 500-sample limit. | +| **Device Shadow Syncer** (Cloud) | Non-Blocking Goroutine | Normal | Asynchronous gRPC dispatch to prevent blocking ingestion pipeline. | +[[CAPTION:Table]] Concurrency mapping for firmware tasks and cloud ingestion components. + +## 7. Tips & Tricks for Level 3 Diagrams + +1. **Avoid Over-Diagramming Procedural Utilities**: Do not create components for generic helper utilities like `crc32.c`, `string_utils.c`, or standard math libraries. Focus on cohesive subsystems that own state, background tasks, or worker pools. +1. **Show Concurrency Interactions**: Always specify whether communication occurs via function calls, RTOS message queues, event flags, Go channels, or gRPC streams. +1. **Model Both Edge and Cloud Separately**: Maintain dedicated Level 3 Component diagrams for each major container. Do not merge firmware task diagrams and cloud microservice component diagrams into a single confusing drawing. diff --git a/content/guides/c4-model/06-structurizr-dsl-reference.md b/content/guides/c4-model/06-structurizr-dsl-reference.md new file mode 100644 index 0000000..8332734 --- /dev/null +++ b/content/guides/c4-model/06-structurizr-dsl-reference.md @@ -0,0 +1,634 @@ +--- +title: Structurizr DSL Reference for Connected Systems +--- + +# Structurizr DSL Reference for Connected Systems + +## 1. Why Structurizr DSL for IoT Architectures? + +When designing connected systems that span physical edge devices, microcontrollers, radio gateways, and cloud microservices, authoring standalone visual diagrams quickly becomes unmaintainable. An update to an MQTT topic format, an ingestion endpoint, or a device hardware component often requires manually redrawing half a dozen diagrams across multiple levels. + +The **Structurizr DSL** (Domain Specific Language) solves this problem by enforcing a **single source of truth**: + +1. **Model Once, View Anywhere**: You define people, software systems, containers, components, deployment nodes, and their relationships once in an architectural model. +1. **Multiple Perspectives**: You declare views (Context, Container, Component, Deployment, Dynamic) that automatically project subsets of that same unified model. +1. **Format Agnostic**: From one `workspace.dsl` file, you can render interactive web views via **Structurizr Lite**, generate **PlantUML C4 diagrams**, or export **Mermaid** graphics for documentation. + +```mermaid +graph LR + DSL["workspace.dsl
Single Architectural Model
(Sensors, MCU, Cloud, DBs)"] + Lite["Structurizr Lite
Interactive Web UI"] + Puml["PlantUML Export
C4 Container & Component Diagrams"] + Merm["Mermaid Export
Git Markdown Rendering"] + PDF["DOCX & PDF Guides
Createdocs Automation"] + + DSL --> Lite + DSL --> Puml + DSL --> Merm + Puml --> PDF +``` +[[CAPTION:Figure]] The Structurizr DSL architecture-as-code workflow for connected IoT systems. + +## 2. Complete Smart Home IoT `workspace.dsl` + +Below is a complete `workspace.dsl` file that models the whole Smart Home Environmental Monitoring system. It runs from the physical sensor and MCU firmware components to the cloud microservices and the mobile app. + +```structurizr +workspace "Smart Home Environmental Monitoring System" "End-to-end architecture model from edge sensor firmware to cloud microservices and mobile user experience." { + + model { + # ===================================================================== + # People / User Personas + # ===================================================================== + homeowner = person "Homeowner" "Monitors home air quality and receives comfort alerts." "Person" + installer = person "Field Technician" "Commissions, provisions, and diagnostics sensors during installation." "Person" + + # ===================================================================== + # External Systems + # ===================================================================== + localHub = softwareSystem "Local Smart Hub" "Local home automation gateway (for example, Home Assistant, Apple Home, Matter hub)." "External" + weatherService = softwareSystem "National Weather API" "Provides external outdoor baseline temperature and pollen data." "External" + pushService = softwareSystem "Push Notification Service" "Apple APNs and Google FCM delivery network." "External" + + # ===================================================================== + # Connected Edge Product: Smart Environmental Sensor + # ===================================================================== + sensorDevice = softwareSystem "Smart Environmental Sensor" "Battery-powered physical IoT edge device measuring ambient environment." "Device" { + + sensorHardware = container "BME688 Sensor Subsystem" "Digital MEMS sensor measuring temperature, humidity, barometric pressure, and VOC gas." "Silicon / MEMS" "Sensor" + + configStorage = container "Non-Volatile Storage (NVS)" "Encrypted SPI NOR flash memory storing Wi-Fi credentials, certificates, and offline logs." "Macronix SPI Flash" "Database" + + sensorFirmware = container "Sensor Application Firmware" "FreeRTOS-based embedded firmware managing data acquisition, power cycling, and networking." "C / FreeRTOS on STM32U575" "Firmware" { + + sensorAcq = component "Sensor Acquisition Manager" "Schedules periodic sensor readings, applies calibration offsets, and filters noise." "C Module / RTOS Task" "FirmwareComponent" + powerOrch = component "Power & Sleep Orchestrator" "Coordinates transitions between active measurement, network transmission, and deep sleep." "C Module / State Machine" "FirmwareComponent" + wifiMgr = component "Wi-Fi & Network Manager" "Manages 802.11 b/g/n association, station state, and network recovery." "C Module / Network Task" "FirmwareComponent" + mqttClient = component "MQTT Ingestion Client" "Manages secure mTLS connections and serialises JSON telemetry payloads." "C Module / coreMQTT" "FirmwareComponent" + fotaAgent = component "FOTA Update Agent" "Downloads signed firmware chunks, validates SHA-256 signatures, and writes to staging flash." "C Module / Bootloader Interface" "FirmwareComponent" + configStore = component "Configuration Store" "Provides thread-safe access to provisioned settings and network credentials." "C Module / LittleFS HAL" "FirmwareComponent" + sensorHal = component "Sensor HAL & Driver" "I2C peripheral driver communicating directly with the BME688 hardware registers." "C Module / I2C Bus Driver" "FirmwareComponent" + } + } + + # ===================================================================== + # Cloud Platform + # ===================================================================== + cloudPlatform = softwareSystem "Environmental Cloud Platform" "Scalable cloud platform for device ingestion, shadow management, and telemetry analytics." "Cloud" { + + mqttBroker = container "Cloud Ingestion Broker" "High-throughput MQTT broker terminating device TLS connections and routing messages." "EMQX / AWS IoT Core" "CloudService" + + ingestionWorker = container "Telemetry Ingestion Worker" "Consumes raw telemetry messages, validates schema, applies calibrations, and writes time-series records." "Go Microservice" "CloudService" { + streamConsumer = component "MQTT Stream Consumer" "Consumes raw telemetry events from message broker queue." "Go Goroutine Pool" "CloudComponent" + payloadValidator = component "Schema & Security Validator" "Validates payload format and authenticates device HMAC signatures." "Go Package" "CloudComponent" + metricCalibrator = component "Metric Normaliser & Calibrator" "Applies sensor altitude offsets and converts VOC resistance to IAQ index." "Pipeline Module" "CloudComponent" + batchWriter = component "Time-Series Batch Writer" "Buffers samples and executes bulk SQL copy operations into database." "Buffer Pool" "CloudComponent" + shadowSyncer = component "Device Shadow Syncer" "Synchronises reported battery, firmware version, and last-seen state." "gRPC Client" "CloudComponent" + } + + deviceRegistry = container "Device Shadow & Registry Service" "Maintains device digital twins, provisioning credentials, desired state, and FOTA campaigns." "Node.js / TypeScript" "CloudService" + + telemetryDb = container "Telemetry Time-Series Store" "Optimised time-series database storing sensor metrics and historical aggregates." "TimescaleDB / Timeseries" "Database" + + metadataDb = container "Relational Metadata Database" "Stores user accounts, home floor plans, device ownership, and alerting rules." "PostgreSQL 16" "Database" + + graphqlApi = container "GraphQL Application Gateway" "Serves unified queries and real-time WebSockets subscriptions to clients." "Node.js / Apollo Server" "CloudService" + } + + # ===================================================================== + # User Client Applications + # ===================================================================== + mobileApp = softwareSystem "Smart Life Mobile App" "Cross-platform mobile application providing real-time telemetry, historical charts, and push alerts." "MobileApp" + + # ===================================================================== + # System-Level Relationships (Level 1) + # ===================================================================== + homeowner -> mobileApp "Views live metrics, trends, and alerts" "HTTPS" + homeowner -> sensorDevice "Views RGB status LED and presses commissioning button" "Physical / Visual" + installer -> sensorDevice "Provisions Wi-Fi credentials and inspects diagnostics" "BLE 5.2" + + sensorDevice -> cloudPlatform "Publishes periodic telemetry and checks for updates" "MQTT over TLS 1.3 / Port 8883" + sensorDevice -> localHub "Broadcasts local metrics and status" "Matter over Wi-Fi / CoAP" + cloudPlatform -> pushService "Dispatches push alert payloads" "HTTPS / REST" + pushService -> mobileApp "Delivers push notifications" "Apple APNs / Google FCM" + cloudPlatform -> weatherService "Fetches outdoor baseline temperature and air quality" "HTTPS / JSON" + mobileApp -> cloudPlatform "Queries device data, historical metrics, and updates configuration" "GraphQL / HTTPS / WSS" + + # ===================================================================== + # Container-Level Relationships (Level 2) + # ===================================================================== + sensorFirmware -> sensorHardware "Triggers measurement cycles and reads raw register values" "I2C Bus @ 400 kHz" + sensorFirmware -> configStorage "Reads/writes credentials, crash dumps, and staged firmware" "SPI Bus @ 32 MHz" + sensorFirmware -> mqttBroker "Publishes telemetry and subscribes to delta updates" "MQTT over TLS 1.3 (mTLS)" + sensorFirmware -> localHub "Emits local telemetry" "Matter / UDP" + + mqttBroker -> ingestionWorker "Routes incoming telemetry messages" "Internal Queue / Kafka" + ingestionWorker -> telemetryDb "Writes batch time-series sensor samples" "SQL / Native TCP" + ingestionWorker -> deviceRegistry "Updates last-seen timestamp and battery state" "gRPC" + + deviceRegistry -> metadataDb "Reads and writes device profiles and ownership records" "SQL" + graphqlApi -> telemetryDb "Queries metric history and rollups" "SQL" + graphqlApi -> deviceRegistry "Reads device shadow and dispatches desired state updates" "gRPC" + graphqlApi -> metadataDb "Authenticates users and reads home layouts" "SQL" + + mobileApp -> graphqlApi "Fetches dashboard metrics and sends remote commands" "HTTPS / WSS" + + # ===================================================================== + # Component-Level Relationships inside Firmware (Level 3) + # ===================================================================== + powerOrch -> sensorAcq "Wakes and triggers scheduled reading" "Direct Call / Event Signal" + sensorAcq -> sensorHal "Reads compensated sensor values" "C API Call" + sensorHal -> sensorHardware "Reads data registers" "I2C HAL Driver" + + sensorAcq -> mqttClient "Enqueues telemetry payload" "FreeRTOS Queue" + powerOrch -> wifiMgr "Commands network connect/disconnect for power conservation" "RTOS Event Group" + wifiMgr -> mqttClient "Notifies IP address acquisition and link state" "Event Notification" + + mqttClient -> configStore "Retrieves client TLS certificates and broker endpoint" "C API Call" + configStore -> configStorage "Reads stored keys and nonces" "LittleFS File IO" + + fotaAgent -> mqttClient "Subscribes to update jobs and downloads signed chunks" "MQTT Callback" + fotaAgent -> configStorage "Writes firmware image to staging partition" "Flash Driver Call" + fotaAgent -> powerOrch "Requests system reboot into bootloader" "System Reset" + + # ===================================================================== + # Component-Level Relationships inside Cloud Ingestion Worker (Level 3) + # ===================================================================== + mqttBroker -> streamConsumer "Routes raw messages" "AMQP / Kafka Queue" + streamConsumer -> payloadValidator "Passes raw payload" "Go Channel" + payloadValidator -> metricCalibrator "Passes validated data" "Go Channel" + metricCalibrator -> batchWriter "Appends processed sample" "In-Memory Ring Buffer" + metricCalibrator -> shadowSyncer "Signals device heartbeat" "Go Channel" + batchWriter -> telemetryDb "Executes batch insert" "SQL / Native TCP" + shadowSyncer -> deviceRegistry "Updates reported shadow state" "gRPC over HTTP/2" + + # ===================================================================== + # Deployment Environment: Physical Smart Home Deployment + # ===================================================================== + deploymentEnvironment "Production" { + deploymentNode "Residential Smart Home" "Physical residence of the homeowner." "Physical Boundary" { + + deploymentNode "Living Room Wall Mount" "Physical wall installation point." "Hardware Enclosure" { + deploymentNode "PCB Assembly" "Custom 4-layer printed circuit board." "Hardware" { + deploymentNode "STM32U575 MCU" "ARM Cortex-M33 running at 160MHz with TrustZone." "Microcontroller" { + containerInstance sensorFirmware + } + deploymentNode "Bosch BME688" "Integrated environmental sensor in 3.0 x 3.0 mm package." "Hardware IC" { + containerInstance sensorHardware + } + deploymentNode "Macronix MX25R 8MB" "External NOR flash memory chip." "Hardware IC" { + containerInstance configStorage + } + } + } + + deploymentNode "Home Wi-Fi Router" "Standard dual-band residential wireless gateway." "Network Device" { + deploymentNode "Local Home Server" "Raspberry Pi 5 home server." "Hardware" { + softwareSystemInstance localHub + } + } + + deploymentNode "User Smartphone" "Apple iPhone 15 running iOS 18." "Mobile Device" { + softwareSystemInstance mobileApp + } + } + + deploymentNode "AWS Cloud Region (eu-west-1)" "Amazon Web Services Ireland data centre." "Cloud Infrastructure" { + deploymentNode "AWS IoT Core" "Managed cloud MQTT message broker and TLS gateway." "AWS Service" { + containerInstance mqttBroker + } + deploymentNode "Amazon Elastic Kubernetes Service (EKS)" "Managed container orchestration cluster." "Kubernetes" { + deploymentNode "Telemetry Ingestion Pods" "Auto-scaling worker pods." "Container Runtime" { + containerInstance ingestionWorker + } + deploymentNode "Device Registry Pods" "Stateful management pods." "Container Runtime" { + containerInstance deviceRegistry + } + deploymentNode "GraphQL API Pods" "Public gateway service pods." "Container Runtime" { + containerInstance graphqlApi + } + } + deploymentNode "Managed Database Tier" "High-availability managed data tier." "Cloud Database" { + deploymentNode "Amazon RDS PostgreSQL" "Multi-AZ relational database." "Database Server" { + containerInstance metadataDb + } + deploymentNode "Amazon Timestream / TimescaleDB" "Scalable time-series storage engine." "Database Server" { + containerInstance telemetryDb + } + } + } + } + } + + # ========================================================================= + # Views Definition + # ========================================================================= + views { + systemContext sensorDevice "SystemContext_Overview" "Overall system context showing device, users, cloud platform, and third-party integrations." { + include * + autoLayout lr + } + + container sensorDevice "Container_EdgeDevice" "Internal containers of the Smart Environmental Sensor edge device." { + include * + autoLayout tb + } + + container cloudPlatform "Container_CloudPlatform" "Internal microservices and database containers of the Cloud Platform." { + include * + autoLayout tb + } + + component sensorFirmware "Component_FirmwareArchitecture" "Component-level architecture of the FreeRTOS embedded application firmware." { + include * + autoLayout tb + } + + component ingestionWorker "Component_CloudIngestionWorker" "Component-level architecture of the Go Telemetry Ingestion Worker." { + include * + autoLayout tb + } + + deployment sensorDevice "Production" "Deployment_ProductionTopology" "Physical hardware and cloud infrastructure deployment mapping." { + include * + autoLayout tb + } + + # ===================================================================== + # Visual Styling and Tag Configurations + # ===================================================================== + styles { + element "Person" { + shape Person + background #08427b + color #ffffff + } + element "Software System" { + background #1168bd + color #ffffff + } + element "Device" { + shape RoundedBox + background #2A6F97 + color #ffffff + } + element "Cloud" { + shape RoundedBox + background #014F86 + color #ffffff + } + element "External" { + background #999999 + color #ffffff + } + element "MobileApp" { + shape MobileDevicePortrait + background #0077B6 + color #ffffff + } + element "Container" { + background #438dd5 + color #ffffff + } + element "Firmware" { + shape RoundedBox + background #1E88E5 + color #ffffff + } + element "Sensor" { + shape Hexagon + background #00897B + color #ffffff + } + element "Database" { + shape Cylinder + background #2E7D32 + color #ffffff + } + element "CloudService" { + shape RoundedBox + background #1565C0 + color #ffffff + } + element "FirmwareComponent" { + shape Component + background #5C6BC0 + color #ffffff + } + element "CloudComponent" { + shape Component + background #0288D1 + color #ffffff + } + relationship "Relationship" { + thickness 2 + dashed false + color #707070 + fontSize 24 + } + } + } +} +``` +[[CAPTION:Figure]] Complete Structurizr DSL specification (`workspace.dsl`) for the Smart Home Environmental Monitoring system. + +## 3. Matching PlantUML C4 Diagrams for DSL Views + +When exporting or pairing the Structurizr model with standard PlantUML, each view declared in `workspace.dsl` generates a corresponding PlantUML C4 diagram. Below are the complete matching PlantUML scripts for all six declared views: + +### A. View 1: System Context Overview (`SystemContext_Overview.puml`) + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title System Context Overview - Smart Environmental Sensor System + +Person(homeowner, "Homeowner", "Monitors home air quality and receives comfort alerts.") +Person(installer, "Field Technician", "Commissions, provisions, and diagnostics sensors during installation.") + +System(sensorDevice, "Smart Environmental Sensor", "Battery-powered physical IoT edge device measuring ambient environment.") +System(cloudPlatform, "Environmental Cloud Platform", "Scalable cloud platform for device ingestion, shadow management, and telemetry analytics.") +System(mobileApp, "Smart Life Mobile App", "Cross-platform mobile application providing real-time telemetry, historical charts, and push alerts.") + +System_Ext(localHub, "Local Smart Hub", "Local home automation gateway (for example, Home Assistant, Matter hub).") +System_Ext(weatherService, "National Weather API", "Provides external outdoor baseline temperature and pollen data.") +System_Ext(pushService, "Push Notification Service", "Apple APNs and Google FCM delivery network.") + +Rel(homeowner, mobileApp, "Views live metrics, trends, and alerts", "HTTPS") +Rel(homeowner, sensorDevice, "Views RGB status LED and presses button", "Physical / Visual") +Rel(installer, sensorDevice, "Provisions Wi-Fi credentials and inspects diagnostics", "BLE 5.2") + +Rel(sensorDevice, cloudPlatform, "Publishes periodic telemetry and checks for updates", "MQTT over TLS 1.3 / Port 8883") +Rel(sensorDevice, localHub, "Broadcasts local metrics and status", "Matter over Wi-Fi / CoAP") +Rel(cloudPlatform, pushService, "Dispatches push alert payloads", "HTTPS / REST") +Rel(pushService, mobileApp, "Delivers push notifications", "Apple APNs / Google FCM") +Rel(cloudPlatform, weatherService, "Fetches outdoor baseline temperature and air quality", "HTTPS / JSON") +Rel(mobileApp, cloudPlatform, "Queries device data, historical metrics, and updates configuration", "GraphQL / HTTPS / WSS") + +@enduml +``` +[[CAPTION:Figure]] Matching PlantUML C4 script for View 1: System Context Overview. + +### B. View 2: Edge Device Container Diagram (`Container_EdgeDevice.puml`) + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Container Diagram - Smart Environmental Sensor (Edge Device) + +System_Boundary(sensorDevice, "Smart Environmental Sensor") { + Container(sensorFirmware, "Sensor Application Firmware", "C / FreeRTOS on STM32U575", "FreeRTOS-based embedded firmware managing data acquisition, power cycling, and networking.") + Container(sensorHardware, "BME688 Sensor Subsystem", "Silicon / MEMS", "Digital MEMS sensor measuring temperature, humidity, barometric pressure, and VOC gas.") + ContainerDb(configStorage, "Non-Volatile Storage (NVS)", "Macronix SPI Flash", "Encrypted SPI NOR flash memory storing Wi-Fi credentials, certificates, and offline logs.") +} + +Container_Ext(mqttBroker, "Cloud Ingestion Broker", "EMQX / AWS IoT Core", "High-throughput MQTT broker terminating device TLS connections.") +System_Ext(localHub, "Local Smart Hub", "Matter over Wi-Fi", "Local home automation gateway.") + +Rel(sensorFirmware, sensorHardware, "Triggers measurement cycles and reads raw register values", "I2C Bus @ 400 kHz") +Rel(sensorFirmware, configStorage, "Reads/writes credentials, crash dumps, and staged firmware", "SPI Bus @ 32 MHz") +Rel(sensorFirmware, mqttBroker, "Publishes telemetry and subscribes to delta updates", "MQTT over TLS 1.3 (mTLS)") +Rel(sensorFirmware, localHub, "Emits local telemetry", "Matter / UDP") + +@enduml +``` +[[CAPTION:Figure]] Matching PlantUML C4 script for View 2: Edge Device Containers. + +### C. View 3: Cloud Platform Container Diagram (`Container_CloudPlatform.puml`) + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Container Diagram - Environmental Cloud Platform + +System_Boundary(cloudPlatform, "Environmental Cloud Platform") { + Container(mqttBroker, "Cloud Ingestion Broker", "EMQX / AWS IoT Core", "High-throughput MQTT broker terminating device TLS connections and routing messages.") + Container(ingestionWorker, "Telemetry Ingestion Worker", "Go Microservice", "Consumes raw telemetry messages, validates schema, applies calibrations, and writes time-series records.") + Container(deviceRegistry, "Device Shadow & Registry Service", "Node.js / TypeScript", "Maintains device digital twins, provisioning credentials, desired state, and FOTA campaigns.") + ContainerDb(telemetryDb, "Telemetry Time-Series Store", "TimescaleDB / Timeseries", "Optimised time-series database storing sensor metrics and historical aggregates.") + ContainerDb(metadataDb, "Relational Metadata Database", "PostgreSQL 16", "Stores user accounts, home floor plans, device ownership, and alerting rules.") + Container(graphqlApi, "GraphQL Application Gateway", "Node.js / Apollo Server", "Serves unified queries and real-time WebSockets subscriptions to clients.") +} + +Container_Ext(sensorFirmware, "Sensor Firmware", "C / FreeRTOS", "Embedded sensor fleet.") +System_Ext(mobileApp, "Smart Life Mobile App", "Flutter Client", "Homeowner user interface.") + +Rel(sensorFirmware, mqttBroker, "Publishes telemetry and subscribes to delta updates", "MQTT over TLS 1.3 (mTLS)") +Rel(mqttBroker, ingestionWorker, "Routes incoming telemetry messages", "Internal Queue / Kafka") +Rel(ingestionWorker, telemetryDb, "Writes batch time-series sensor samples", "SQL / Native TCP") +Rel(ingestionWorker, deviceRegistry, "Updates last-seen timestamp and battery state", "gRPC") + +Rel(deviceRegistry, metadataDb, "Reads and writes device profiles and ownership records", "SQL") +Rel(graphqlApi, telemetryDb, "Queries metric history and rollups", "SQL") +Rel(graphqlApi, deviceRegistry, "Reads device shadow and dispatches desired state updates", "gRPC") +Rel(graphqlApi, metadataDb, "Authenticates users and reads home layouts", "SQL") + +Rel(mobileApp, graphqlApi, "Fetches dashboard metrics and sends remote commands", "HTTPS / WSS") + +@enduml +``` +[[CAPTION:Figure]] Matching PlantUML C4 script for View 3: Cloud Platform Containers. + +### D. View 4: Firmware Component Architecture (`Component_FirmwareArchitecture.puml`) + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Component Diagram - Sensor Application Firmware + +Container_Boundary(sensorFirmware, "Sensor Application Firmware") { + Component(sensorAcq, "Sensor Acquisition Manager", "C Module / RTOS Task", "Schedules periodic sensor readings, applies calibration offsets, and filters noise.") + Component(powerOrch, "Power & Sleep Orchestrator", "C Module / State Machine", "Coordinates transitions between active measurement, network transmission, and deep sleep.") + Component(wifiMgr, "Wi-Fi & Network Manager", "C Module / Network Task", "Manages 802.11 b/g/n association, station state, and network recovery.") + Component(mqttClient, "MQTT Ingestion Client", "C Module / coreMQTT", "Manages secure mTLS connections and serialises JSON telemetry payloads.") + Component(fotaAgent, "FOTA Update Agent", "C Module / Bootloader Interface", "Downloads signed firmware chunks, validates SHA-256 signatures, and writes to staging flash.") + Component(configStore, "Configuration Store", "C Module / LittleFS HAL", "Provides thread-safe access to provisioned settings and network credentials.") + Component(sensorHal, "Sensor HAL & Driver", "C Module / I2C Bus Driver", "I2C peripheral driver communicating directly with the BME688 hardware registers.") +} + +Container_Ext(sensorHardware, "BME688 Sensor Subsystem", "Silicon IC", "Digital MEMS sensor measuring environmental climate.") +ContainerDb_Ext(configStorage, "Non-Volatile Storage (NVS)", "Macronix Flash", "SPI NOR flash memory.") +Container_Ext(mqttBroker, "Cloud Ingestion Broker", "Cloud MQTT", "Ingests telemetry frames.") + +Rel(powerOrch, sensorAcq, "Wakes and triggers scheduled reading", "Direct Call / Event Signal") +Rel(sensorAcq, sensorHal, "Reads compensated sensor values", "C API Call") +Rel(sensorHal, sensorHardware, "Reads data registers", "I2C HAL Driver") + +Rel(sensorAcq, mqttClient, "Enqueues telemetry payload", "FreeRTOS Queue") +Rel(powerOrch, wifiMgr, "Commands network connect/disconnect for power conservation", "RTOS Event Group") +Rel(wifiMgr, mqttClient, "Notifies IP address acquisition and link state", "Event Notification") + +Rel(mqttClient, configStore, "Retrieves client TLS certificates and broker endpoint", "C API Call") +Rel(configStore, configStorage, "Reads stored keys and nonces", "LittleFS File IO") + +Rel(fotaAgent, mqttClient, "Subscribes to update jobs and downloads signed chunks", "MQTT Callback") +Rel(fotaAgent, configStorage, "Writes firmware image to staging partition", "Flash Driver Call") +Rel(fotaAgent, powerOrch, "Requests system reboot into bootloader", "System Reset") + +Rel(mqttClient, mqttBroker, "Publishes telemetry payload", "mTLS Socket") + +@enduml +``` +[[CAPTION:Figure]] Matching PlantUML C4 script for View 4: Firmware Components. + +### E. View 5: Cloud Ingestion Worker Component Architecture (`Component_CloudIngestionWorker.puml`) + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Component Diagram - Telemetry Ingestion Worker + +Container_Boundary(ingestionWorker, "Telemetry Ingestion Worker (Go Service)") { + Component(streamConsumer, "MQTT Stream Consumer", "Go Goroutine Pool", "Consumes raw telemetry events from message broker queue.") + Component(payloadValidator, "Schema & Security Validator", "Go Package", "Validates payload format and authenticates device HMAC signatures.") + Component(metricCalibrator, "Metric Normaliser & Calibrator", "Pipeline Module", "Applies sensor altitude offsets and converts VOC resistance to IAQ index.") + Component(batchWriter, "Time-Series Batch Writer", "Buffer Pool", "Buffers samples and executes bulk SQL copy operations into database.") + Component(shadowSyncer, "Device Shadow Syncer", "gRPC Client", "Synchronises reported battery, firmware version, and last-seen state.") +} + +Container_Ext(mqttBroker, "Cloud Ingestion Broker", "EMQX / AWS IoT", "Dispatches device messages to worker queue.") +ContainerDb_Ext(telemetryDb, "Telemetry Time-Series Store", "TimescaleDB", "Time-series database storing sensor metrics.") +Container_Ext(deviceRegistry, "Device Registry Service", "Node.js Service", "Maintains device digital twins and shadows.") + +Rel(mqttBroker, streamConsumer, "Routes raw messages", "AMQP / Kafka Queue") +Rel(streamConsumer, payloadValidator, "Passes raw payload", "Go Channel") +Rel(payloadValidator, metricCalibrator, "Passes validated data", "Go Channel") +Rel(metricCalibrator, batchWriter, "Appends processed sample", "In-Memory Ring Buffer") +Rel(metricCalibrator, shadowSyncer, "Signals device heartbeat", "Go Channel") + +Rel(batchWriter, telemetryDb, "Executes batch insert", "SQL / Native TCP") +Rel(shadowSyncer, deviceRegistry, "Updates reported shadow state", "gRPC over HTTP/2") + +@enduml +``` +[[CAPTION:Figure]] Matching PlantUML C4 script for View 5: Cloud Ingestion Worker Components. + +### F. View 6: Physical and Cloud Deployment Topology (`Deployment_ProductionTopology.puml`) + +```plantuml +@startuml +!include + +LAYOUT_WITH_LEGEND() + +title Deployment Diagram - Production Hardware & Cloud Topology + +Deployment_Node(home, "Residential Smart Home", "Customer Site") { + Deployment_Node(wall, "Living Room Wall Mount", "Polycarbonate Enclosure") { + Deployment_Node(pcb, "PCB Assembly", "4-Layer FR-4 Board") { + Deployment_Node(mcu, "STM32U575 MCU", "ARM Cortex-M33 @ 160MHz") { + Container(sensorFirmware, "Sensor Firmware", "C / FreeRTOS", "Application image") + } + Deployment_Node(sensor_ic, "Bosch BME688", "3.0 x 3.0 mm LGA Package") { + Container(sensorHardware, "BME688 Sensor", "MEMS Silicon", "Digital environmental sensor") + } + Deployment_Node(flash_ic, "Macronix MX25R 8MB", "USON-8 Package") { + ContainerDb(configStorage, "NVS Storage", "SPI NOR Flash", "LittleFS filesystem") + } + } + } + + Deployment_Node(router, "Home Wi-Fi Router", "Residential Gateway") { + Deployment_Node(server, "Local Home Server", "Raspberry Pi 5") { + System_Ext(localHub, "Local Smart Hub", "Home Assistant / Matter") + } + } + + Deployment_Node(phone, "User Smartphone", "Apple iPhone 15") { + System_Ext(mobileApp, "Smart Life App", "Flutter Mobile Client") + } +} + +Deployment_Node(aws, "AWS Cloud Region (eu-west-1)", "Amazon Web Services") { + Deployment_Node(iot_core, "AWS IoT Core", "Managed Gateway") { + Container(mqttBroker, "Ingestion Broker", "EMQX / AWS IoT", "MQTT TLS endpoint") + } + Deployment_Node(eks, "EKS Kubernetes Cluster", "Managed Containers") { + Deployment_Node(worker_pod, "Ingestion Pods", "Go Runtime") { + Container(ingestionWorker, "Ingestion Worker", "Go Microservice", "Stream processing") + } + Deployment_Node(registry_pod, "Registry Pods", "Node.js Runtime") { + Container(deviceRegistry, "Registry Service", "Node.js", "Device shadows") + } + Deployment_Node(api_pod, "Gateway Pods", "Node.js Runtime") { + Container(graphqlApi, "GraphQL Gateway", "Apollo Server", "Client API") + } + } + Deployment_Node(db_tier, "Managed Database Tier", "AWS Cloud DBs") { + ContainerDb(telemetryDb, "Telemetry Store", "TimescaleDB", "Time-series data") + ContainerDb(metadataDb, "Metadata Store", "RDS PostgreSQL", "User & device metadata") + } +} + +Rel(sensorFirmware, sensorHardware, "Reads raw data", "I2C Bus @ 400 kHz") +Rel(sensorFirmware, configStorage, "Reads credentials & logs", "SPI Bus @ 32 MHz") +Rel(sensorFirmware, mqttBroker, "Publishes telemetry", "MQTT over TLS 1.3") +Rel(sensorFirmware, localHub, "Broadcasts events", "Matter over Wi-Fi") + +Rel(mqttBroker, ingestionWorker, "Routes stream", "Internal Queue") +Rel(ingestionWorker, telemetryDb, "Batch inserts", "SQL / TCP") +Rel(ingestionWorker, deviceRegistry, "Syncs shadow", "gRPC") + +Rel(deviceRegistry, metadataDb, "Reads/writes metadata", "SQL") +Rel(graphqlApi, telemetryDb, "Queries history", "SQL") +Rel(graphqlApi, deviceRegistry, "Queries shadow", "gRPC") +Rel(graphqlApi, metadataDb, "Queries user records", "SQL") + +Rel(mobileApp, graphqlApi, "Fetches dashboard data", "HTTPS / WSS") + +@enduml +``` +[[CAPTION:Figure]] Matching PlantUML C4 script for View 6: Physical and Cloud Deployment Topology. + +## 4. Running Structurizr Lite Locally + +This repository provides built-in tools to launch and export Structurizr DSL models directly without manual Docker configurations: + +### A. Launching the Interactive Web Viewer + +Run the local helper script from the root of the repository: + +```bash +python tools/structurizr/structurizr-local.py docs/guides/c4-model-guide/workspace.dsl +``` + +This launches Structurizr Lite in a containerised instance. Open your browser to `http://localhost:8080` to interactively pan, zoom, inspect component dependencies, and switch between System Context, Container, Component, and Deployment views. + +### B. Exporting Diagrams to PlantUML and Mermaid + +To export the views defined in `workspace.dsl` into standard PlantUML C4 files or Mermaid markdown snippets: + +```bash +# Using Structurizr CLI export +structurizr-cli export -w docs/guides/c4-model-guide/workspace.dsl -f plantuml/c4plantuml -o docs/guides/c4-model-guide/gen/ + +# Using the repository automated exporter +python tools/structurizr/structurizr-export.py docs/guides/c4-model-guide/workspace.dsl +``` + +The exporter generates `.puml` files corresponding to each declared view (for example, `SystemContext_Overview.puml`, `Container_EdgeDevice.puml`, `Component_FirmwareArchitecture.puml`, `Component_CloudIngestionWorker.puml`, `Deployment_ProductionTopology.puml`). + +## 5. Best Practices for Structurizr DSL in Embedded Projects + +| Practice | Recommendation | Why It Matters | +| :--- | :--- | :--- | +| **Separate Firmware from Hardware** | Model physical sensors as `#Sensor` hardware containers and the application code as `#Firmware` containers. | Clarifies the boundary between physical silicon and executable machine code. | +| **Label Hardware Interfaces** | Specify baud rates, bus standards, and voltages on relations (for example, `"I2C Bus @ 400 kHz"`, `"SPI @ 32 MHz"`). | Eliminates ambiguity for electrical and firmware engineers reviewing pinouts and timing. | +| **Model Power Modes with Tags** | Use custom tags such as `#AlwaysOn` and `#SwitchedPowerRail` on hardware deployment nodes. | Highlights high-current radio peripherals that must be dynamically power-gated to preserve battery life. | +| **Keep Components Architectural** | Model RTOS tasks, worker pools, event queues, and state machines as components. Do not model single helper functions or static utility files. | Prevents diagram bloat and keeps the view focused on system concurrency and data flow. | +[[CAPTION:Table]] Structurizr DSL best practices for embedded and connected systems. diff --git a/content/guides/c4-model/07-deployment-and-dynamics.md b/content/guides/c4-model/07-deployment-and-dynamics.md new file mode 100644 index 0000000..60e9073 --- /dev/null +++ b/content/guides/c4-model/07-deployment-and-dynamics.md @@ -0,0 +1,172 @@ +--- +title: "Deployment, Tips & Tricks, and Anti-Patterns" +--- + +# Deployment, Tips & Tricks, and Anti-Patterns + +## 1. Physical Deployment and Hardware Topology + +In standard enterprise and web applications, software containers run inside virtual machines, cloud instances, or Kubernetes pods. In **connected IoT and embedded systems**, software containers run on specific silicon packages, connected by printed circuit board (`PCB`) traces. They are powered by batteries or energy-harvesting circuits, and subject to real-world electrical constraints. + +A **C4 Deployment Diagram** bridges software containers to their physical hardware execution environment. + +### A. Physical Architecture Overview + +The Smart Environmental Sensor runs its firmware container on an **STM32U575** microcontroller and reads the **BME688** sensor over an I2C bus. It logs offline data to an external **SPI NOR Flash**, and transmits over an integrated 2.4 GHz radio: + +```mermaid +flowchart TB + subgraph Power Supply and Energy Domain + Battery["Lithium Polymer (LiPo) Battery
[3.7V Nominal / 1200 mAh]"] + LDO["Low-Dropout Regulator (LDO)
[Constant 3.3V Always-On Domain]"] + LoadSwitch["Switched Power Rail
[TPS22918 Load Switch - Controlled by MCU]"] + end + + subgraph SensorBoard["Smart Sensor Hardware Board (4-Layer PCB)"] + MCU["STM32U575 Microcontroller
[QFN-48 Package / ARM Cortex-M33 @ 160 MHz]
Runs FreeRTOS Sensor Application"] + Sensor["Bosch BME688 Sensor
[3.0 x 3.0 mm LGA Package]
Digital Gas, Temperature, Humidity & Pressure"] + Flash["Macronix MX25R 8MB NOR Flash
[USON-8 Package]
NVS Configuration, Certificates & Staged Firmware"] + Antenna["2.4 GHz Ceramic Chip Antenna
[Matched 50-Ohm RF Network]
802.11 b/g/n Wi-Fi & BLE 5.2 Transmission"] + end + + Battery --> LDO + Battery --> LoadSwitch + LDO -- "Powers MCU core & RTC (Always-On)" --> MCU + LDO -- "Powers secure NVS storage" --> Flash + LoadSwitch -- "Powers sensor during measurement" --> Sensor + MCU -- "GPIO Rail Enable Pin" --> LoadSwitch + + MCU <== "I2C Bus (SDA, SCL) @ 400 kHz with 4.7k Pull-ups" ==> Sensor + MCU <== "Quad-SPI Bus (4 Data, CLK, CS) @ 32 MHz" ==> Flash + MCU <== "50-Ohm Microstrip RF Trace" ==> Antenna +``` +[[CAPTION:Figure]] Physical Deployment and Electrical Topology diagram mapping software containers to hardware silicon, power rails, and peripheral buses. + +### B. PlantUML C4 Deployment Diagram + +The PlantUML deployment model below formalises the physical hardware topology of the device and the cloud infrastructure: + +```plantuml +@startuml +!include + +Deployment_Node(home, "Residential Smart Home", "Physical Customer Site") { + Deployment_Node(wall, "Living Room Sensor Enclosure", "Polycarbonate Wall Mount") { + Deployment_Node(pcb, "Sensor Mainboard PCB", "4-Layer FR-4 Board") { + Deployment_Node(mcu, "STM32U575 MCU", "ARM Cortex-M33, 786KB SRAM, 2MB Flash") { + Container(fw, "Sensor Application Firmware", "C / FreeRTOS", "Runs measurement tasks, network stack, and state machines") + } + Deployment_Node(sensor_ic, "Bosch BME688", "Integrated MEMS Gas & Climate Sensor") { + Container(sensor_chip, "Sensor Subsystem", "Silicon IC", "Measures temperature, humidity, pressure, and VOC gases") + } + Deployment_Node(flash_ic, "Macronix MX25R 8MB", "External SPI NOR Flash") { + ContainerDb(flash_db, "Non-Volatile Storage (NVS)", "LittleFS Partition", "Stores credentials, certificates, and staged FOTA binaries") + } + } + } +} + +Deployment_Node(aws, "Amazon Web Services (AWS)", "Public Cloud Infrastructure") { + Deployment_Node(iot_core, "AWS IoT Core", "Managed Cloud Service") { + Container(broker, "Cloud MQTT Broker", "MQTT over TLS 1.3", "Terminates device sessions and validates client certificates") + } + Deployment_Node(eks, "EKS Cluster", "Managed Kubernetes") { + Container(ingest, "Telemetry Ingestion Worker", "Go Service", "Validates payloads and batch-writes to database") + Container(api, "GraphQL Gateway", "Node.js / Apollo", "Serves authenticated queries and real-time streams") + } + Deployment_Node(db_tier, "Managed Databases", "RDS / Timestream") { + ContainerDb(tsdb, "Telemetry Database", "TimescaleDB", "Time-series storage for environmental metrics") + } +} + +Rel(fw, sensor_chip, "Reads measurement data", "I2C Bus @ 400 kHz") +Rel(fw, flash_db, "Reads credentials / writes staged FOTA", "Quad-SPI @ 32 MHz") +Rel(fw, broker, "Publishes telemetry / receives delta state", "MQTT over TLS 1.3 / Port 8883") +Rel(broker, ingest, "Routes device messages", "Internal Message Queue") +Rel(ingest, tsdb, "Inserts batch metrics", "SQL / Native TCP") +Rel(api, tsdb, "Queries historical trends", "SQL") +@enduml +``` +[[CAPTION:Figure]] PlantUML C4 Deployment diagram showing edge hardware packaging and cloud infrastructure hosting. + +## 2. Dynamic Sequences: Modelling Time and Lifecycles + +While standard C4 diagrams capture static relationships, connected IoT systems are inherently dynamic. Microcontrollers sleep for 99% of their lifespan to conserve energy, waking periodically to measure sensors, negotiate Wi-Fi associations, publish MQTT telemetry, and immediately return to low-power sleep. + +A **C4 Dynamic Diagram** or UML sequence diagram illustrates this runtime sequence: + +```mermaid +sequenceDiagram + autonumber + participant Timer as Hardware RTC Timer + participant Core as MCU Application Core + participant Rail as Sensor Power Switch + participant Sensor as BME688 Sensor IC + participant WiFi as Wi-Fi & Network Subsystem + participant Cloud as Cloud MQTT Broker + + Note over Core: Ultra-Low-Power Deep Sleep (2.1 uA) + Timer->>Core: RTC Wakeup Alarm Interrupt + Core->>Rail: Assert GPIO (Enable Sensor Power Rail) + Core->>Core: Initialize System Clocks & FreeRTOS Scheduler + Core->>Sensor: Initialize Sensor & Trigger Forced Measurement Mode + Sensor-->>Core: Data Ready Interrupt (Temperature, Humidity, VOC) + Core->>Rail: Deassert GPIO (Disable Sensor Power Rail) + + Core->>Core: Compensate Readings & Construct JSON Telemetry Payload + Core->>WiFi: Wake Wi-Fi Subsystem & Connect to AP + WiFi-->>Core: Association & DHCP Lease Acquired + Core->>Cloud: Establish mTLS Handshake & Publish MQTT Payload + Cloud-->>Core: MQTT PUBACK Received + + Core->>WiFi: Graceful Disconnect & Power Down Radio + Core->>Timer: Program Next RTC Wakeup Alarm (+300 Seconds) + Core->>Core: Enter Deep Sleep Mode +``` +[[CAPTION:Figure]] Sequence diagram illustrating the sensor acquisition, network transmission, and power-gating lifecycle. + +## 3. Practical Architecture Tips & Tricks for IoT Teams + +### A. Establish Clear Layer Boundaries + +Avoid mixing physical electronics schematics with logical firmware components: + +- **Level 2 (Containers)**: Treat the running firmware binary as a software execution unit. Do not draw GPIO pins or pull-up resistors here. +- **Level 3 (Components)**: Focus on software modules, RTOS tasks, message queues, and HAL drivers. +- **Physical Deployment**: Place the PCB, connectors, bus routing, and power supplies strictly in the Deployment view. + +### B. Make Communication Protocols Explicit + +An unlabelled arrow in an IoT architecture diagram is a major risk. Every relationship arrow must explicitly state: + +1. **The application protocol** (for example, MQTT, CoAP, HTTPS, Matter, Modbus). +1. **The security transport layer** (for example, TLS 1.3, DTLS 1.2, AES-128-CCM, plaintext). +1. **The physical or network carrier** (for example, Wi-Fi 802.11 b/g/n, BLE 5.2, Thread, Ethernet, I2C, SPI, UART). + +### C. Automate Diagram Validation in CI/CD + +Prevent architecture diagrams from becoming obsolete by integrating validation into your development pipeline: + +```yaml +# Example GitHub Actions step +- name: Validate C4 Structurizr Model + run: | + docker run --rm -v $(pwd):/workspace structurizr/cli \ + validate -w /workspace/docs/guides/c4-model-guide/workspace.dsl +``` + +Using tools like `tools/docs-check/check_docs.py` and pre-commit hooks ensures that all referenced diagram files, links, and versions are automatically verified whenever code changes are committed. + +## 4. Architectural Anti-Patterns to Avoid + +When architecting connected systems, watch for these common pitfalls: + +| Anti-Pattern | Description | Impact | Recommended Solution | +| :--- | :--- | :--- | :--- | +| **The "Everything is a Container" Trap** | Modelling every C source file, module, or struct as a container at Level 2. | Diagrams become unreadable and lose architectural abstraction. | Restrict containers strictly to separate binaries, isolated cores, auxiliary chips, or storage media. Source modules belong at Level 3. | +| **The "Passive Wire" Trap** | Drawing connecting arrows without labelling the protocol, baud rate, or security transport. | Leaves firmware and cloud teams guessing at message formats and authentication requirements. | Always label relations with protocol and payload (for example, `[MQTT over mTLS 1.3 / Port 8883]`, `[I2C @ 400 kHz]`). | +| **The "Black Box Cloud" Fallacy** | Drawing the entire cloud platform as a single monolithic circle labelled "Cloud". | Hides critical latency, scaling bottlenecks, ingestion brokers, and database boundaries. | Decompose the cloud into distinct containers: Ingestion Broker, Processing Services, Time-Series Storage, and User APIs. | +| **Conflating Physical and Logical** | Showing PCB copper traces and power supplies on a Level 2 software container diagram. | Confuses software developers who need to understand data flow and execution logic. | Keep Level 2 focused on software execution and storage units. Move electrical and PCB wiring to a dedicated Physical Deployment diagram. | +| **Hardware Omission** | Pretending embedded software executes in an environment with infinite RAM, persistent power, and zero latency. | Leads to designs that fail under low-battery, brownout, or intermittent connectivity conditions. | Explicitly model hardware watchdogs, battery constraints, flash write cycles, and offline storage queues. | +| **Abandonware Diagrams** | Treating diagrams as static slide images drawn once during proposal phases and never updated. | Diagrams become obsolete within weeks, leading to architectural debt and confusion. | Keep diagram source text in version control (using Structurizr DSL or PlantUML). Integrate checks into Git pre-commit hooks. | +[[CAPTION:Table]] Anti-patterns and pitfalls in embedded and connected systems architecture modelling. diff --git a/content/guides/c4-model/files/workspace.dsl b/content/guides/c4-model/files/workspace.dsl new file mode 100644 index 0000000..9809559 --- /dev/null +++ b/content/guides/c4-model/files/workspace.dsl @@ -0,0 +1,295 @@ +workspace "Smart Home Environmental Monitoring System" "End-to-end architecture model from edge sensor firmware to cloud microservices and mobile user experience." { + + model { + # ===================================================================== + # People / User Personas + # ===================================================================== + homeowner = person "Homeowner" "Monitors home air quality and receives comfort alerts." "Person" + installer = person "Field Technician" "Commissions, provisions, and diagnostics sensors during installation." "Person" + + # ===================================================================== + # External Systems + # ===================================================================== + localHub = softwareSystem "Local Smart Hub" "Local home automation gateway (for example, Home Assistant, Apple Home, Matter hub)." "External" + weatherService = softwareSystem "National Weather API" "Provides external outdoor baseline temperature and pollen data." "External" + pushService = softwareSystem "Push Notification Service" "Apple APNs and Google FCM delivery network." "External" + + # ===================================================================== + # Connected Edge Product: Smart Environmental Sensor + # ===================================================================== + sensorDevice = softwareSystem "Smart Environmental Sensor" "Battery-powered physical IoT edge device measuring ambient environment." "Device" { + + sensorHardware = container "BME688 Sensor Subsystem" "Digital MEMS sensor measuring temperature, humidity, barometric pressure, and VOC gas." "Silicon / MEMS" "Sensor" + + configStorage = container "Non-Volatile Storage (NVS)" "Encrypted SPI NOR flash memory storing Wi-Fi credentials, certificates, and offline logs." "Macronix SPI Flash" "Database" + + sensorFirmware = container "Sensor Application Firmware" "FreeRTOS-based embedded firmware managing data acquisition, power cycling, and networking." "C / FreeRTOS on STM32U575" "Firmware" { + + sensorAcq = component "Sensor Acquisition Manager" "Schedules periodic sensor readings, applies calibration offsets, and filters noise." "C Module / RTOS Task" "FirmwareComponent" + powerOrch = component "Power & Sleep Orchestrator" "Coordinates transitions between active measurement, network transmission, and deep sleep." "C Module / State Machine" "FirmwareComponent" + wifiMgr = component "Wi-Fi & Network Manager" "Manages 802.11 b/g/n association, station state, and network recovery." "C Module / Network Task" "FirmwareComponent" + mqttClient = component "MQTT Ingestion Client" "Manages secure mTLS connections and serialises JSON telemetry payloads." "C Module / coreMQTT" "FirmwareComponent" + fotaAgent = component "FOTA Update Agent" "Downloads signed firmware chunks, validates SHA-256 signatures, and writes to staging flash." "C Module / Bootloader Interface" "FirmwareComponent" + configStore = component "Configuration Store" "Provides thread-safe access to provisioned settings and network credentials." "C Module / LittleFS HAL" "FirmwareComponent" + sensorHal = component "Sensor HAL & Driver" "I2C peripheral driver communicating directly with the BME688 hardware registers." "C Module / I2C Bus Driver" "FirmwareComponent" + } + } + + # ===================================================================== + # Cloud Platform + # ===================================================================== + cloudPlatform = softwareSystem "Environmental Cloud Platform" "Scalable cloud platform for device ingestion, shadow management, and telemetry analytics." "Cloud" { + + mqttBroker = container "Cloud Ingestion Broker" "High-throughput MQTT broker terminating device TLS connections and routing messages." "EMQX / AWS IoT Core" "CloudService" + + ingestionWorker = container "Telemetry Ingestion Worker" "Consumes raw telemetry messages, validates schema, applies calibrations, and writes time-series records." "Go Microservice" "CloudService" { + streamConsumer = component "MQTT Stream Consumer" "Consumes raw telemetry events from message broker queue." "Go Goroutine Pool" "CloudComponent" + payloadValidator = component "Schema & Security Validator" "Validates payload format and authenticates device HMAC signatures." "Go Package" "CloudComponent" + metricCalibrator = component "Metric Normaliser & Calibrator" "Applies sensor altitude offsets and converts VOC resistance to IAQ index." "Pipeline Module" "CloudComponent" + batchWriter = component "Time-Series Batch Writer" "Buffers samples and executes bulk SQL copy operations into database." "Buffer Pool" "CloudComponent" + shadowSyncer = component "Device Shadow Syncer" "Synchronises reported battery, firmware version, and last-seen state." "gRPC Client" "CloudComponent" + } + + deviceRegistry = container "Device Shadow & Registry Service" "Maintains device digital twins, provisioning credentials, desired state, and FOTA campaigns." "Node.js / TypeScript" "CloudService" + + telemetryDb = container "Telemetry Time-Series Store" "Optimised time-series database storing sensor metrics and historical aggregates." "TimescaleDB / Timeseries" "Database" + + metadataDb = container "Relational Metadata Database" "Stores user accounts, home floor plans, device ownership, and alerting rules." "PostgreSQL 16" "Database" + + graphqlApi = container "GraphQL Application Gateway" "Serves unified queries and real-time WebSockets subscriptions to clients." "Node.js / Apollo Server" "CloudService" + } + + # ===================================================================== + # User Client Applications + # ===================================================================== + mobileApp = softwareSystem "Smart Life Mobile App" "Cross-platform mobile application providing real-time telemetry, historical charts, and push alerts." "MobileApp" + + # ===================================================================== + # System-Level Relationships (Level 1) + # ===================================================================== + homeowner -> mobileApp "Views live metrics, trends, and alerts" "HTTPS" + homeowner -> sensorDevice "Views RGB status LED and presses commissioning button" "Physical / Visual" + installer -> sensorDevice "Provisions Wi-Fi credentials and inspects diagnostics" "BLE 5.2" + + sensorDevice -> cloudPlatform "Publishes periodic telemetry and checks for updates" "MQTT over TLS 1.3 / Port 8883" + sensorDevice -> localHub "Broadcasts local metrics and status" "Matter over Wi-Fi / CoAP" + cloudPlatform -> pushService "Dispatches push alert payloads" "HTTPS / REST" + pushService -> mobileApp "Delivers push notifications" "Apple APNs / Google FCM" + cloudPlatform -> weatherService "Fetches outdoor baseline temperature and air quality" "HTTPS / JSON" + mobileApp -> cloudPlatform "Queries device data, historical metrics, and updates configuration" "GraphQL / HTTPS / WSS" + + # ===================================================================== + # Container-Level Relationships (Level 2) + # ===================================================================== + sensorFirmware -> sensorHardware "Triggers measurement cycles and reads raw register values" "I2C Bus @ 400 kHz" + sensorFirmware -> configStorage "Reads/writes credentials, crash dumps, and staged firmware" "SPI Bus @ 32 MHz" + sensorFirmware -> mqttBroker "Publishes telemetry and subscribes to delta updates" "MQTT over TLS 1.3 (mTLS)" + sensorFirmware -> localHub "Emits local telemetry" "Matter / UDP" + + mqttBroker -> ingestionWorker "Routes incoming telemetry messages" "Internal Queue / Kafka" + ingestionWorker -> telemetryDb "Writes batch time-series sensor samples" "SQL / Native TCP" + ingestionWorker -> deviceRegistry "Updates last-seen timestamp and battery state" "gRPC" + + deviceRegistry -> metadataDb "Reads and writes device profiles and ownership records" "SQL" + graphqlApi -> telemetryDb "Queries metric history and rollups" "SQL" + graphqlApi -> deviceRegistry "Reads device shadow and dispatches desired state updates" "gRPC" + graphqlApi -> metadataDb "Authenticates users and reads home layouts" "SQL" + + mobileApp -> graphqlApi "Fetches dashboard metrics and sends remote commands" "HTTPS / WSS" + + # ===================================================================== + # Component-Level Relationships inside Firmware (Level 3) + # ===================================================================== + powerOrch -> sensorAcq "Wakes and triggers scheduled reading" "Direct Call / Event Signal" + sensorAcq -> sensorHal "Reads compensated sensor values" "C API Call" + sensorHal -> sensorHardware "Reads data registers" "I2C HAL Driver" + + sensorAcq -> mqttClient "Enqueues telemetry payload" "FreeRTOS Queue" + powerOrch -> wifiMgr "Commands network connect/disconnect for power conservation" "RTOS Event Group" + wifiMgr -> mqttClient "Notifies IP address acquisition and link state" "Event Notification" + + mqttClient -> configStore "Retrieves client TLS certificates and broker endpoint" "C API Call" + configStore -> configStorage "Reads stored keys and nonces" "LittleFS File IO" + + fotaAgent -> mqttClient "Subscribes to update jobs and downloads signed chunks" "MQTT Callback" + fotaAgent -> configStorage "Writes firmware image to staging partition" "Flash Driver Call" + fotaAgent -> powerOrch "Requests system reboot into bootloader" "System Reset" + + # ===================================================================== + # Component-Level Relationships inside Cloud Ingestion Worker (Level 3) + # ===================================================================== + mqttBroker -> streamConsumer "Routes raw messages" "AMQP / Kafka Queue" + streamConsumer -> payloadValidator "Passes raw payload" "Go Channel" + payloadValidator -> metricCalibrator "Passes validated data" "Go Channel" + metricCalibrator -> batchWriter "Appends processed sample" "In-Memory Ring Buffer" + metricCalibrator -> shadowSyncer "Signals device heartbeat" "Go Channel" + batchWriter -> telemetryDb "Executes batch insert" "SQL / Native TCP" + shadowSyncer -> deviceRegistry "Updates reported shadow state" "gRPC over HTTP/2" + + # ===================================================================== + # Deployment Environment: Physical Smart Home Deployment + # ===================================================================== + deploymentEnvironment "Production" { + deploymentNode "Residential Smart Home" "Physical residence of the homeowner." "Physical Boundary" { + + deploymentNode "Living Room Wall Mount" "Physical wall installation point." "Hardware Enclosure" { + deploymentNode "PCB Assembly" "Custom 4-layer printed circuit board." "Hardware" { + deploymentNode "STM32U575 MCU" "ARM Cortex-M33 running at 160MHz with TrustZone." "Microcontroller" { + containerInstance sensorFirmware + } + deploymentNode "Bosch BME688" "Integrated environmental sensor in 3.0 x 3.0 mm package." "Hardware IC" { + containerInstance sensorHardware + } + deploymentNode "Macronix MX25R 8MB" "External NOR flash memory chip." "Hardware IC" { + containerInstance configStorage + } + } + } + + deploymentNode "Home Wi-Fi Router" "Standard dual-band residential wireless gateway." "Network Device" { + deploymentNode "Local Home Server" "Raspberry Pi 5 home server." "Hardware" { + softwareSystemInstance localHub + } + } + + deploymentNode "User Smartphone" "Apple iPhone 15 running iOS 18." "Mobile Device" { + softwareSystemInstance mobileApp + } + } + + deploymentNode "AWS Cloud Region (eu-west-1)" "Amazon Web Services Ireland data centre." "Cloud Infrastructure" { + deploymentNode "AWS IoT Core" "Managed cloud MQTT message broker and TLS gateway." "AWS Service" { + containerInstance mqttBroker + } + deploymentNode "Amazon Elastic Kubernetes Service (EKS)" "Managed container orchestration cluster." "Kubernetes" { + deploymentNode "Telemetry Ingestion Pods" "Auto-scaling worker pods." "Container Runtime" { + containerInstance ingestionWorker + } + deploymentNode "Device Registry Pods" "Stateful management pods." "Container Runtime" { + containerInstance deviceRegistry + } + deploymentNode "GraphQL API Pods" "Public gateway service pods." "Container Runtime" { + containerInstance graphqlApi + } + } + deploymentNode "Managed Database Tier" "High-availability managed data tier." "Cloud Database" { + deploymentNode "Amazon RDS PostgreSQL" "Multi-AZ relational database." "Database Server" { + containerInstance metadataDb + } + deploymentNode "Amazon Timestream / TimescaleDB" "Scalable time-series storage engine." "Database Server" { + containerInstance telemetryDb + } + } + } + } + } + + # ========================================================================= + # Views Definition + # ========================================================================= + views { + systemContext sensorDevice "SystemContext_Overview" "Overall system context showing device, users, cloud platform, and third-party integrations." { + include * + autoLayout lr + } + + container sensorDevice "Container_EdgeDevice" "Internal containers of the Smart Environmental Sensor edge device." { + include * + autoLayout tb + } + + container cloudPlatform "Container_CloudPlatform" "Internal microservices and database containers of the Cloud Platform." { + include * + autoLayout tb + } + + component sensorFirmware "Component_FirmwareArchitecture" "Component-level architecture of the FreeRTOS embedded application firmware." { + include * + autoLayout tb + } + + component ingestionWorker "Component_CloudIngestionWorker" "Component-level architecture of the Go Telemetry Ingestion Worker." { + include * + autoLayout tb + } + + deployment sensorDevice "Production" "Deployment_ProductionTopology" "Physical hardware and cloud infrastructure deployment mapping." { + include * + autoLayout tb + } + + # ===================================================================== + # Visual Styling and Tag Configurations + # ===================================================================== + styles { + element "Person" { + shape Person + background #08427b + color #ffffff + } + element "Software System" { + background #1168bd + color #ffffff + } + element "Device" { + shape RoundedBox + background #2A6F97 + color #ffffff + } + element "Cloud" { + shape RoundedBox + background #014F86 + color #ffffff + } + element "External" { + background #999999 + color #ffffff + } + element "MobileApp" { + shape MobileDevicePortrait + background #0077B6 + color #ffffff + } + element "Container" { + background #438dd5 + color #ffffff + } + element "Firmware" { + shape RoundedBox + background #1E88E5 + color #ffffff + } + element "Sensor" { + shape Hexagon + background #00897B + color #ffffff + } + element "Database" { + shape Cylinder + background #2E7D32 + color #ffffff + } + element "CloudService" { + shape RoundedBox + background #1565C0 + color #ffffff + } + element "FirmwareComponent" { + shape Component + background #5C6BC0 + color #ffffff + } + element "CloudComponent" { + shape Component + background #0288D1 + color #ffffff + } + relationship "Relationship" { + thickness 2 + dashed false + color #707070 + fontSize 24 + } + } + } +} diff --git a/content/guides/c4-model/index.md b/content/guides/c4-model/index.md new file mode 100644 index 0000000..09ead45 --- /dev/null +++ b/content/guides/c4-model/index.md @@ -0,0 +1,85 @@ +--- +title: C4 modelling for IoT and connected systems +--- + +# C4 modelling for IoT and connected systems + +## 1. Introduction and Purpose + +The C4 model, created by Simon Brown, is a hierarchical, visual way to describe software architecture. It has four levels of abstraction: **System Context (Level 1)**, **Containers (Level 2)**, **Components (Level 3)** and **Code (Level 4)**. + +When applied to **Internet of Things (IoT) products and connected devices**, C4 bridges the gap between hardware engineers, firmware developers, cloud architects, and product managers. A complete IoT product is not merely an isolated piece of firmware or a standalone cloud platform; it is an integrated, end-to-end cyber-physical system: + +```mermaid +graph LR + subgraph Edge Product + A["Physical IoT Device
Sensors, MCU, High-Level Firmware"] + end + subgraph Network Transport + B["Local Wireless / Gateway
Wi-Fi, Thread, Cellular, BLE"] + end + subgraph Cloud Platform + C["Cloud Ingestion & Services
Brokers, Microservices, Databases"] + end + subgraph User Experience + D["User Client Apps
Mobile Apps, Web Portals"] + end + + A <--> B + B <--> C + C <--> D +``` +[[CAPTION:Figure]] The end-to-end IoT product scope spanned by C4 architecture modelling. + +This guidebook provides an authoritative, practical reference for engineering teams. Throughout this guide we use one reference system, **The Smart Home Environmental Monitor**. It is a connected temperature, humidity and air quality sensor that talks to a cloud platform and a mobile app. + +```mermaid +graph TD + subgraph Foundations & Tooling + F1["C4 Foundations
Origins & Philosophy
The Google Maps Metaphor"] + F2["Diagrams-as-Code Tooling
PlantUML, Mermaid & Structurizr DSL
Text-based Versioned Architecture"] + end + subgraph C4 Zoom Levels + L1["Level 1: System Context
Product Ecosystem
Homeowner, Sensor Device, Cloud Platform, Local Hub"] + L2["Level 2: Containers
Device & Cloud Subsystems
MCU, Firmware, Flash, Ingestion Broker, TimescaleDB"] + L3["Level 3: Components
Modular Architecture
Firmware RTOS Tasks & Cloud Microservices"] + end + subgraph Concrete Realisation + R1["Structurizr DSL Reference
workspace.dsl
Model-First Architecture"] + R2["Physical Deployment & Dynamics
Silicon, Buses, Power Rails & Sequences
Anti-Patterns & Team Practices"] + end + + F1 --> F2 + F2 --> L1 + L1 --> L2 + L2 --> L3 + L3 --> R1 + R1 --> R2 +``` +[[CAPTION:Figure]] Complete 7-part engineering roadmap for C4 architecture modelling in connected systems. + +## 2. Core Concepts: Translating C4 for IoT Systems + +To model connected devices and cloud platforms without getting lost in low-level register minutiae, use these straightforward definitions: + +| C4 Level | Enterprise Software Definition | Connected IoT System Adaptation | +| :--- | :--- | :--- | +| **Foundations & Tooling** | Diagramming conventions and tools. | The origins of C4, why traditional diagrams fail, and how to use PlantUML, Mermaid, and Structurizr DSL in version-controlled repositories. | +| **Level 1: System Context** | The system as a black box surrounded by user personas and external systems. | The overall product ecosystem: the homeowner/operator, the physical IoT device, local wireless gateways, cloud platforms, and third-party integrations (for example, Apple Home, Google Home, Home Assistant). | +| **Level 2: Containers** | Separately running processes, executables, or datastores (web app, API, database). | Major execution units across **both the device and cloud**: the device processor running the firmware image, non-volatile flash storage partitions, the cloud MQTT/HTTPS ingestion broker, cloud microservices, time-series telemetry databases, and the mobile application. | +| **Level 3: Components** | Modular groupings of classes/modules within a container. | **Inside Firmware**: Functional subsystems (Sensor Acquisition Manager, Wi-Fi/Network Manager, MQTT Client, Storage Manager, Power/Sleep Controller).
**Inside Cloud**: Microservice modules (Ingestion Controller, Device Shadow Manager, Alerting Engine, Notification Dispatcher). | +| **Level 4: Deployment & DSL** | Class diagrams, interfaces, and physical deployment nodes. | High-value state machines (sleep/wake cycles), physical PCB topology (buses, power rails), and complete Diagrams-as-Code implementations in **Structurizr DSL** and **PlantUML**. | +[[CAPTION:Table]] C4 model levels adapted for end-to-end IoT and connected systems. + +> [!TIP] +> Keep the firmware view focused on **architecture**, not implementation trivia. At Level 2, treat the microcontroller firmware as a cohesive container. At Level 3, model its major functional subsystems and event queues. Do not model every individual `.c` and `.h` file. + +## 3. Guidebook Contents + +1. **[The C4 Model: Origins, Philosophy, and Core Mechanics](01-c4-model-foundations.md)**: Historical background, Simon Brown's breakthrough, the Google Maps metaphor, and core principles. +1. **[Diagrams as Code: Tooling, DSLs, and Repository Integration](02-diagrams-as-code-tooling.md)**: Comparing PlantUML, Mermaid, and Structurizr DSL; macro anatomy, repository integration, and selection guide. +1. **[System Context (Level 1)](03-system-context.md)**: Product ecosystem scope, external actors, Mermaid, and matching PlantUML C4 Context diagram. +1. **[Container Diagrams (Level 2)](04-container-diagrams.md)**: End-to-end device and cloud subsystem containers, protocols, and matching PlantUML C4 Container diagram. +1. **[Component Diagrams (Level 3)](05-component-diagrams.md)**: High-level firmware architecture and cloud microservices, with matching PlantUML C4 Component diagram. +1. **[Structurizr DSL Reference](06-structurizr-dsl-reference.md)**: A complete, copy-pasteable `workspace.dsl` file modelling the entire Smart Home IoT ecosystem with custom styles and multi-view declarations. +1. **[Deployment, Dynamics, and Best Practices](07-deployment-and-dynamics.md)**: Physical deployment diagrams, power rails, dynamic sleep/wake sequence diagrams, and architectural anti-patterns to avoid. diff --git a/content/guides/code-review.md b/content/guides/code-review.md new file mode 100644 index 0000000..e634e99 --- /dev/null +++ b/content/guides/code-review.md @@ -0,0 +1,645 @@ +--- +title: Reviewing code +--- + +# Reviewing code + +**How to think about code, how to read it, and how to give feedback that makes the code and the team better** + +> [!NOTE] Companion Checklist +> This document teaches the **Philosophy, Mindset, and Reading Technique** for peer review. For the concrete, itemised checklist to verify pull requests against (Functionality, Style, Architecture, Concurrency, Safety), see the companion [Code review checklist](../templates/code-review-checklist.md). + +## Roles and responsibilities + +### Author (PR owner) +You own: +- Scope and clarity of the PR +- Tests and evidence +- Addressing comments (or explicitly disagreeing with a reason) +- Keeping it small enough to review + +### Reviewer +You own: +- Reading it properly (not rubber-stamping) +- Identifying risk and asking the right questions +- Being specific and constructive +- Replying promptly (or saying when you can’t) + +### Maintainer / Approver (if different) +You own: +- Merge decisions +- Enforcing repo quality gates +- Making sure the change fits the roadmap/architecture + +## A Code Review is Not a Hunt for Faults + +Bad reviews are driven by: + +> “I am here to point out mistakes.” + +Good reviews start from: + +> “I’m here to understand the intent, help improve the design, and make the product safer.” + +**You are reviewing *with* the author**, not *against* them.\ +Think “pair-design-audit”, not “police investigation”. + +A code review has three goals: + +1. Correctness: does it do what was intended? +1. Maintainability: can someone else understand and change it safely later? +1. Risk reduction: are we introducing hidden technical debt, regressions, timing issues? + +Note: **you are not judging the developer**, you are judging the **future cost** of the change. + +The best reviewers are calm, curious, and collaborative. + +## Don’t Read Code — Understand It + +We are all taught to **read code line by line**.\ +This is the slowest and least effective way to review. + +Great reviewers **start at the top**: + +### Start with Context + +Before looking at the first line, ask: + +- What is the problem being solved? +- What requirement or bug is this linked to? +- What is the design intent? +- Where does this fit in the architecture? +- Is this a new behaviour, a change, or a refactor? + +If you cannot answer those **without opening the diff**, the PR isn’t ready. + +A clear PR description saves hours: + +- Why this change exists +- What the solution approach is +- What was considered and rejected +- Any known limitations +- What needs special attention + +> **Tip:** If you don’t understand the intent after reading the PR description, stop. The review isn’t a guessing game. + +## Read the Code as a Story, Not as Text + +Good reviewers read code **in layers**: + +### Layer 1 — High-Level Design + +Look at: + +- the data flow +- the abstractions +- module boundaries +- the decisions in the PR description + +Ask: + +- Is this solution **simple and obvious**, or **clever and mysterious**? +- Are there obvious alternative shapes that are more readable? + +If you disagree with the **shape**, don’t bother nitpicking variable names — fix the foundation first. + +### Layer 2 — Interfaces and Contracts + +Check **function boundaries**, not internals: + +- What goes in? +- What comes out? +- What is guaranteed? +- What happens in errors? +- How are side effects handled? + +**Great design is visible at the interfaces.** + +### Layer 3 — Internal Logic + +Only **after** you understand the shape, examine key branches: + +- difficult logic +- multi-step algorithms +- error paths +- concurrency primitives + +This is where bugs hide. + +Avoid reading **every line** unless the change is tiny. + +> **Practitioner rule:** You don’t need to read everything — you need to read the **right 15%**. + +## Mindset: Curious, Not Cynical + +Poor reviews are full of statements: + +> “This is wrong.”\ +> “Why did you do it like this?” + +Good reviews ask: + +> “What trade-off made this the right solution?”\ +> “Help me understand why we avoid approach X?”\ +> “Is there a simpler way?” + +Assume the author did the **best they could** with the information at the time. + +Treat the review like a **joint investigation**, not an exam. + +## The Soft Skills Matter More Than the Code + +The **best reviewers**: + +- ask questions +- explain reasoning +- avoid sarcasm or power plays +- propose alternatives +- leave final editorial decisions to the owner (unless safety is involved) + +Example good phrasing: + +- “Would we gain clarity if we extracted this loop?” +- “This reminds me of pattern X — could we use it?” +- “If we hit error here, does it leak resources?” +- “I don’t fully understand the boundary conditions — can we add one more test?” + +Example bad phrasing: + +- “This is wrong.” +- “Why didn’t you use pattern X? Everyone knows that.” +- “Looks ugly.” + +Feedback is **instruction + reasoning**, not judgement. + +## Think in Terms of Risk + +A big mistake in reviews is treating everything as equal. + +Not all lines carry the same risk. + +Ask yourself: + +- Could this cause a crash? +- Could this silently corrupt data? +- Could this break timing or deadlines? +- Could this cause a security issue? +- Could this cause later maintenance pain? + +Review time is **finite**. You must **spend it where risk matters**. + +Examples of **high-risk areas**: + +- cryptography +- error paths +- time-critical code +- concurrency primitives +- memory allocation +- FOTA upgrade logic +- state machines +- hardware register access +- untested legacy paths +- safety logic + +Examples of **low-risk**: + +- formatting +- naming style +- unimportant internal helper changes + +Correctness > Maintainability > Cleanliness. + +Fix **critical things** first. + +## Know Your Cognitive Limits + +Your brain tires quickly doing deep code analysis.\ +Most people can review **60–90 minutes total per day** before quality drops. + +SmartBear’s research shows: + +- defect discovery rate falls sharply after \~500 LOC per review session +- small reviews catch more serious bugs + +So: + +- prefer small PRs +- review in short bursts +- never push through fatigue + +If a PR is **too big** to understand comfortably, that’s a **process failure**, not a reviewer failure. + +## Avoid Reading the Code in Diff Mode First + +Diffs ruin your brain.\ +They show changes **out of context**, and your eyes chase **red/green noise**. + +Ideal order: + +1. PR description +1. Design/architecture doc snippet (if provided) +1. Browse whole files (understand shape) +1. Only then read the diff + +You are here to **understand**, not mechanically inspect edits. + +## Look for Intent Signals + +In good code, you can answer: + +- What is this doing? +- Why is it this shape? +- What are the trade-offs? +- What if it fails halfway? + +Look for signals of intention: + +- clear names +- limits checked +- error paths defined +- tests that show prescribed behaviour + +If intention is unclear, request a comment that explains **why**, not **what**. + +Example good comment: + +> “We use polling here instead of interrupts because the hardware module is unstable during self-test.” + +## Embed Test Thinking + +When you read code, **imagine how you would break it**: + +- what if input is zero? +- what if message arrives out of order? +- what if timer triggers early? +- what if the UART drops bytes? +- what if pointer is NULL? +- what if config is wrong? +- what if battery drops during FOTA? + +If the author hasn’t tested these scenarios, note it. + +Great review feedback: + +> “Can we add a test for when the sensor returns invalid data?” + +Not: + +> “Your code doesn’t handle invalid data.” + +(Same point, very different reception.) + +## Separate Design Feedback from Style Feedback + +If you mix: + +- architectural critique and +- formatting nitpicks + +in the same message stream, the important stuff gets lost. + +Order your feedback: + +1. design issues +1. correctness issues +1. edge cases +1. style and clarity + +If you start with “spacing”, your mindset is in the wrong place. + +## Favour Simplicity Over Cleverness + +If you see clever patterns that save 4 lines but add 4 hours of future debugging, change them. + +The reviewer’s job is to represent the **future reader**, not the current author. + +Quote from *Clean Code*: + +> “Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live.” + +Funny, but true — in embedded work, somebody will debug your code in 2 years under pressure with poor tools. + +Make it easy for them. + +## Don’t Force Your Preferences + +You will see code that **isn’t how you would write it**, but is still correct, clear, and safe. + +If it’s **different, but good enough**, let it go. + +Reserve push-back for: + +- correctness +- clarity +- maintainability +- risk +- safety + +Not personal taste. + +Good review comment: + +> “I prefer A, but B is fine and safe — no change required.” + +## Ask for Tests When They Matter + +When reading tricky logic, ask: + +- Where is the test for the pathological case? +- Where is the test for invalid input? +- Where is the test for low battery? +- Where is the test for overflow? + +In embedded projects, **unit tests cannot cover everything**, but they must cover **logic**, not **hardware access**. + +Encourage **state-based tests**, **table-based tests**, and **simplifying flags** that drive behaviour. + +## Use the Checklist as a Safety Net + +Once you’ve done the **thinking part**, use your checklist to check you haven’t missed subtle issues. + +Your checklist is excellent for this purpose.\ +(Referencing the user’s checklist here)\ +The checklist gives the **mechanical completeness check**, not the **thinking**. + +Example:\ +**Did we check that every input is validated?**\ +→ Checklist confirms. + +But the **insight** comes from the reviewer’s **mental model**. + +Checklist is the **seatbelt**, not the **driver**.\ +[Code review checklist](../templates/code-review-checklist.md)-code\_review\_checklist + +## Adopt a Review Routine + +Here’s a **5-step routine** that builds skill: + +1. Read the PR description +1. Trace the feature from requirement → code\ + understand intent + +1. Review high-level design +1. Drill into key logic paths +1. Run the checklist pass + +This **separates thinking from checking**. + +## Recognise When to Stop + +If design is wrong at the architectural level: + +- stop reviewing line-by-line +- request design update first + +Example:\ +re-implementing a state machine in multiple places.\ +No amount of formatting will save that. + +## Your Role is to Raise Awareness, Not Demand Perfection + +Sometimes you find an issue that **could** be improved, but does not block correctness, safety, or clarity. + +In those cases: + +- raise it +- explain your reasoning +- leave decision to author + +This builds trust and encourages review participation. + +## Build a Positive Culture + +Good review habits in teams: + +- thank people for thoughtful feedback +- praise good patterns explicitly +- acknowledge improvements +- model behaviour for juniors +- ask questions more than give orders +- treat review as **mentoring**, not policing + +Review should feel like **a technical conversation**, not an interrogation. + +## Conclusion – The Reviewer’s Mindset + +At the end of a review, ask yourself: + +- Do I understand what this code does? +- Do I understand why it is shaped this way? +- Would I be comfortable owning this code? +- If I modify this in six months, would I be safe? +- Would I bet the product on this behaviour? + +If the answer is yes on all, approve. + +If the answer is no: + +- help the author close the gap. + +## Practical Advice Summary + +- Start at the **design**, not the diff +- Understand **intent** before nitpicking +- Review in **layers** +- Think **risk-based** +- Work in **short sessions** +- Use the **checklist last** +- Ask questions instead of accusing +- Optimise for **future readability** +- Keep **personal style** separate from quality +- Encourage **tests where logic demands it** +- Focus feedback on **behaviour**, not **style** + +## Pull request descriptions + +Give reviewers the context before they read a line of code. Use the [pull request template](../templates/pull-request-and-commits.md) in the Templates section, and keep it in the repository as `.github/pull_request_template.md` or your host's equivalent. + +## Example Comments (Good vs Bad) + +This teaches the **tone** and **behaviour** expected. + +### Bad Comments + +Author will feel judged, not supported: + +- “This is wrong.” +- “Why didn’t you just follow X pattern?” +- “This makes no sense.” +- “Fix your formatting.” +- “We always use approach Y.” +- “You don’t understand state machines.” + +### Good Comments + +Same outcome, better communication: + +- “Could you explain why this approach was chosen over X?” +- “I’m unsure what happens on the error path if the pointer is null — could we clarify it?” +- “Would extracting this loop into `process_message()` make sequencing clearer?” +- “If `returnCode != 0`, do we leak resources here?” +- “If we follow the existing state machine pattern, this might be simpler — thoughts?” +- “Could we add one test for the negative path? I think it would protect against regression.” + +### Excellent Comments (Mentoring tone) + +You are teaching **thinking**: + +- “This looks like the same pattern we used in `module_x`. If we reuse it, we gain consistency and reduce risk.” +- “This logic is clever but hard to read. Could simplicity help here even if it costs a few lines?” +- “If this value crosses ISR boundary, we may need atomic access — what do you think?” +- “This approach is good. In a future refactor, we might consider extracting a table-driven state machine — not blocking now.” + +## One-Page Training Card (Printable) + +**Title:** *How to Review Code (Embedded Systems & MISRA)* + +### Mindset + +- You review **intent**, not just syntax +- Assume **competent author** +- Ask questions, don’t attack +- Aim for **safer**, **simpler**, **clearer** + +### Review in Layers + +1. Why does this change exist?\ + (read PR description) + +1. What is the design shape?\ + (read high-level diff) + +1. Where are the risks?\ + (timing, memory, state) + +1. Check logic paths\ + (especially error handling) + +1. Run checklist pass\ + (MISRA, boundary conditions) + +### What Matters + +- Correctness +- Safety +- Maintainability +- Simplicity +- Risk reduction + +**NOT**: + +- personal taste +- stylistic details first + +### Ask Yourself + +- Would I be comfortable owning this? +- Is behaviour obvious from reading it? +- Are failure paths safe? +- Does it follow existing patterns? +- Could this crash in production? + +### Common Traps + +- Reviewing everything equally +- Reading diff first +- 2-hour review sessions +- Fighting style wars +- Ignoring error paths +- Cleverness over clarity + +### Embedded Safety Focus + +- Atomic access (interrupts) +- Overflow and wrap behaviour +- State machine consistency +- Hardware side-effects +- Memory budgets +- FOTA stability path +- Watchdog handling + +### Quality Principle + +> *Simple code is safer.* + +## Team Rules for Review Etiquette + +These are **team agreements** — not optional. + +### We agree that + +1. Reviews are collaborative, not confrontational +1. Feedback is **specific**, **actionable**, and **reasoned** +1. Ask questions instead of issuing commands +1. Avoid **tone policing** or sarcasm +1. Highlight **positive choices** +1. If design shape is wrong, **stop**, don’t nitpick +1. Split large PRs — **small units** are safer +1. Everyone is **teaching and learning** +1. Decisions are recorded in **PR discussion**, not lost in chat +1. Respect **DoR — Ready for Review gate** +1. Code must meet **minimum safety gates**, not personal taste +1. If discussion is stuck, **pair-review** the solution +1. The reviewer must imagine **future cost**, not current ease + +## Bibliography / Recommended Reading + +A curated list relevant to **embedded, safety, and review technique**. + +### Books + +- Best Kept Secrets of Peer Code Review — SmartBear (Rigby & Bird)\ + Practical, research-based, not fluff. + +- Code Complete — Steve McConnell\ + Still the best general text on how to reason about code. + +- Clean Code — Robert C. Martin\ + Opinionated, but useful for readability thinking. + +- Clean Architecture — Martin\ + Good for thinking about module boundaries. + +- Rapid Development — Steve McConnell\ + Project perspective; risk thinking. + +- Peopleware — DeMarco & Lister\ + Culture and psychology. + +- Software Engineering at Google — Winters, Manshreck, Wright\ + Modern review culture and tooling thinking. + +- Effective C — Robert Seacord\ + Better C habits; relevant for embedded. + +- Embedded C Coding Standards — Michael Barr / Netrino\ + Good alignment with MISRA thinking. + +### Papers / Short Material + +- “Modern Code Review: A Case Study at Microsoft” — Rigby, Bird +- “Best Kept Secrets of Peer Code Review” (SmartBear research summaries) +- Fagan Inspection Process — original IBM research +- SEI material on inspections and defect removal efficiency + +### Standards / Guidelines + +- MISRA C:2012 + Amendments\ + Not just rules, but rationale. + +- ISO 26262-6 (Software)\ + Functional safety thinking. + +- IEC 61508-3\ + General industrial functional safety. + +- EN 50128\ + Railway safety software (good patterns). + +- Barr Embedded C Rules + +### Web Resources + +- NASA Coding Guidelines +- CERT Secure Coding Guidelines for C +- Barr Group webinars (state machines, safety patterns) diff --git a/content/guides/cybersecurity/01-threat-modeling-foundations.md b/content/guides/cybersecurity/01-threat-modeling-foundations.md new file mode 100644 index 0000000..8904abd --- /dev/null +++ b/content/guides/cybersecurity/01-threat-modeling-foundations.md @@ -0,0 +1,125 @@ +--- +title: Threat Modelling Foundations +--- + +# Threat Modelling Foundations + +Threat modelling is a structured engineering process for identifying potential security threats, vulnerabilities, and attack vectors in a system before hardware is manufactured or software is deployed to production. + +In connected and IoT systems, threat modelling is essential because: +- **Remediation Costs Escalate**: A silicon or hardware defect discovered after manufacturing can cost millions in hardware recalls. +- **Physical Exposure**: Unlike cloud servers, connected devices operate in unmonitored physical environments accessible to attackers with oscilloscopes, debuggers, and logic analyzers. +- **High Consequence**: Compromised devices can disrupt critical infrastructure, manipulate industrial actuators, leak private telemetry, or pivot into enterprise cloud backends. + +--- + +## 1. The STRIDE Framework for Connected Systems + +The STRIDE methodology (originally conceived at Microsoft) categorizes threats relative to key security properties: + +| STRIDE Category | Security Property Violated | Connected Systems Manifestation | Mitigating Controls | +|:---|:---|:---|:---| +| **Spoofing** | Authenticity | Rogue device impersonating a legitimate edge sensor to send forged sensor data. | Hardware Root of Trust, per-device X.509 certificates, mTLS 1.3. | +| **Tampering** | Integrity | Modifying firmware in external SPI NOR flash or injecting forged command packets. | Cryptographically signed firmware, secure boot, authenticated encryption (AES-GCM). | +| **Repudiation** | Non-repudiability | Attacker issues an unauthorized valve-close actuation command and deletes local logs. | Append-only audit logging, cryptographically signed actuator transaction logs. | +| **Information Disclosure** | Confidentiality | Sniffing plaintext telemetry over Wi-Fi/cellular or dumping plaintext keys from flash memory. | TLS 1.3 encryption in transit, AES-256-XTS flash encryption at rest, PUF key storage. | +| **Denial of Service** | Availability | Jamming 2.4 GHz RF channels, flooding MQTT broker with QoS 2 bursts, battery exhaustion. | Rate limiting, hardware watchdog timers, low-power duty cycling, RF frequency hopping. | +| **Elevation of Privilege** | Authorisation | Exploiting a buffer overflow in an RTOS packet parser to gain supervisor execution mode. | Memory Protection Units (MPU), stack canaries, ARM TrustZone-M, least privilege execution. | + +--- + +## 2. Trust Boundaries & Data Flow Diagrams (DFDs) + +A Data Flow Diagram (DFD) shows how data moves through a system, and marks the **Trust Boundaries**. These are the conceptual or physical barriers where data passes between different levels of privilege or trust. + +```mermaid +flowchart LR + subgraph Untrusted_Physical_Domain["Untrusted Physical Domain"] + Attacker["Physical Attacker / Probe"] + RF["Radio Frequency Intercept"] + end + + subgraph Edge_Device["Connected Edge Device (Trust Boundary A)"] + Sensor["Sensor Transducer"] + MCU["Microcontroller / SoC"] + Flash["External SPI Flash"] + Crypto["Hardware Secure Element"] + + Sensor -->|Analog / I2C| MCU + MCU <-->|Encrypted / Signed| Flash + MCU <-->|"I2C / SPI (Secured)"| Crypto + end + + subgraph Cloud_Platform["Enterprise Cloud Ingress (Trust Boundary B)"] + Gateway["API / MQTT Gateway"] + AuthService["Device Identity & Auth"] + DataLake["Telemetry Storage"] + + Gateway <--> AuthService + Gateway --> DataLake + end + + RF -.->|Passive Sniffing| MCU + Attacker -.->|Bus Snooping / JTAG| Flash + MCU == mTLS 1.3 / Cellular ==> Gateway +``` + +### Critical Trust Boundaries +1. **Perimeter Physical Boundary (Boundary A)**: Between physical surroundings and the device circuit board. Mitigated by potting, enclosure tamper switches, and disabling debug interfaces. +2. **Chip-to-Chip Interconnect Boundary**: Between the host MCU and external SPI flash / sensor chips. Mitigated by encrypted buses and signed payloads. +3. **Network Boundary (Boundary B)**: Between the device edge transceiver and cloud edge load balancers. Mitigated by mutual authentication and ephemeral TLS handshakes. +4. **Internal Privilege Boundary**: Between unprivileged application tasks and privileged RTOS kernel / secure world (ARM TrustZone). + +--- + +## 3. DREAD Risk Assessment Model + +Once threats are cataloged via STRIDE, they must be ranked to prioritize engineering remediation. The DREAD model scores each threat from 1 to 10 across five criteria: + +$$\mathrm{Risk\ Score} = \frac{\mathrm{Damage} + \mathrm{Reproducibility} + \mathrm{Exploitability} + \mathrm{Affected\ Users} + \mathrm{Discoverability}}{5}$$ + +| Metric | High (8–10) | Medium (4–7) | Low (1–3) | +|:---|:---|:---|:---| +| **Damage Potential (D)** | Full system bricking, master secret leakage, physical harm. | Denial of single device service, partial non-sensitive data leak. | Minor informational leak, temporary aesthetic glitch. | +| **Reproducibility (R)** | 100% reliable exploit via automated script. | Requires specific timing, temperature, or transient conditions. | Extremely difficult, probabilistic race condition. | +| **Exploitability (E)** | Script-kiddie level; public proof of concept tool available. | Requires moderate reverse engineering skills and equipment ($500). | Requires state-sponsored lab, decapping chip, FIB equipment ($100k+). | +| **Affected Users (A)** | Entire installed fleet (e.g. 500,000 devices). | A specific hardware batch or geographic subnet. | Single isolated physical unit. | +| **Discoverability (D)** | Obvious in plaintext network traffic or open debug port. | Requires decompiling firmware binaries. | Obscure logic flaw buried deep in proprietary microcode. | + +--- + +## 4. Attack Tree Example: Compromising Edge Device Cryptographic Key + +Attack trees provide a formal, hierarchical method to model how an adversary can achieve a specific malicious objective. + +```mermaid +graph TD + Root["Root: Extract Device Private Key"] + + A["Extract via Network"] + B["Extract via Hardware Debug Port"] + C["Extract from Flash Storage"] + D["Extract via Side-Channel Attack"] + + Root --> A + Root --> B + Root --> C + Root --> D + + A --> A1["Exploit Remote Code Execution (RCE) in TLS Stack"] + A --> A2["Memory Leak via Heartbleed-style Vulnerability"] + + B --> B1["Connect JTAG/SWD Debugger"] + B1 --> B2["Dump SRAM Memory During Execution"] + + C --> C1["Desolder SPI NOR Flash Chip"] + C1 --> C2["Read Flash via External EEPROM Programmer"] + + D --> D1["Differential Power Analysis (DPA) on AES Engine"] + D --> D2["Electromagnetic Radiation Analysis (EMA)"] +``` + +### Countermeasure Mapping +- **For B (Debug Port)**: Blow permanent silicon eFuses during production line provisioning to disable JTAG/SWD permanently (`SWD_DISABLE=1`). +- **For C (Flash Extraction)**: Store root keys exclusively inside a hardware Secure Element (e.g., ATECC608, NXP SE050, or STSAFE-A110) where private keys never leave the silicon die. +- **For D (Side-Channel)**: Use crypto co-processors equipped with internal clock jittering, dummy instruction injection, and differential masking logic. diff --git a/content/guides/cybersecurity/02-hardware-and-silicon-security.md b/content/guides/cybersecurity/02-hardware-and-silicon-security.md new file mode 100644 index 0000000..a61938f --- /dev/null +++ b/content/guides/cybersecurity/02-hardware-and-silicon-security.md @@ -0,0 +1,161 @@ +--- +title: Hardware and Silicon Security +--- + +# Hardware and Silicon Security + +Software and firmware security guarantees are meaningless if the underlying silicon and circuit board can be compromised. Hardware security forms the immovable anchor of the entire security architecture—known as the **Hardware Root of Trust (RoT)**. + +--- + +## 1. Hardware Root of Trust (RoT) Architecture + +A Hardware Root of Trust is an inherently trusted execution block embedded into the silicon die that cannot be modified by software or firmware. + +```mermaid +flowchart TD + subgraph Silicon_Boundary["Microcontroller Silicon Die"] + subgraph BootROM["Masked Boot ROM (Immutable)"] + ROM_Code["Factory Boot Code"] + Public_Key_Hash["Manufacturer Public Key Hash"] + end + + subgraph OTP_Fuse["eFuse / OTP Memory"] + Device_UUID["Unique Device ID (UUID)"] + Master_Key["Encrypted Device Master Key"] + Security_Bits["JTAG Lock & Rollback Counter"] + end + + subgraph Crypto_Engine["Hardware Crypto Accelerator"] + TRNG["True Random Number Generator"] + AES["AES-256-GCM"] + ECC["ECDSA P-256 / Ed25519 Engine"] + SHA["SHA-256 / SHA-3"] + end + + subgraph Secure_Storage["Secure Element / Enclave"] + Private_Key["Device Private Key (Non-exportable)"] + end + end + + BootROM -->|Verifies Stage 1| SBL["Stage 1 Bootloader in Flash"] + Crypto_Engine -->|Hardware Decryption| SRAM["Secure Internal SRAM"] +``` + +### Core RoT Responsibilities +1. **Initial Execution**: The processor resets exclusively into immutable masked ROM burned during silicon lithography. +2. **Key Storage**: Cryptographic private keys reside in tamper-resistant registers, isolated by hardware memory controllers from standard CPU core reads. +3. **Cryptographic Engine**: Dedicated hardware offload for cryptographic primitives (AES, SHA, ECC, TRNG) isolated from general execution pipelines. + +--- + +## 2. Authenticated Secure Boot Chain + +Secure Boot establishes an unbroken chain of trust starting from immutable silicon ROM up to the user-space application tasks. + +```mermaid +sequenceDiagram + autonumber + participant ROM as Masked Boot ROM (Silicon) + participant FSBL as First Stage Bootloader (Flash) + participant SSBL as Second Stage Bootloader (MCUboot) + participant OS as RTOS Kernel & Apps + + Note over ROM: CPU Reset Vector triggered + ROM->>ROM: Read Public Key Hash from eFuses + ROM->>FSBL: Read FSBL Header and Signature + ROM->>ROM: Compute SHA-256 of FSBL & Verify Signature + alt Signature Valid + ROM->>FSBL: Jump to FSBL execution in SRAM + else Signature Invalid + ROM->>ROM: HALT execution & Assert Tamper Pin + end + + Note over FSBL: Initialise clocks and DDR/Flash controllers + FSBL->>SSBL: Verify SSBL Signature with Manufacturer Public Key + alt Valid + FSBL->>SSBL: Jump to SSBL (MCUboot) + else Invalid + FSBL->>FSBL: Fall back to Recovery Image or HALT + end + + Note over SSBL: Inspect A/B Slot Header & Rollback Counter + SSBL->>OS: Verify Application Signature & Hash + alt Valid + SSBL->>OS: Transfer Execution to RTOS Kernel + else Invalid + SSBL->>FSBL: Rollback to Slot B Previous Good Firmware + end +``` + +### Critical Rules for Secure Boot +- **No Signature Bypass**: If signature verification fails, execution must immediately halt; fallback to an unverified recovery mode is strictly prohibited. +- **Constant-Time Verification**: Cryptographic signature validation must run in constant time to prevent timing attacks. +- **Secure Memory Cleansing**: Before execution passes to user code, all bootloader cryptographic keys, hashes, and internal intermediate state in SRAM must be zeroed out. + +--- + +## 3. Silicon eFuses & One-Time Programmable (OTP) Memory + +Electronic fuses (eFuses) are microscopic semiconductor elements that can be permanently blown by applying a controlled programming voltage ($V_{PP}$). Once blown, the transition from `0` to `1` is physically irreversible. + +| eFuse Block | Typical Bit Size | Function & Lifecycle Purpose | +|:---|:---|:---| +| `MAC_ADDR / UUID` | 48–128 bits | Immutable factory hardware identifier. | +| `PUBKEY_HASH` | 256–384 bits | Cryptographic SHA-256/384 hash of the authorized firmware signing public key. | +| `FLASH_ENC_KEY` | 256 bits | Symmetric key for on-the-fly hardware flash decryption. Generated on first boot and locked from read-back. | +| `DEBUG_DISABLE` | 1–4 bits | Permanently blows JTAG and SWD hardware debug interfaces. | +| `ROLLBACK_COUNTER` | 32–64 bits | Monotonically increasing bit-field preventing downgrade attacks to vulnerable firmware versions. | +| `SECURE_BOOT_EN` | 1 bit | Locks the processor into mandatory signature enforcement permanently. | + +> [!CAUTION] +> Blowing eFuses is an irreversible physical operation. Production programming tooling must implement multi-stage sanity checks, voltage verification, and confirmation prompts before committing fuse bits. + +--- + +## 4. Physically Unclonable Functions (PUF) + +A Physically Unclonable Function (PUF) exploits microscopic, uncontrollable physical variations introduced during semiconductor wafer fabrication (such as threshold voltage variations in SRAM cell transistor pairs). + +- **How It Works**: When an uninitialised SRAM array powers up, each cell randomly settles into a `0` or `1` state with a highly deterministic bias unique to that specific silicon die. +- **Key Generation on Demand**: A helper algorithm and error-correction code (ECC) extract an encryption key from this fingerprint at runtime. +- **Zero Key at Rest**: When the device is powered off, the key ceases to exist anywhere in the physical universe—eliminating flash-memory dumping attacks. + +--- + +## 5. Physical Tamper Resistance & Debug Interface Hardening + +### Disabling Debug Ports +Leaving JTAG or Serial Wire Debug (SWD) active in production is a fatal vulnerability. Attackers can halt execution, inspect SRAM registers containing private keys, or inject malicious shellcode. + +- **Pre-Production Phase**: Enable password-authenticated debug modes (for example, ARM ADIv5 authentication). +- **Production Phase**: Blow the silicon eFuses (`SWD_DISABLE=1`) to physically disconnect the internal debug multiplexer. +- **Pin Reassignment**: Reconfigure SWDIO/SWCLK GPIO pins as general-purpose I/O or tie them to ground through pull-downs. + +### Mitigation Against Fault Injection & Glitch Attacks +Physical attackers use voltage glitching (dropping supply voltage $V_{DD}$ for 50 nanoseconds) or clock glitching to cause the CPU instruction decoder to skip an authorisation conditional jump: + +```c +/* Vulnerable Code: susceptible to single-instruction glitch skipping */ +if (verify_signature(firmware) == SUCCESS) { + boot_firmware(); +} +``` + +```c +/* Hardened Defense: Dual-check with complementary state variables */ +volatile uint32_t check1 = 0x55555555; +volatile uint32_t check2 = 0xAAAAAAAA; + +check1 = verify_signature_stage1(firmware); +insert_random_delay(); +check2 = verify_signature_stage2(firmware); + +if ((check1 == SIGNATURE_VALID) && (check2 == SIGNATURE_VALID)) { + // Second check ensures a single skipped instruction cannot bypass verification + boot_firmware(); +} else { + tamper_alarm_trigger(); + system_hard_reset(); +} +``` diff --git a/content/guides/cybersecurity/03-transport-and-network-security.md b/content/guides/cybersecurity/03-transport-and-network-security.md new file mode 100644 index 0000000..03878aa --- /dev/null +++ b/content/guides/cybersecurity/03-transport-and-network-security.md @@ -0,0 +1,111 @@ +--- +title: Transport and Network Security +--- + +# Transport and Network Security + +Edge devices transmit mission-critical telemetry and receive remote actuation commands over public, unmonitored networks (cellular 4G/5G, Wi-Fi, Ethernet, LoRaWAN, satellite). Network communication must assume an untrusted transport medium vulnerable to eavesdropping, packet modification, and man-in-the-middle (MitM) interception. + +--- + +## 1. Mutual TLS (mTLS 1.3) Architecture + +Standard web TLS only authenticates the server to the client. In connected systems, **Mutual TLS (mTLS)** is mandatory: both the cloud ingress gateway and the edge device cryptographically prove their identities to each other before any application data is exchanged. + +```mermaid +sequenceDiagram + autonumber + participant Device as Connected Edge Device + participant Gateway as Cloud IoT Ingress (mTLS) + participant PKI as Enterprise Device PKI / CA + + Note over Device: Reads Device Leaf Cert & Private Key + Device->>Gateway: ClientHello (Supported Ciphers, TLS 1.3, KeyShare) + Gateway->>Device: ServerHello (KeyShare, Server Certificate, CertificateRequest) + Note over Device: Verifies Server Cert against embedded Root CA + Device->>Gateway: Certificate (Device X.509 Cert) + Device->>Gateway: CertificateVerify (Signed with Device Private Key) + Device->>Gateway: Finished + Note over Gateway: Verifies Device Cert against Intermediate Device CA + Gateway->>Device: Finished + Note over Device,Gateway: Handshake Complete: Ephemeral AES-256-GCM Session Key Established + Device->>Gateway: Encrypted Application Telemetry (MQTT/HTTP) +``` + +### Why TLS 1.3 is Mandatory over TLS 1.2 +- **Zero Insecure Ciphers**: TLS 1.3 permanently eliminates vulnerable legacy algorithms (RC4, 3DES, CBC mode ciphers, MD5, SHA-1, static RSA key exchange). +- **Mandatory Forward Secrecy**: Every connection negotiates ephemeral Diffie-Hellman keys (`ECDHE`). Even if a device's long-term private key is extracted years later, previously recorded network traffic cannot be decrypted. +- **Faster Handshake**: Handshake overhead is reduced from two round trips (2-RTT) to one round trip (1-RTT), saving battery power and cellular bandwidth. + +--- + +## 2. Public Key Infrastructure (PKI) for IoT Fleets + +A scalable, secure X.509 Public Key Infrastructure requires strict hierarchical separation: + +```mermaid +graph TD + RootCA["Offline Root Certificate Authority (Self-Signed, 4096-bit RSA / P-384, Air-Gapped)"] + + SubCA1["Intermediate Device CA: Model X (Online HSM, Valid 5 Years)"] + SubCA2["Intermediate Cloud Server CA (Online HSM, Valid 2 Years)"] + + RootCA --> SubCA1 + RootCA --> SubCA2 + + Dev1["Device Leaf Cert: SN-0001 (Valid 3 Years, ECDSA P-256)"] + Dev2["Device Leaf Cert: SN-0002 (Valid 3 Years, ECDSA P-256)"] + + Server1["Cloud MQTT Endpoint: mqtt.iot.company.com"] + + SubCA1 --> Dev1 + SubCA1 --> Dev2 + SubCA2 --> Server1 +``` + +### Key Provisioning Guidelines +- **Never Share Private Keys**: Each device must generate or be injected with a unique, cryptographically distinct private key. Shared symmetric passwords across a fleet are strictly prohibited. +- **ECC over RSA**: Use Elliptic Curve Cryptography (`secp256r1` or `ed25519`) instead of 2048/4096-bit RSA. A 256-bit ECC key delivers equivalent security to a 3072-bit RSA key with an 85% smaller payload, saving flash space and computational battery life. +- **Air-Gapped Root CA**: The Root CA must never be directly connected to the internet. It exists exclusively to sign Intermediate CAs and CRLs, stored in tamper-proof Hardware Security Modules (HSMs). + +--- + +## 3. Automated Certificate Enrolment (EST & SCEP) + +Manual certificate injection during field deployment does not scale and introduces human operational risks. Devices should leverage **Enrolment over Secure Transport (EST, RFC 7030)**: + +1. **Bootstrap Phase**: The factory injects an initial Manufacturer Device Identity certificate (IDevID, IEEE 802.1AR) tied to the hardware Root of Trust. +1. **First Cloud Contact**: The device boots, connects to the provisioning server using its IDevID, and issues a Certificate Signing Request (CSR) via EST over HTTPS. +1. **Operational Issuance**: The PKI validates the hardware identity, generates an operational Local Device Identity certificate (LDevID), and installs it securely on the device. +1. **Automated Renewal**: Before certificate expiration (for example, 30 days remaining), the firmware automatically requests a renewed leaf certificate via EST without requiring manual physical servicing. + +--- + +## 4. Secure Application Protocols + +### MQTT over TLS (Port 8883) +MQTT is the dominant publish-subscribe protocol for resource-constrained edge telemetry. + +| Feature | Standard (Unsafe) MQTT | Hardened Enterprise MQTT Standard | +|:---|:---|:---| +| **Port & Wire Security** | Port 1883 (Plaintext TCP) | Port 8883 (Encrypted mTLS 1.3) | +| **Client Authentication** | Static username/password | Client X.509 certificate subject matching Common Name (UUID) | +| **Topic Authorisation** | Unrestricted wildcards (`#`, `+`) | Fine-grained Access Control Lists (ACLs) per device UUID | +| **Topic Namespace** | Global shared topics (`sensors/data`) | Isolated tenancy: `devices/{client_id}/telemetry` | +| **Payload Integrity** | Unchecked raw bytes | Protobuf / JSON schema with optional HMAC-SHA256 signature | + +### CoAP over DTLS (RFC 7252 / RFC 7925) +For extremely resource-constrained devices communicating over UDP, Constrained Application Protocol (CoAP) secured with Datagram Transport Layer Security (DTLS 1.2 / 1.3) provides: +- Low-overhead binary message headers (4 bytes base header). +- Connection ID (CID) support to survive network NAT rebinding and IP address changes without renegotiating full handshakes. +- Pre-shared Key (PSK) or lightweight raw public key (RPK) cipher suites where full X.509 certificate chains exceed packet MTU limits. + +--- + +## 5. Network Microsegmentation & Zero Trust Edge + +Connected devices must operate under a **Zero Trust** architecture: + +1. **No Inbound Open Ports**: Devices must never listen on open public TCP/UDP ports (for example, Telnet or unauthenticated HTTP servers). All communication channels must be outbound connections initiated by the device toward cloud endpoints. +1. **Local Interface Isolation**: If a device includes multiple network interfaces (for example, an Ethernet port for an industrial sensor bus and a cellular modem for cloud reporting), the interfaces must be strictly isolated via kernel network namespaces and packet filtering rules (`iptables` / `nftables`). Bridging or forwarding packets between untrusted local networks and enterprise networks is forbidden. +1. **DNS-over-TLS (DoT) / DNSSEC**: Guard against DNS poisoning and redirect attacks by encrypting DNS lookups or using hardcoded, verified server public keys. diff --git a/content/guides/cybersecurity/04-secure-firmware-updates-ota.md b/content/guides/cybersecurity/04-secure-firmware-updates-ota.md new file mode 100644 index 0000000..1df7b07 --- /dev/null +++ b/content/guides/cybersecurity/04-secure-firmware-updates-ota.md @@ -0,0 +1,125 @@ +--- +title: "Secure Firmware Updates & OTA" +--- + +# Secure Firmware Updates & OTA + +Over-the-Air (OTA) firmware updates are the single most critical capability in connected systems engineering. Without reliable, secure OTA update mechanisms, a newly discovered zero-day vulnerability in an RTOS or crypto library renders an entire deployed fleet permanently exposed, potentially necessitating costly physical recalls. + +However, the update mechanism itself is often the most dangerous attack vector: an adversary who compromises the OTA pipeline can push malicious firmware to hundreds of thousands of devices simultaneously. + +--- + +## 1. Dual-Slot A/B Partition Architecture + +To guarantee high availability and prevent device bricking due to sudden power loss or corrupted downloads, production hardware must implement a symmetric dual-bank (A/B) flash partition layout. + +```mermaid +flowchart TD + subgraph Flash_Memory["Internal / External Flash Memory Layout"] + direction TB + BL["Bootloader Sector (Protected / Read-Only)"] + SlotA["Slot A (Active Running Firmware v2.1.0)"] + SlotB["Slot B (Candidate Staging Firmware v2.2.0)"] + Scratch["Scratch / Swap Buffer (Optional)"] + UserData["Encrypted Non-Volatile User Data (NVS)"] + end + + BL -->|Checks Slot Validity| Decision{"Which Slot to Boot?"} + Decision -->|Slot A: Valid & Confirmed| RunA["Execute Slot A"] + Decision -->|Slot B: New Valid Staged Update| TestB["Execute Slot B in Trial Mode"] +``` + +### Partition Sizing & Invalidation Rules +- **Physical Isolation**: Slot A and Slot B must each be sized to accommodate 100% of the maximum allowable application binary size plus the crypto header. +- **Power-Loss Atomicity**: Flash programming writes data sequentially. If power drops mid-transfer, Slot B remains in an invalid state. The bootloader detects this (via checksum or broken signature) and continues booting Slot A unimpeded. + +--- + +## 2. Cryptographic Signature Verification Flow + +Every firmware update bundle must be signed offline in a secure build environment before distribution. + +```mermaid +sequenceDiagram + autonumber + participant Cloud as OTA Cloud Dispatcher + participant Device as Device OTA Agent + participant Bootloader as Secure Bootloader (MCUboot) + + Cloud->>Device: Update Manifest (Version 2.2.0, SHA-256 Hash, Ed25519 Signature, Size) + Note over Device: Validates Manifest Signature & Size + Device->>Cloud: Request Chunked Download (HTTPS / CoAP) + loop Stream Binary Chunks + Cloud->>Device: Transfer 4KB Chunk + Device->>Device: Write Chunk to Inactive Slot B & Accumulate SHA-256 + end + Note over Device: Complete SHA-256 verification of Slot B + Device->>Device: Verify Ed25519 Signature with embedded Vendor Public Key + Device->>Device: Set Boot Flag: "Test Boot Slot B on next restart" + Device->>Device: Trigger Controlled System Reboot + + Note over Bootloader: Device Boots + Bootloader->>Bootloader: Inspect Boot Flag -> Sees Slot B Trial Mode + Bootloader->>Bootloader: Re-verify Slot B Crypto Signature & Anti-Rollback Counter + alt Signature & Counter OK + Bootloader->>Device: Launch Slot B Kernel (Trial Mode) + Note over Device: Device performs Self-Test (Network, Sensors, Cloud Checkin) + alt Self-Test Passed within 60s + Device->>Device: Mark Slot B as "CONFIRMED_PERMANENT" + Device->>Device: Increment Hardware Anti-Rollback eFuse + else Self-Test Failed or Watchdog Timeout + Device->>Bootloader: Hardware Watchdog Triggers Reset + Bootloader->>Device: Revert Boot Flag -> Boot Slot A (Fallback) + end + else Invalid Signature or Outdated Counter + Bootloader->>Device: Invalidate Slot B -> Boot Slot A + end +``` + +--- + +## 3. Monotonic Anti-Rollback Counter Enforcement + +A **downgrade attack** occurs when an attacker intercepts a legitimate, authentically signed older firmware version that contains a known, patchable vulnerability and flashes it onto the target device. Because the old binary is signed by the manufacturer, simple signature checks will pass. + +### Silicon Monotonic Counters +To defeat downgrade attacks, devices must track a hardware-enforced **Security Version Number (SVN)**: + +1. **Firmware Header Declaration**: Each firmware header embeds a monotonically increasing integer (`security_version = 7`). +1. **Hardware Counter Check**: During boot, the bootloader compares the binary's `security_version` against the silicon eFuse counter value ($C_{\mathrm{fuse}}$). + - If $\mathrm{security\_version} < C_{\mathrm{fuse}}$: **REJECT & HALT**. (Downgrade attack detected). + - If $\mathrm{security\_version} \ge C_{\mathrm{fuse}}$: **PERMIT**. +1. **Fuse Burning on Confirmation**: Once the new firmware completes its post-boot self-tests and is confirmed stable, the device permanently burns additional eFuse bits to raise $C_{\mathrm{fuse}}$ to match the new security version. + +--- + +## 4. Fault-Tolerant Trial Mode & Watchdog Timers + +A major risk in remote updates is a build that passes boot verification but fails 5 seconds later, for example by failing to connect to the cellular network. The device is then permanently unreachable. + +### The Self-Test / Confirmation Pattern +1. **Unconfirmed State**: When a new image boots for the first time, it enters state `STATE_TRIAL`. +1. **Hardware Watchdog Countdown**: An internal hardware watchdog timer is primed for 120 seconds. +1. **Connectivity Verification**: The application must: + - Initialise all hardware drivers without fault. + - Establish network connectivity. + - Complete an authenticated handshake with the cloud service. +1. **Marking Permanent**: Only when all checks succeed does the application call `boot_set_confirmed()`. +1. **Automatic Fallback**: If the application hangs, panics, or suffers an assert failure before confirming, the hardware watchdog resets the processor. On the subsequent boot, the bootloader observes that the trial image failed to confirm and automatically restores the previous working firmware in Slot A. + +--- + +## 5. Fleet Deployment Strategies & Blast Radius Control + +Never deploy a firmware update to 100% of an active fleet in a single batch. Deployments must follow a phased **Ring Deployment Model**: + +| Phase / Ring | Target Audience | Fleet % | Soak Time | Gate Criteria | +|:---|:---|:---|:---|:---| +| **Ring 0: Canary Lab** | Internal test benches & automated hardware rigs | 0.1% (50 units) | 48 hours | Zero crash dumps, 100% test harness pass rate. | +| **Ring 1: Internal Dogfood** | Internal company deployments & employee units | 1% (500 units) | 7 days | Battery drain metrics within ±3% of baseline. | +| **Ring 2: Pilot Fleet** | Selected beta customers with low-impact workloads | 5% (2,500 units) | 14 days | Error log rate below 0.01% of transactions. | +| **Ring 3: Staged Rollout** | General customer fleet partitioned by geographic timezone | 25% $\to$ 50% $\to$ 100% | 21 days | Continuous monitoring of cloud reconnection latencies. | + +> [!TIP] +> Always implement an automated cloud **Emergency Pause Circuit-Breaker**. If the crash report rate for a candidate version exceeds a predetermined threshold (for example, >0.1%), the cloud OTA service must instantly halt all in-progress downloads across the fleet. diff --git a/content/guides/cybersecurity/05-standards-and-compliance.md b/content/guides/cybersecurity/05-standards-and-compliance.md new file mode 100644 index 0000000..544f02a --- /dev/null +++ b/content/guides/cybersecurity/05-standards-and-compliance.md @@ -0,0 +1,99 @@ +--- +title: "Standards & Regulatory Compliance" +--- + +# Standards & Regulatory Compliance + +Connected systems and IoT engineering have transitioned from an era of voluntary best-practice recommendations into a regime of strict, legally binding statutory regulations. Failure to comply with international product security standards now carries severe penalties, including market bans, mandatory product recalls, and multi-million pound fines. + +This chapter details the primary international cybersecurity standards and provides actionable compliance checklists for engineering teams. + +--- + +## 1. ETSI EN 303 645: European Standard for Consumer IoT + +ETSI EN 303 645 is the recognized global baseline standard for consumer and commercial Internet of Things devices. It specifies 13 baseline security provisions: + +```mermaid +mindmap + root((ETSI EN 303 645)) + Prohibit Universal Passwords + Vulnerability Reporting Policy + Software Update Mechanism + Secure Key Storage + Secure Communications + Minimize Attack Surfaces + Personal Data Protection + Software Integrity Validation + System Telemetry Resilience + Outage Monitoring + Consumer Data Erasure + Installation Ease + Telemetry Validation +``` + +### Top 3 Mandatory Technical Provisions +1. **Provision 5.1-1: No Universal Default Passwords**: Pre-set, factory-wide default passwords (for example, `admin`/`admin` or `root`/`password`) are banned. Devices must either require the user to configure a unique password on first setup or generate a unique random password per device printed on a tamper-evident label. +1. **Provision 5.2-1: Implement a Vulnerability Disclosure Policy (VDP)**: The manufacturer must provide a publicly accessible contact channel (for example, a security email address or web portal) for ethical security researchers to report vulnerabilities, with acknowledged receipt within 5 business days. +1. **Provision 5.3-1: Secure Software Updates**: Software updates must be cryptographically signed, verified before installation, and delivered over authenticated channels (see Chapter 4). + +--- + +## 2. UK PSTI Act 2022 (In Force 2024) + +The **UK Product Security and Telecommunications Infrastructure (PSTI) Act 2022** is legally enforceable legislation covering all internet-connectable products manufactured, imported, or sold in the United Kingdom. + +### The Three Core Legal Mandates +1. **Unique Passwords**: Passwords must be unique per product or force the user to set a strong password during initial onboarding. +1. **Vulnerability Disclosure Point of Contact**: A dedicated public URL and security email must be published to receive vulnerability reports. +1. **Defined Minimum Support Period**: Manufacturers must state at the point of sale (on product packaging and product websites) the exact minimum calendar date until which the product will receive security updates (for example, *"Guaranteed security updates until at least December 2031"*). + +> [!WARNING] +> Non-compliance with the UK PSTI Act carries fines of up to **£10 million or 4% of worldwide gross revenue**, alongside mandatory product seizures and sales bans enforced by the Office for Product Safety and Standards (OPSS). + +--- + +## 3. EU Cyber Resilience Act (CRA) + +The **EU Cyber Resilience Act (CRA)** introduces comprehensive, mandatory cybersecurity requirements for all "products with digital elements" sold within the European Single Market. + +### Key CRA Requirements +- **CE Marking**: Products without formal cybersecurity conformity assessment cannot bear the CE mark and cannot legally be sold in the European Union. +- **Security Throughout Lifecycle**: Mandatory security support and vulnerability patching for the expected product lifetime or a minimum of 5 years. +- **24-Hour Vulnerability Reporting**: Manufacturers must report actively exploited vulnerabilities to the EU Cybersecurity Agency (ENISA) and the relevant national CSIRT within 24 hours of becoming aware of them. +- **Software Bill of Materials (SBOM)**: Manufacturers must maintain machine-readable SBOMs (CycloneDX or SPDX format) cataloging every open-source library, RTOS kernel, and third-party driver embedded in the firmware. + +--- + +## 4. Standards Comparison Matrix + +| Regulatory Standard | Jurisdiction | Enforcement | Scope | Core Technical Focus | +|:---|:---|:---|:---|:---| +| **ETSI EN 303 645** | Global / Europe | Voluntary baseline & regulatory benchmark | Consumer & commercial IoT | 13 cybersecurity provisions: no default passwords, VDP, signed updates. | +| **UK PSTI Act** | United Kingdom | Legally binding (Statutory Law) | All connectable consumer devices | Unique passwords, public VDP, declared minimum support date. | +| **NIST IR 8259A/B** | United States | Federal guidance & IoT labeling benchmark | IoT devices & federal procurement | 6 device capabilities: device ID, configuration, data protection, logical interfaces, updates, cybersecurity state. | +| **EU Cyber Resilience Act (CRA)** | European Union | Legally binding (EU Regulation) | All hardware & software products | CE mark requirement, SBOM, 24-hour vulnerability notification, secure by default. | +| **IEC 62443-4-2** | Global Industrial | Contractual & certification standard | Industrial automation & control systems (IACS) | Security Levels (SL 1–4): hardware tamper defence, role-based access, network segmentation. | + +--- + +## 5. Vulnerability Disclosure Policy (VDP) & `security.txt` + +To comply with ETSI EN 303 645 and UK PSTI, every product organisation must publish an RFC 9116 compliant `security.txt` file at the root of their public web domains (`https://example.com/.well-known/security.txt`): + +```ini +# Production Security Vulnerability Reporting Policy +Contact: mailto:security@example.com +Encryption: https://example.com/pgp-key.asc +Acknowledgments: https://example.com/security/hall-of-fame +Preferred-Languages: en +Canonical: https://example.com/.well-known/security.txt +Policy: https://example.com/security-policy.html +Expires: 2027-12-31T23:59:59.000Z +``` + +### Response SLA Commitments +1. **Initial Acknowledgment**: Send confirmation receipt to the reporting researcher within 48 hours. +1. **Triage Assessment**: Complete initial vulnerability reproduction and severity scoring (CVSS v3.1) within 5 business days. +1. **Remediation & Patch**: Develop, test, sign, and release an OTA security patch within 30–90 days (depending on CVSS severity). +1. **Coordinated Disclosure**: Publish a public Security Advisory (CVE) simultaneously with patch availability to protect the user base. diff --git a/content/guides/cybersecurity/index.md b/content/guides/cybersecurity/index.md new file mode 100644 index 0000000..ce68858 --- /dev/null +++ b/content/guides/cybersecurity/index.md @@ -0,0 +1,56 @@ +--- +title: Cyber security and threat modelling for connected systems +--- + +# Cyber security and threat modelling for connected systems + + +## Executive Summary + +Connected devices, IoT edge nodes, and industrial telemetry systems operate in hostile physical and logical environments. Unlike traditional cloud microservices protected behind perimeter firewalls and managed data centers, edge hardware is subject to physical tampering, intermittent untrusted networks, constrained compute budgets, and multi-decade deployment lifecycles. + +This guide establishes an end-to-end engineering standard and threat modelling methodology for architecting, building, and operating secure connected systems. + +```mermaid +graph TD + A[Silicon & Hardware Layer] -->|Hardware Root of Trust| B[Boot & Firmware Layer] + B -->|Cryptographic Verification| C[Runtime & Edge Applications] + C -->|mTLS 1.3 & Zero Trust| D[Network & Ingress Telemetry] + D -->|Cloud Ingestion & Auth| E[Enterprise Cloud & Data Platform] + + style A fill:#e1f5fe,stroke:#0288d1 + style B fill:#e8f5e9,stroke:#388e3c + style C fill:#fff3e0,stroke:#f57c00 + style D fill:#ede7f6,stroke:#512da8 + style E fill:#fce4ec,stroke:#c2185b +``` + +--- + +## Scope & Target Audience + +This guide applies across all connected systems and embedded software projects, serving: + +- **System & Software Architects**: Designing trust boundaries, cryptosystems, and threat models. +- **Embedded & Firmware Engineers**: Implementing secure boot, hardware RoT, memory protection, and OTA pipelines. +- **DevOps & Cloud Engineers**: Managing public key infrastructure (PKI), device provisioning, and fleet telemetry. +- **Security & Compliance Auditors**: Auditing products against ETSI EN 303 645, UK PSTI, NIST IR 8259, and EU CRA mandates. + +--- + +## Guide Structure + +| Chapter | Topic | Core Focus Areas | +|:---|:---|:---| +| **[01. Threat Modelling Foundations](01-threat-modeling-foundations.md)** | Methodology & Analysis | STRIDE framework, DREAD scoring, Trust Boundaries, Data Flow Diagrams (DFDs), and attack trees. | +| **[02. Hardware & Silicon Security](02-hardware-and-silicon-security.md)** | Edge Device Hardening | Hardware Root of Trust (RoT), Secure Boot, eFuse/OTP, PUFs, crypto engines, JTAG lock, and side-channel defence. | +| **[03. Transport & Network Security](03-transport-and-network-security.md)** | Secure Communications | Mutual TLS (mTLS 1.3), X.509 device certificates, automated enrolment (EST/SCEP), MQTT/TLS, and network isolation. | +| **[04. Secure Firmware Updates & OTA](04-secure-firmware-updates-ota.md)** | Fleet Lifecycle Maintenance | Dual-slot A/B flash partitions, Ed25519/ECDSA signature verification, anti-rollback monotonic counters, and phased rollouts. | +| **[05. Standards & Regulatory Compliance](05-standards-and-compliance.md)** | Governance & Certification | ETSI EN 303 645, UK PSTI Act 2022, NIST IR 8259, EU Cyber Resilience Act (CRA), and Vulnerability Disclosure Policies (VDP). | + +## Key Engineering Principles + +1. **Security by Design, Not by Patch**: Security controls must be embedded into silicon, hardware schematics, and protocol designs from day zero. +1. **Assume Breach at the Edge**: Physical possession of a device must never compromise fleet-wide master secrets or cloud service credentials. +1. **Defence in Depth**: Every layer (silicon, bootloader, RTOS, application, transport, cloud) must enforce independent security barriers. +1. **Automated Cryptographic Agility**: Hardware and firmware architectures must accommodate key rotations, certificate renewals, and cipher suite upgrades over the full product lifetime. diff --git a/content/guides/decision-records.md b/content/guides/decision-records.md new file mode 100644 index 0000000..7369b99 --- /dev/null +++ b/content/guides/decision-records.md @@ -0,0 +1,218 @@ +--- +title: Writing ADRs and RFCs +--- + +# Writing ADRs and RFCs + +An RFC proposes a significant change and gathers views before anyone decides. An ADR records a decision once it is made, with the context that led to it. The rules for both are in [EG-008](../guidelines/decision-records.md); this guide is about writing them well. + +## Architecture decision records + +This part explains **what an ADR is**, **when to write one** and what to **include**. It also covers how to use ADRs well in projects that follow docs as code or a structured architecture practice. + +### What Is an ADR? + +An **ADR (Architecture Decision Record)** is a short, focused document that captures: + +- A **decision** that was made +- The **context** in which it was made +- The **reasoning** behind the decision + +Think of it as the historical logbook of your architecture. It replaces tribal knowledge with written rationale. + +### When to Write an ADR + +Write an ADR when: + +- A **non-trivial decision** has been made about architecture, design, tooling, process, or standards +- The decision has **long-term impact** +- You want to **record alternatives considered** +- You want others to understand **why** something was done + +You don't need an ADR for: + +- Obvious or universally accepted patterns +- Reversible or low-impact choices + +### ADR vs RFC + +An RFC drives debate on potential solutions; an ADR records the chosen outcome. See companion standard [Writing ADRs and RFCs](decision-records.md). + +| Dimension | RFC (Request for Comments) | ADR (Architecture Decision Record) | +| :--- | :--- | :--- | +| **Primary Purpose** | Propose options, solicit feedback, drive consensus | Record final, agreed architectural choice and rationale | +| **Stage in Lifecycle** | Early / exploratory (pre-decision) | Concluding / binding (post-decision) | +| **Mutability** | Mutable (evolves during review and debate) | Immutable (historical log once accepted) | +| **Scope of Alternatives** | Explores multiple viable options in depth | Summarises chosen option vs rejected alternatives | +| **Typical Flow** | RFC drafted ➔ Reviewed by stakeholders | RFC accepted ➔ ADR recorded into catalogue | + +### The template + +Use the [ADR template](../templates/adr.md) in the Templates section. + +### Context + +> What led to this decision? What problem or scenario required a choice to be made? + +### Decision + +> State the decision clearly and succinctly. Be direct. + +### Rationale + +> Why this decision? What were the trade-offs? +> Include supporting facts, reasoning, constraints, and priorities. + +### Consequences + +> What will change as a result? Are there any risks, costs, or follow-up actions? + +### Alternatives Considered + +> (Optional) List other options considered and why they were rejected. + +### Related + +> Link to related RFCs, ADRs, diagrams, repos, or documents. + +``` + +## When to Use an ADR (Decision Tree) + +```mermaid +graph TD + A[Was a technical or architectural decision made?] -->|Yes| B{Is it non-trivial and long-lasting?} + A -->|No| Z[No ADR needed] + + B -->|Yes| C[Create an ADR to record the decision] + B -->|No| D[Log the decision in documentation or a changelog only] + + C --> E{Was this decision based on an RFC?} + E -->|Yes| F[Link the ADR to the RFC for full context] + E -->|No| G[Summarise rationale directly in the ADR] +``` + +## Tips + +- Keep it short and focused — one ADR = one decision +- Use consistent naming, like `ADR-0042-use-jwt-for-auth.md` +- Store ADRs in a `/adr` directory in your repo +- Number them chronologically +- Never delete ADRs — if a decision changes, supersede it + +## Requests for comments + +This part explains **what an RFC is**, **when to write one**, what to **include** and how it **relates to ADRs**, with diagrams to help standardise technical proposals. + +### What Is an RFC? + +An RFC (Request for Comments) is a lightweight document used to: + +- Propose technical or architectural changes +- Discuss ideas with the team before implementation +- Drive consensus around design decisions +- Act as historical documentation of thought process and context + +### When to Write an RFC + +Write an RFC when: + +- You are proposing a **non-trivial change** to design, process, or system architecture +- You want input or buy-in from other teams or stakeholders +- You want to explore **multiple options** before making a decision +- The change will have **broad or long-term impact** + +Do **not** write an RFC for: + +- Trivial changes or bug fixes +- Changes already fully decided and documented (in that case, use an ADR) + +### RFC vs ADR + +An RFC drives debate on potential solutions; an ADR records the chosen outcome. See [Architecture decision records](#architecture-decision-records) above. + +| Dimension | RFC (Request for Comments) | ADR (Architecture Decision Record) | +| :--- | :--- | :--- | +| **Primary Purpose** | Propose options, solicit feedback, drive consensus | Record final, agreed architectural choice and rationale | +| **Stage in Lifecycle** | Early / exploratory (pre-decision) | Concluding / binding (post-decision) | +| **Mutability** | Mutable (evolves during review and debate) | Immutable (historical log once accepted) | +| **Scope of Alternatives** | Explores multiple viable options in depth | Summarises chosen option vs rejected alternatives | +| **Typical Flow** | RFC drafted ➔ Reviewed by stakeholders | RFC accepted ➔ ADR recorded into catalogue | + +You can **write both**: +An RFC is accepted → it leads to an ADR that records the final decision. + +### The template + +Use the [RFC template](../templates/rfc.md) in the Templates section. + +### Summary + +> One-paragraph summary of the problem and proposal. + +### Motivation + +> Why this RFC exists. What problem or opportunity it addresses. + +### Proposal + +> The proposed change, including diagrams, APIs, architecture, etc. + +### Alternatives Considered + +> Options evaluated and why they were not chosen. + +### Impact + +> What changes as a result? Compatibility, performance, security? + +### Migration Strategy + +> How to implement this safely and in stages, if applicable. + +### Open Questions + +> Areas needing input or still under debate. + +### Future Considerations + +> Forward-looking notes and how this might evolve. + +--- + +### Changelog + +- YYYY-MM-DD: Initial draft +- YYYY-MM-DD: Updates after review +``` + +## Decision Tree: RFC vs ADR + +```mermaid +graph TD + A[Do you need to make a technical or architectural change?] -->|Yes| B{Is the change expected to +be debated, designed, or reviewed?} + A -->|No| Z[No RFC or ADR needed] + + B -->|Yes| C[Create an RFC to explain context, options, and proposal] + B -->|No| D{Is the change a decision +that's been made already?} + + C --> E{Will this RFC lead to +a long-term architectural decision?} + E -->|Yes| F[Create an ADR once decision is made +and link back to the RFC] + E -->|No| G[No ADR needed; RFC tracks the context only] + + D -->|Yes| H[Create an ADR to record the decision and rationale] + D -->|No| I[Document informally if needed; no formal record required] + + F --> J[Keep RFC and ADR cross-referenced for traceability] +``` + +## Tips + +- Use clear, non-jargon language +- Include diagrams where it helps +- Break down proposals into **logical parts** +- Ask for review and assign an owner diff --git a/content/guides/deep-dives/cbor.md b/content/guides/deep-dives/cbor.md new file mode 100644 index 0000000..9cf9014 --- /dev/null +++ b/content/guides/deep-dives/cbor.md @@ -0,0 +1,189 @@ +--- +title: CBOR +--- + +# CBOR + +## Purpose and scope + +This document explains the Concise Binary Object Representation (CBOR) at a byte level: the type system, the encoding rules, and the worked examples needed to read or write a CBOR payload by hand. It is a companion document to *LwM2M: A Deep Explainer*, which uses CBOR (via SenML CBOR) as one of its payload formats, and to *CoAP: A Deep Explainer*, which carries CBOR as an opaque payload without caring about its structure. It assumes familiarity with binary/hex representations but no prior exposure to CBOR or similar formats. + +## 1. Why a binary format at all + +JSON is a text format: every byte of a JSON document is human-readable ASCII, which is exactly why it is inefficient on a constrained device. The number `1000000` encoded as JSON text is 7 bytes; encoded as CBOR it is 5 bytes, and a boolean `true` that costs 4 ASCII bytes in JSON costs a single byte in CBOR. On an NB-IoT link where every byte transmitted costs airtime and battery (see the repetition mechanism in the NB-IoT explainer), and where parsing has to happen on a microcontroller with a few tens of kilobytes of RAM, both the wire size and the parsing cost matter. + +CBOR (RFC 8949) is designed to solve this without inventing a new data model: it encodes exactly the same tree of maps, arrays, numbers, strings, and booleans that JSON does, just as compact binary rather than text. A CBOR decoder does not need a schema to parse a payload -- the type and length of every item is self-describing from its first byte, which is what makes generic CBOR libraries possible. + +## 2. The major type system + +Every CBOR data item begins with a single **initial byte**, split into two fields: + +```wavedrom +{reg: [ + {bits: 5, name: 'Additional info'}, + {bits: 3, name: 'Major type'} +], config: {bits: 8, lanes: 1}} +``` + +The major type says what kind of item this is; the additional info either encodes a small value directly or says how many following bytes encode a larger value/length. + +| Major type | Value | Meaning | +|---|---|---| +| 0 | Unsigned integer | 0 to 2^64-1 | +| 1 | Negative integer | -1 down to -2^64 | +| 2 | Byte string | Raw bytes | +| 3 | Text string | UTF-8 text | +| 4 | Array | Ordered sequence of items | +| 5 | Map | Key/value pairs | +| 6 | Tag | Semantic tag wrapping the following item | +| 7 | Simple/float | Booleans, null, floating point, "break" | + +### Additional info encoding + +The bottom 5 bits of the initial byte follow one rule, used identically across every major type: + +| Additional info value | Meaning | +|---|---| +| 0-23 | The value itself, encoded directly in these bits | +| 24 | Value follows as 1 more byte | +| 25 | Value follows as 2 more bytes | +| 26 | Value follows as 4 more bytes | +| 27 | Value follows as 8 more bytes | +| 31 | Indefinite length (see section 5) | + +This single rule is why CBOR needs no separate "length-prefix" mechanism per type -- the same 5-bit scheme sizes an integer's value, a string's length, an array's item count, or a map's pair count. + +## 3. Worked examples + +### Small unsigned integer: `10` + +``` +0x0A +``` + +Major type 0 (unsigned int, top 3 bits `000`), additional info 10 (bottom 5 bits `01010`) -- the value is encoded entirely in the initial byte. One byte, done. + +### Larger unsigned integer: `1000000` + +``` +0x1A 000F4240 +``` + +`0x1A` = major type 0, additional info 26 ("value follows as 4 more bytes"). The following 4 bytes, `00 0F 42 40`, are 1,000,000 in big-endian. Five bytes total, versus 7 ASCII bytes for the JSON text `1000000`. + +```wavedrom +{reg: [ + {bits: 8, name: '0x40'}, + {bits: 8, name: '0x42'}, + {bits: 8, name: '0x0F'}, + {bits: 8, name: '0x00'}, + {bits: 5, name: '26'}, + {bits: 3, name: '0'} +], config: {bits: 40, lanes: 1}} +``` + +### Negative integer: `-500` + +``` +0x39 01F3 +``` + +Major type 1 (negative integer, `001`), additional info 25 ("2 more bytes follow"). CBOR encodes negative integers as `-(n+1)`, so to encode -500 it stores `n = 499 = 0x01F3`. This offset-by-one trick means major type 1 can represent -2^64 with the same bit width that major type 0 uses for 2^64-1, rather than wasting a sign bit. + +### Text string: `"gas"` + +``` +0x63 676173 +``` + +`0x63` = major type 3 (text string, `011`), additional info 3 (length 3 bytes). The following bytes `67 61 73` are the UTF-8 encoding of "g", "a", "s". + +### Array: `[1, 2, 3]` + +``` +0x83 01 02 03 +``` + +`0x83` = major type 4 (array, `100`), additional info 3 (3 items follow). Then three individual data items, each self-encoded: `01`, `02`, `03`. + +### Map: `{"id": 1}` + +``` +0xA1 6269 64 01 +``` + +`0xA1` = major type 5 (map, `101`), additional info 1 (1 key/value pair follows). Then the key as a text string -- `62` (text string, length 2), `69 64` ("i", "d") -- followed by the value `01` (unsigned int 1). A map is encoded as a flat sequence of alternating key, value, key, value items; nothing marks where one pair ends and the next begins except walking the items in order. + +### Putting it together: `{"id": 1, "name": "meter"}` + +``` +0xA2 # map, 2 pairs + 62 6964 # key: "id" + 01 # value: 1 + 64 6E616D65 # key: "name" + 65 6D65746572 # value: "meter" +``` + +Reading this byte-by-byte: `A2` (map, 2 pairs); `62 69 64` (text string length 2 = "id"); `01` (unsigned int 1); `64 6E 61 6D 65` (text string length 4 = "name"); `65 6D 65 74 65 72` (text string length 5 = "metre"). A generic CBOR decoder walks this without ever being told in advance that a map, then a string, then an int is coming -- every item announces its own type and length. + +## 4. Simple values, booleans, and floats (major type 7) + +Major type 7 is a catch-all for values that are not numbers, strings, arrays, or maps: + +| Encoding | Meaning | +|---|---| +| `0xF4` | `false` | +| `0xF5` | `true` | +| `0xF6` | `null` | +| `0xF7` | `undefined` | +| `0xF9` xxxx | half-precision (16-bit) float | +| `0xFA` xxxxxxxx | single-precision (32-bit) float | +| `0xFB` xxxxxxxxxxxxxxxx | double-precision (64-bit) float | +| `0xFF` | "break" -- closes an indefinite-length item (see section 5) | + +A sensor reading such as `21.4` would typically be encoded as a 32-bit float (`0xFA` + 4 bytes) unless the application has already agreed on a fixed-point integer representation to save the extra bytes. + +## 5. Indefinite-length items + +Sometimes the encoder does not know an item's final length in advance -- streaming a byte string as it arrives, for instance. Additional info value 31 marks an item as indefinite-length: instead of a length, the item is terminated by a `0xFF` "break" byte once all its chunks (for strings) or elements (for arrays/maps) have been written. + +``` +0x9F 01 02 03 FF +``` + +`0x9F` = array, additional info 31 (indefinite length) -- items follow until `FF` (break) is seen. This is functionally equivalent to `0x83 01 02 03` but useful when the encoder is generating items faster than it can count them, which matters less for small fixed-schema payloads (most LwM2M resources) than for genuinely streamed data. + +> **Warning:** Indefinite-length encoding is explicitly disallowed under **Canonical CBOR** (deterministic encoding, RFC 8949 §4.2), which many constrained-device implementations require so that two encoders always produce byte-identical output for the same logical data -- useful for signing or hashing a payload. If a payload needs to be signed or compared by hash, confirm the encoder is using definite lengths and the canonical map-key ordering below. + +## 6. Canonical CBOR and map key ordering + +Because a map is just a flat sequence of key/value pairs with no inherent ordering requirement, two encoders can produce different (but semantically identical) byte sequences for the same map depending on key order. Canonical CBOR fixes this by requiring map keys to be sorted (by the bytewise encoding of the key, not by application-level meaning) and by requiring the shortest possible encoding of every length and integer (never using the 2-byte form where the 1-byte form would do). + +This matters wherever a payload's exact bytes need to be reproducible -- signature verification being the obvious case -- and is easy to get wrong when hand-rolling an encoder rather than using a conformant library. + +## 7. Tags (major type 6) + +A tag wraps the following data item with extra semantic meaning that a generic decoder can choose to interpret or ignore. For example, tag 1 (`0xC1`) means "the following integer or float is a Unix timestamp": + +``` +0xC1 1A 653C 8A00 +``` + +`0xC1` = tag 1, followed by `1A 653C8A00` -- an unsigned integer (major type 0, 4-byte form) representing seconds since the epoch. A decoder that understands tag 1 renders this as a date; one that does not can still safely read the underlying integer. + +## 8. CBOR compared to alternatives + +| | CBOR | JSON | Protocol Buffers | MessagePack | +|---|---|---|---|---| +| Format | Binary | Text | Binary | Binary | +| Schema required to decode | No | No | Yes (.proto file) | No | +| Self-describing | Yes | Yes | No | Yes | +| Typical size vs JSON | Smaller | Baseline | Smallest (with schema) | Similar to CBOR | +| Extensibility | Tags | N/A | Field numbers + versioning | Limited | +| IETF standard | Yes (RFC 8949) | Yes (RFC 8259) | No (Google-defined) | No (community spec) | + +CBOR's advantage over Protocol Buffers for a device-management protocol is that no schema needs to be distributed, compiled, or kept in version-lockstep between client and server -- a genuine benefit when a server may need to talk to devices running several different firmware versions with slightly different object definitions. The cost is that CBOR payloads carry more self-description overhead than a schema-compiled Protobuf message would for the same data. + +## 9. Why this matters for LwM2M + +LwM2M does not use raw CBOR directly; it uses **SenML CBOR**, which is CBOR encoding a specific SenML (Sensor Measurement Lists, RFC 8428) record structure -- an array of maps, each map using short integer keys (rather than the human-readable string keys shown in the examples above) to represent a resource path, a value, and a timestamp, all wrapped in the byte-level rules described in this document. Everything above still applies unchanged: SenML CBOR is CBOR, just with an agreed application-level schema for what the maps contain. The LwM2M deep dive covers that schema; this document covers the byte-level mechanics underneath it. diff --git a/content/guides/deep-dives/certificates-and-keys.md b/content/guides/deep-dives/certificates-and-keys.md new file mode 100644 index 0000000..4e14b03 --- /dev/null +++ b/content/guides/deep-dives/certificates-and-keys.md @@ -0,0 +1,320 @@ +--- +title: Certificates and keys +--- + +# Certificates and keys + +## Why this document exists + +PKI, X.509, certificate chains, PSK, GMAC -- these terms are usually taught as a pile of unrelated acronyms. They are not unrelated. Each mechanism exists because the previous one had a specific, nameable weakness. This document works through that chain of reasoning in logical order: from the first problem (two parties want to communicate secretly) through to the last (how a device or browser decides, algorithmically, whether to trust a certificate it has never seen before). + +Read top to bottom. Each section assumes the previous one. + +## Key vs certificate: the distinction that matters most + +These two terms get used almost interchangeably in casual conversation, but they are different classes of object. + +- **A key** is a piece of cryptographic material -- a number, or a pair of mathematically related numbers -- used directly by an algorithm to encrypt, decrypt, sign, or verify. A key on its own carries no identity information. It is just math. +- **A certificate** is a data structure that *makes a claim about* a public key: specifically, that this public key belongs to this named entity. A certificate contains a public key as one of its fields, plus identity information, validity dates, and a signature from an issuer vouching for the binding between the two. A certificate without the corresponding private key held elsewhere is inert -- it proves nothing by itself; it is only useful when paired with an entity that can prove possession of the matching private key. + +In short: a key is the cryptographic object; a certificate is a signed statement that binds a specific public key to a specific identity. Part 1 and Part 2 below are about keys and what they do mathematically. Part 4 introduces certificates, once the reason for needing them (proving whose key it is) has been established. + +## Part 1: The core problem -- confidentiality needs a shared secret + +Two parties, call them a client and a server, want to exchange data that a third party cannot read. The mathematically simple way to do this is symmetric encryption: both sides hold the *same* secret key, and use it with an algorithm such as AES to turn plaintext into ciphertext and back again. + +```mermaid +flowchart LR + A[Plaintext] -->|AES encrypt with key K| B[Ciphertext] + B -->|AES encrypt with key K| A +``` + +This is fast and well understood. AES-128 or AES-256 in an authenticated mode is not the weak point of modern systems. The weak point is the phrase "both sides hold the same secret key." How did the key get onto both devices in the first place, without an eavesdropper also obtaining it? + +### Pre-shared keys (PSK): the honest, limited answer + +A pre-shared key is exactly what it sounds like: the key is provisioned onto both endpoints out-of-band, before the encrypted session starts -- flashed into firmware at manufacture, entered manually, or distributed over some separately-secured channel. There is no cryptographic cleverness here. It works precisely as well as the out-of-band distribution process works. + +> **Note:** PSK is not a lesser cousin of PKI. It is a correct and often superior choice when the number of endpoints is small and known in advance -- a factory-provisioned fleet of embedded devices talking to one back-end, for instance. The problems start when the endpoint population grows, when keys need to be provisioned to devices you have never physically touched, or when a single compromised key needs to be revoked without also having to physically revisit every other device. + +The failure modes of PSK are entirely operational, not mathematical: + +- **Distribution at scale.** Getting a unique secret onto ten thousand devices without it transiting an insecure channel is a logistics problem, not a crypto problem. +- **Revocation.** If one device is compromised and its key extracted, every other device sharing that key (if keys are shared across a batch, which they often are for practicality) is now also compromised. +- **No third-party introduction.** PSK only works between parties who have *already* met, out-of-band. It cannot help a browser trust a web server it has never seen before, run by an organisation it has never dealt with. + +That last limitation is what forces the next step. The internet needed a way for parties who have never met to establish a shared secret over a channel that is, by default, insecure and observed. That requirement is what asymmetric cryptography was built to solve. + +## Part 2: Asymmetric cryptography -- splitting the key in two + +The mathematical trick of asymmetric (public-key) cryptography is a key pair with an asymmetric relationship: a public key and a private key, mathematically linked, where: + +- data encrypted with the public key can only be decrypted with the matching private key, and +- data signed with the private key can be verified by anyone holding the public key, without them being able to forge a new signature. + +The public key can be handed to a stranger, printed on a business card, or published on a website. It is not a secret and does not need out-of-band protection. Only the private key needs protection, and it never has to leave the device that generated it. + +```mermaid +flowchart TB + subgraph Alice + APriv[Alice's private key -- secret] + APub[Alice's public key -- public] + end + subgraph Bob + BPriv[Bob's private key -- secret] + BPub[Bob's public key -- public] + end + APub -->|handed to anyone, including Bob| Bob + BPub -->|handed to anyone, including Alice| Alice +``` + +This solves the "never met before" problem for exchanging a secret. It does not, by itself, solve the "am I actually talking to Bob, or to someone impersonating Bob" problem. That distinction matters and is picked up again in Part 4. For now, treat the mechanics as sound and look at what algorithms actually implement them. + +### RSA + +RSA's security rests on the difficulty of factoring the product of two large primes. Key generation produces a public/private pair from two large primes; encryption and signing are modular exponentiation operations using those keys. RSA is old, extremely well analysed, and slow relative to the alternatives below -- both in key generation and in per-operation cost, and it needs large keys (2048-bit minimum today, 3072 or 4096 for longer-term assurance) to remain secure, which makes it a poor fit for constrained devices. + +### Elliptic Curve Cryptography (ECC) and ECDSA + +ECC achieves equivalent security to RSA with much smaller keys, because its hard problem (the elliptic curve discrete logarithm problem) scales more favourably than integer factorisation. A 256-bit ECC key gives roughly the same security margin as a 3072-bit RSA key. This matters enormously for embedded and IoT contexts: smaller keys mean smaller certificates, less flash storage, less time spent in expensive modular arithmetic on a constrained MCU, and less data transiting a narrowband link. + +ECDSA is the signature scheme built on ECC. It is what most modern TLS certificates actually use for the signing operation, in preference to RSA, for exactly the size and speed reasons above. + +> **Note:** ECDSA has a well-known operational hazard: signature generation requires a fresh, high-quality random nonce for every signature. Reuse the nonce (or use a poor RNG) across two signatures and the private key can be recovered algebraically from the two signatures alone. This is not a theoretical concern -- it is the mechanism behind several real-world private key recoveries. Deterministic nonce generation (RFC 6979) removes the dependency on the RNG at signing time and is the safer default where the library supports it. + +### EdDSA (Ed25519 / Ed448) + +EdDSA is a newer signature scheme, also elliptic-curve-based, designed specifically to remove the nonce-reuse hazard above: the nonce is derived deterministically from the private key and the message, so there is no RNG call at signing time to get wrong. It is also faster than ECDSA for both signing and verification on the same curve size. Adoption in X.509 and TLS is growing but is behind ECDSA, mostly for compatibility reasons with older infrastructure. + +### The asymmetric operations are still too slow for bulk data + +Every one of the algorithms above is orders of magnitude slower than AES for encrypting bulk data. Nobody encrypts an entire video stream with RSA or ECC directly. Instead, asymmetric crypto is used for exactly one job: to let two strangers agree, over an insecure channel, on a *symmetric* session key -- which then does the actual bulk encryption using AES. This is the point of a TLS handshake, and it is why Part 1 and Part 2 are not competing approaches but two stages of the same pipeline. + +```mermaid +sequenceDiagram + participant C as Client + participant S as Server + C->>S: Hello, propose cipher suites + S->>C: Certificate (contains server's public key) + Note over C: Verify certificate (Part 4) + C->>S: Encrypted pre-master secret (using server's public key) + Note over C,S: Both derive the same symmetric session key + C->>S: Application data, encrypted with AES using session key + S->>C: Application data, encrypted with AES using session key +``` + +## Part 3: Integrity -- hashing, MACs, and GMAC + +Confidentiality (Part 1 and 2) answers "can a third party read this." It does not answer "has this data been altered in transit, and did it really come from who it claims to have come from." That is the job of hashing and message authentication. + +### Hash functions + +A cryptographic hash function (SHA-256, SHA-384) takes an arbitrary-length input and produces a fixed-length digest, with three properties that matter here: it is effectively impossible to find two different inputs producing the same digest (collision resistance), impossible to reconstruct the input from the digest (pre-image resistance), and any change to the input, however small, produces an unpredictably different digest (the avalanche effect). A hash alone proves nothing about *who* produced the data -- anyone can hash anything. It only proves that data matches a digest computed earlier. + +### MACs: adding a shared secret to the hash + +A Message Authentication Code takes a hash-like construction and mixes in a secret key, so that only someone holding the key could have produced that specific tag for that specific message. HMAC is the standard construction: it runs the message through a hash function twice, with the key mixed in both times, in a specific nested pattern designed to resist length-extension attacks that would otherwise apply to naive key-then-hash constructions. + +### GMAC: authentication as a side effect of the cipher itself + +GMAC is the authentication-only variant of GCM (Galois/Counter Mode), an AES mode of operation that was designed to give both encryption and authentication from a single pass over the data, rather than needing a separate encrypt step and a separate HMAC step. GCM encrypts using AES in counter mode (turning the block cipher into a stream cipher) and simultaneously computes an authentication tag using multiplication in a Galois field (GF(2^128)) over the ciphertext blocks. GMAC is what you get if you take that authentication tag computation on its own, without the encryption -- useful when you need to authenticate data that does not itself need to be kept confidential, such as packet headers alongside an encrypted payload. + +> **Warning:** GCM/GMAC security depends critically on the nonce (IV) never repeating for a given key. Reuse a nonce with the same key and an attacker can recover the authentication key and forge subsequent messages -- this is a catastrophic, well-documented failure mode, not a minor one. Any system using AES-GCM needs a nonce management strategy (counter-based, ensuring uniqueness across restarts) that is at least as carefully engineered as the key management itself. + +The practical reason GCM/GMAC has displaced separate encrypt-then-HMAC constructions in most new protocol design is performance: one pass over the data instead of two, and the Galois field multiplication is amenable to hardware acceleration, which matters on constrained silicon as much as on servers. + +At this point the toolbox is complete: symmetric ciphers for bulk confidentiality, asymmetric ciphers to bootstrap a shared symmetric key between strangers, and hash/MAC constructions for integrity and authenticity. What remains unsolved is the identity problem flagged at the end of Part 2: when a server hands over its public key, how does the client know that key genuinely belongs to the organisation it claims to belong to, rather than to an attacker sitting in the middle of the connection? + +## Part 4: The identity problem -- why a public key alone is not enough + +Handed a bare public key, a client has no way to distinguish "this is really example.com's key" from "this is an attacker's key, and the attacker is intercepting the connection and presenting their own key instead." Asymmetric crypto guarantees that whoever holds the matching private key can decrypt or sign -- it says nothing about who that someone is. + +The fix is to have a trusted third party vouch for the binding between "this public key" and "this identity," in a way the client can verify without having to trust the server making the claim about itself. That vouching mechanism is a digital certificate, and the infrastructure that issues, distributes, and revokes them is Public Key Infrastructure. + +### X.509: the certificate as a structured, signed statement + +An X.509 certificate is, at its core, a data structure containing: + +- the subject's identity (a distinguished name, and for modern web certificates, a list of the domain names it covers via the Subject Alternative Name extension) +- the subject's public key +- the issuer's identity (who is vouching for this binding) +- a validity period (not-before / not-after) +- a serial number, unique per issuing CA +- extensions -- key usage constraints, whether the subject is itself permitted to act as a CA (the Basic Constraints extension), certificate revocation and OCSP endpoint locations, and so on +- a digital signature over all of the above, produced using the *issuer's* private key + +That last point is the entire mechanism. The certificate is not self-asserted. It is a claim signed by someone else's private key, which means verifying it requires the issuer's public key -- which itself typically arrives in another certificate, signed by yet another issuer. This is where the chain comes from. + +```mermaid +flowchart TB + Root["Root CA certificate\n(self-signed)"] -->|signs| Inter["Intermediate CA certificate"] + Inter -->|signs| Leaf["Leaf/server certificate\n(e.g. example.com)"] +``` + +### The chain of trust, and where it has to stop + +Each certificate is verified using the public key from the certificate above it in the chain. This cannot recurse forever -- something has to be trusted axiomatically, not proven. That something is a small set of root CA certificates, self-signed, pre-installed in the operating system, browser, or embedded device's trust store by the platform vendor. Trusting a root CA is a decision made once, out of band, by whoever curates that trust store -- it is the same "trust established out of band" idea as a pre-shared key in Part 1, just applied at a much larger scale and pushed as far up the chain as possible so it only has to happen for a handful of root certificates rather than for every server on the internet. + +Intermediate CAs exist mainly for operational and risk-management reasons: the root CA's private key is kept offline in as close to permanent isolation as is achievable, because if it is ever compromised every certificate the CA has ever issued is suspect. Intermediates are the working keys, used day-to-day to sign leaf certificates, and can be revoked and rotated without touching the root. + +### Verifying a chain, algorithmically + +Given a leaf certificate and the intermediates the server sends alongside it, a client: + +1. checks the leaf certificate's signature against the intermediate's public key +2. checks the intermediate's signature against the next certificate up the chain +3. repeats until it reaches a certificate that matches one already present in its local trust store +4. checks that every certificate in the chain is within its validity period +5. checks that no certificate in the chain has been revoked (via a Certificate Revocation List, or an OCSP query to the issuer) +6. checks that the leaf certificate's subject (specifically its Subject Alternative Name entries) matches the hostname the client actually asked for + +```mermaid +sequenceDiagram + participant C as Client + participant Store as Local trust store + C->>C: Receive leaf + intermediate certs from server + C->>C: Verify leaf signature using intermediate's public key + C->>C: Verify intermediate signature using root's public key + C->>Store: Is this root already trusted? + Store-->>C: Yes + C->>C: Check validity dates on every cert in chain + C->>C: Check revocation status (CRL/OCSP) + C->>C: Check hostname matches SAN entries on leaf + C->>C: Chain accepted -- proceed with handshake +``` + +Failure at any single step is meant to fail the whole connection, not degrade gracefully. This is the correct behaviour and the reason certificate errors in browsers are hard blocks rather than warnings for most classes of failure -- a broken chain is indistinguishable, from the client's point of view, from an active machine-in-the-middle attack. + +> **Important:** Revocation checking is the weakest link in this whole sequence in practice. CRLs can be large and stale by the time they are fetched; OCSP requires an extra round trip to the issuer (and issuers historically have had patchy uptime, which is why "OCSP soft-fail" -- proceeding anyway if the check times out -- became common, quietly undermining the guarantee). OCSP stapling, where the server itself periodically fetches and presents a signed, time-stamped OCSP response alongside its certificate, exists specifically to remove the client's dependency on reaching the CA's OCSP responder at connection time. + +## Part 5: Encoding -- PEM, DER, and what the file extensions actually mean + +Everything above describes what a key or certificate *is* logically. None of it describes the bytes on disk, and this is where a lot of avoidable confusion happens in practice. + +- **DER (Distinguished Encoding Rules)** is the binary encoding. It is the canonical, unambiguous byte representation of the ASN.1 structure that a key or certificate is defined as. Compact, not human-readable. +- **PEM (Privacy-Enhanced Mail)** is not a different structure -- it is DER, base64-encoded, wrapped in a plain-text header and footer such as `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----`. PEM exists so the data can be pasted into a text field, emailed, or stored in a text-based config file without corruption. Decode the base64 body of a PEM file and the DER bytes underneath are identical to a DER file of the same certificate. + +> **Note:** File extensions are a naming convention, not a format guarantee. A `.crt` or `.cer` file might contain PEM text or raw DER binary -- the only reliable way to know is to open it and check for the `-----BEGIN` header. This inconsistency is a frequent source of "invalid certificate" errors when a tool expecting one encoding is handed the other. + +| Extension | Typically contains | Encoding | +| --- | --- | --- | +| `.pem` | Certificate, key, or chain | PEM (text) | +| `.crt` / `.cer` | Certificate | PEM or DER -- check the file | +| `.der` | Certificate or key | DER (binary) | +| `.key` | Private key | PEM or DER -- check the file | +| `.csr` | Certificate Signing Request -- the unsigned request sent to a CA, containing the public key and requested identity | PEM (usually) | +| `.p12` / `.pfx` | PKCS#12 bundle -- certificate, private key, and chain together in one encrypted, password-protected binary container | Binary (PKCS#12) | +| `.jks` | Java KeyStore -- Java-specific bundle format, conceptually similar to PKCS#12 | Binary (proprietary) | + +`.p12`/`.pfx` is worth calling out separately: it is the one common format that bundles the private key alongside the certificate in a single file, which makes it convenient for import/export between systems but also means the file itself is as sensitive as the private key it contains. + +## Part 6: Where the private key actually lives -- HSM, TPM, and secure elements + +Every mechanism described so far assumes the private key never leaks. That assumption is doing a lot of work, and it is worth being explicit about what actually enforces it. + +The naive option is a private key stored as a file on the device's filesystem or flash, protected only by OS file permissions or, on an embedded device, by nothing in particular beyond physical access being difficult. If an attacker gets code execution or physical flash access, the key is gone, and everything built on top of it (Part 2 onward) is undermined -- the maths was never the weak point. + +The alternative is to generate and hold the private key inside dedicated hardware that is designed so the key can never be read out, only used: + +- **HSM (Hardware Security Module).** A dedicated, often rack-mounted or PCIe device used primarily server-side and by CAs. Root and intermediate CA private keys are the canonical use case -- generated inside the HSM, used to sign via an API call, and never exposed outside its tamper-resistant boundary. +- **TPM (Trusted Platform Module).** A standardised chip (or firmware equivalent) found on most PCs and increasingly on embedded boards, providing secure key storage plus attestation -- proof to a remote party of what firmware/software state the device booted into. +- **Secure element (SE).** A small, low-cost dedicated chip aimed squarely at embedded and IoT applications (Microchip ATECC608, NXP SE050, and similar parts) -- generates and stores keys, performs ECDSA/ECDH operations on-chip, and is designed to resist physical extraction attempts. This is the relevant option for metering and similar constrained-device deployments, where an HSM is impractical but a bare flash-stored key is not an acceptable risk. + +> **Important:** The common thread across all three is that the private key is generated inside the boundary and never leaves it in plaintext form. Signing and decryption operations are requested via an API and performed internally; the result comes out, the key does not. This is what "hardware-backed" key protection actually means, and it is the difference that matters when assessing whether a device's key material could survive firmware extraction or a supply-chain compromise. + +## Part 7: Updating keys and certificates on a deployed device + +Everything above establishes trust once. Real deployments need a way to change that trust over time -- because a certificate expires, an algorithm is deprecated, a key is suspected compromised, or policy simply mandates periodic rotation. There are two distinct things that can be updated, and it matters which one is meant: + +- **Certificate renewal**: the same key pair continues to be used; only the certificate (identity claim, validity period, signature) is replaced. Cheaper and lower-risk, since the private key never moves or changes. +- **Key rotation**: a new key pair is generated, and a new certificate is issued for the new public key. Necessary if the old key is suspected compromised, or if migrating to a stronger algorithm (e.g. RSA to ECC). + +### Standard protocols for in-field renewal + +Rather than inventing a bespoke mechanism, two standard protocols exist specifically so a device can request renewal itself, over the network, using its *existing* (still-valid) certificate as proof of who it is: + +- **EST (Enrollment over Secure Transport, RFC 7030)** -- a simple HTTPS-based protocol for certificate enrollment and re-enrollment, widely supported and relatively lightweight to implement on constrained devices. +- **CMPv2 (Certificate Management Protocol)** -- an older, more feature-complete IETF protocol covering initial enrollment, renewal, revocation, and key update, common in telecoms and industrial deployments. + +Both follow the same basic shape: the device authenticates to the CA/registration authority using its current certificate, submits a certificate signing request (containing either the same public key for renewal, or a newly generated one for rotation), and receives a newly signed certificate in response. + +```mermaid +sequenceDiagram + participant D as Device + participant CA as CA / Registration Authority + D->>D: Generate new key pair (if rotating) and CSR + D->>CA: Authenticate using current, still-valid certificate + D->>CA: Submit CSR (EST or CMPv2) + CA->>D: Issue new certificate + D->>D: Install new cert (and key, if rotated) alongside the old one + D->>D: Verify new cert/key works end-to-end + D->>D: Switch active credential to the new one + D->>CA: Old certificate revoked (only after cutover confirmed) +``` + +> **Warning:** The critical safety property is "make before break": the new credential is installed and verified working *before* the old one is disabled or revoked. A device that deletes its only working certificate before confirming the replacement is reachable and functional has bricked its own trust relationship, and for a remotely deployed device with no physical access, that can mean a truck roll to recover it. + +### Manufacturer identity vs operational identity + +Many device PKI designs deliberately separate two certificates with different lifecycles: + +- an **IDevID** (Initial/Manufacturer Device Identity) -- installed at the factory, signed by the manufacturer's CA, used only to bootstrap trust and prove the device's provenance +- an **LDevID** (Local/Operational Device Identity) -- issued once the device is deployed and enrolled with the operator's own PKI, used for day-to-day operational authentication, and rotated independently of the manufacturer certificate + +This separation means the operator can rotate operational certificates on whatever schedule it needs, without ever having to touch or re-issue the manufacturer-installed identity that proves the device is genuine. + +### Updating a pre-shared key + +For PSK-based deployments (Part 1), "updating the key" usually means one of two things: a full re-provisioning event (physically re-flashing or re-entering a new PSK, which has all the same distribution problems as the original provisioning), or deriving a new session key from the existing PSK using a key derivation function (KDF) combined with a fresh nonce or counter -- rotating the working key without ever having to redistribute the underlying secret. The latter is far more common in practice, precisely because out-of-band redistribution at scale is the expensive part. + +## Part 8: Putting it all back together + +The sequence, end to end: + +1. Bulk data needs confidentiality → symmetric encryption (AES) is fast enough but needs a shared secret. +2. Getting that shared secret onto two strangers' devices without an eavesdropper seeing it is hard → PSK solves it out-of-band for small, known populations; asymmetric cryptography (RSA / ECC / EdDSA) solves it over the insecure channel itself, for arbitrary strangers. +3. A public key on its own proves nothing about *whose* key it is → X.509 certificates bind a public key to an identity via a signature from an issuer. +4. Trusting the issuer's signature just pushes the same problem up a level → certificate chains resolve this by delegating trust upward to a small, pre-trusted set of root CAs, verified once out of band and baked into the trust store. +5. Meanwhile, every message exchanged once the session key is established still needs tamper detection → HMAC or, more commonly today, AES-GCM/GMAC, giving integrity and authenticity either alongside or as part of the same operation as the encryption. +6. None of this matters if the private key can simply be read off the device → HSMs, TPMs, and secure elements exist so the key is used but never extracted. +7. Trust established once still needs to change over time → certificate renewal, key rotation, and standard protocols like EST/CMPv2 let a device update its own credentials in the field, without ever breaking the working connection before the replacement is confirmed. + +Every mechanism in this document exists because the mechanism before it left one specific gap open. That is the thread to hold onto -- the acronyms are not a list to memorise, they are a sequence of patches, each one closing the hole the last one left. + +## Glossary + +| Term | Definition | +| --- | --- | +| Key | A piece of cryptographic material used to encrypt, decrypt, sign, or verify. Carries no identity information by itself. | +| Certificate | A signed data structure binding a public key to a claimed identity, plus validity dates and issuer information. | +| PSK | Pre-Shared Key -- a symmetric key provisioned onto both endpoints out-of-band, before any session starts. | +| PKI | Public Key Infrastructure -- the systems, roles, and processes for issuing, distributing, and revoking certificates. | +| X.509 | The standard defining the structure of a digital certificate. | +| CA | Certificate Authority -- an entity that signs certificates, vouching for the identity-to-key binding. | +| Root CA | A self-signed CA certificate trusted axiomatically, pre-installed in a trust store. | +| Intermediate CA | A CA certificate signed by a root (or another intermediate), used for day-to-day signing so the root key can stay offline. | +| Certificate chain | The ordered sequence of certificates from a leaf certificate up to a trusted root. | +| CSR | Certificate Signing Request -- an unsigned request containing a public key and identity details, submitted to a CA. | +| CRL | Certificate Revocation List -- a signed list of revoked certificate serial numbers, published by a CA. | +| OCSP | Online Certificate Status Protocol -- a real-time query to check whether a specific certificate has been revoked. | +| OCSP stapling | The server itself fetches and presents a signed OCSP response alongside its certificate, avoiding a client-side round trip. | +| SAN | Subject Alternative Name -- the certificate extension listing the hostnames a certificate is valid for. | +| RSA | An asymmetric algorithm based on the difficulty of factoring large primes. | +| ECC | Elliptic Curve Cryptography -- asymmetric cryptography using elliptic curve mathematics, giving smaller keys for equivalent security. | +| ECDSA | The digital signature algorithm built on ECC; requires a fresh random nonce per signature. | +| EdDSA | A newer elliptic-curve signature scheme (Ed25519/Ed448) using deterministic nonce generation. | +| Hash | A one-way function producing a fixed-length digest of arbitrary input data, used for integrity checks. | +| HMAC | A keyed hash construction proving both integrity and authenticity using a shared secret. | +| GCM | Galois/Counter Mode -- an AES mode providing encryption and authentication together in a single pass. | +| GMAC | The authentication-only variant of GCM, used when data needs authenticating but not encrypting. | +| Nonce / IV | A value used once per key in a cipher mode; reuse under the same key breaks the security guarantee (critical for ECDSA, GCM, and GMAC). | +| HSM | Hardware Security Module -- dedicated hardware that generates and holds private keys, performing operations without ever exposing the key. | +| TPM | Trusted Platform Module -- a standardised chip providing secure key storage and attestation, common on PCs and some embedded boards. | +| Secure element | A small dedicated chip for embedded/IoT devices, generating and storing keys with on-chip crypto operations. | +| PEM | A text encoding: base64 with header/footer lines, wrapping an underlying DER structure. | +| DER | The binary encoding of a certificate or key's underlying ASN.1 structure. | +| PKCS#12 | A binary, password-protected container bundling a certificate, private key, and chain together (`.p12`/`.pfx`). | +| IDevID | Initial/Manufacturer Device Identity -- a factory-installed certificate proving device provenance. | +| LDevID | Local/Operational Device Identity -- an operator-issued certificate used for day-to-day authentication, rotated independently of the IDevID. | +| EST | Enrollment over Secure Transport (RFC 7030) -- an HTTPS-based protocol for certificate enrollment and renewal. | +| CMPv2 | Certificate Management Protocol -- an IETF protocol covering enrollment, renewal, revocation, and key update. | diff --git a/content/guides/deep-dives/coap.md b/content/guides/deep-dives/coap.md new file mode 100644 index 0000000..56439b3 --- /dev/null +++ b/content/guides/deep-dives/coap.md @@ -0,0 +1,237 @@ +--- +title: CoAP +--- + +# CoAP + +## Purpose and scope + +This document explains the Constrained Application Protocol (CoAP) at a mechanism level: the message format, the reliability model, and the extensions (Observe, Block-wise) that let it stand in for a full application protocol on constrained devices. It is a companion document to *LwM2M: A Deep Explainer*, which builds its transaction model directly on top of what is described here, and to *CBOR: A Deep Explainer*, which covers the payload encoding CoAP carries. It assumes familiarity with general networking concepts (IP, UDP, TCP) but not prior exposure to constrained-device protocols. + +## 1. Why CoAP exists + +HTTP is not a good fit for constrained devices. A typical HTTP/1.1 request/response pair, even without a body, runs to hundreds of bytes of ASCII headers, and HTTP assumes a reliable, ordered, connection-oriented transport (TCP) with the overhead that implies -- a three-way handshake, congestion control state, and a socket held open per connection. + +CoAP (RFC 7252) keeps the parts of HTTP that made it a good application model -- methods, response codes, URIs, content negotiation -- and re-implements them as a compact binary protocol over UDP. The design goals are: + +- A message header of 4 bytes, not hundreds +- No mandatory connection state +- Optional reliability built into the protocol itself, rather than borrowed from the transport +- A one-to-one mapping to HTTP semantics, so it can be proxied to and from HTTP at a gateway +- Native support for asynchronous notification (Observe) and multicast, neither of which HTTP does well + +> **Note:** CoAP is often described as "REST over UDP." That is a reasonable one-line summary, but the reliability, deduplication, and congestion control CoAP builds on top of UDP are real protocol machinery, not something you get for free -- this is most of what this document covers. + +## 2. Where CoAP sits in the stack + +```mermaid +flowchart TB + A[Application: LwM2M / other] --> B[CoAP: request/response, Observe, Block-wise] + B --> C[DTLS: optional security layer] + C --> D[UDP] + D --> E[IP] +``` + +CoAP normally runs on UDP port 5683 (unencrypted) or 5684 (over DTLS, "coaps://"). Because it sits directly on UDP, there is no connection setup latency before the first request can be sent -- a meaningful saving on high-latency, high-cost radio links like NB-IoT, where every round trip costs airtime and battery. + +## 3. The two-layer model + +CoAP separates concerns into two layers that are easy to conflate but serve different purposes: + +- **Message layer** -- handles deduplication and optional reliability over UDP, using Message IDs +- **Request/response layer** -- handles the REST-style semantics (method, URI, response code), using Tokens + +A single CoAP exchange uses both: the message layer gets the bytes there reliably (if Confirmable), and the request/response layer matches the reply to the right in-flight request. + +### Message types + +Every CoAP message carries one of four types: + +| Type | Meaning | +|---|---| +| CON (Confirmable) | "Please acknowledge this." Retransmitted until ACKed or it times out. | +| NON (Non-confirmable) | Fire-and-forget. No retransmission, no ACK expected. | +| ACK (Acknowledgement) | Confirms receipt of a CON message. May carry the actual response as a "piggybacked" reply. | +| RST (Reset) | "I received this but cannot process it" -- e.g. the recipient has no idea what this Message ID refers to. | + +The most common pattern is a CON request answered by a piggybacked ACK: the ACK both confirms delivery and carries the response payload in one message, saving a round trip. + +```mermaid +sequenceDiagram + participant Client + participant Server + Client->>Server: CON [MID=0x1234] GET /status (Token=0xA1) + Server-->>Client: ACK [MID=0x1234] 2.05 Content (Token=0xA1) + Note over Client,Server: One round trip -- ACK piggybacks the response +``` + +When the server cannot answer immediately (it needs to do work first), it may separate the ACK from the response: an empty ACK confirms the message arrived, and a later CON carrying the real response arrives independently, itself requiring its own ACK. + +### Message IDs and deduplication + +Every message carries a 16-bit Message ID (MID), chosen by the sender. A CON message and its ACK/RST share the same MID -- this is purely a message-layer concept, scoped to one hop and one direction of retransmission. If the same CON message arrives twice (because an ACK was lost and the sender retransmitted), the receiver recognises the repeated MID and re-sends its cached ACK without reprocessing the request. + +> **Warning:** MID deduplication is per-endpoint and time-windowed (EXCHANGE_LIFETIME, nominally around 247 seconds by the RFC 7252 defaults). Do not assume MIDs are unique forever, and do not use them as an application-layer identifier -- that is what Tokens are for. + +### Tokens and request/response matching + +The Token (0-8 bytes, chosen by the client) identifies a request/response pair independently of the message layer. This matters because a single logical request/response exchange can span multiple underlying messages (a separate response, or a Block-wise transfer spanning many blocks) -- the Token stays constant across all of them, while the MID changes on every individual message. + +```mermaid +flowchart LR + A["MID: identifies one UDP datagram, one hop"] -.independent axis.- B["Token: identifies one logical request/response, may span many datagrams"] +``` + +### Retransmission and congestion control + +For CON messages, the sender retransmits on a randomised exponential backoff, starting from ACK_TIMEOUT (default 2 seconds, randomised up to 3 seconds) and doubling on each retry, up to MAX_RETRANSMIT (default 4) attempts. This is deliberately similar in shape to TCP's backoff, so that many CoAP endpoints on a shared, congested link (a cellular NB-IoT cell, for instance) do not collectively hammer it. + +## 4. Message format + +A CoAP message is a compact binary structure, not a text header block. The overall layout is a fixed 4-byte header followed by variable-length fields: + +```mermaid +flowchart TB + A["4-byte fixed header: Version, Type, Token Length, Code, Message ID"] --> B["Token: 0-8 bytes"] + B --> C["Options: variable, delta-encoded, sorted by option number"] + C --> D["0xFF payload marker"] + D --> E["Payload: raw bytes"] +``` + +The fixed header itself packs five fields into 32 bits: + +```wavedrom +{reg: [ + {bits: 2, name: 'Ver'}, + {bits: 2, name: 'T'}, + {bits: 4, name: 'TKL'}, + {bits: 8, name: 'Code'}, + {bits: 16, name: 'Message ID'} +], config: {bits: 32, lanes: 1}} +``` + +That is a 2-bit version, 2-bit type (CON/NON/ACK/RST), 4-bit Token Length, an 8-bit Code, and a 16-bit Message ID -- exactly 4 bytes, with no padding. Everything after that is variable-length: the Token (as many bytes as Token Length said), then zero or more Options, then -- if a payload is present -- a single `0xFF` marker byte followed by the raw payload bytes. + +### Codes + +The Code field reuses HTTP's class/detail split but packs it into a single byte as `c.dd` (3-bit class, 5-bit detail), so `2.05` is "Success, Content" and `4.04` is "Client Error, Not Found" -- deliberately readable as the HTTP status codes they mirror. + +```wavedrom +{reg: [ + {bits: 5, name: 'Detail (dd)'}, + {bits: 3, name: 'Class (c)'} +], config: {bits: 8, lanes: 1}} +``` + +| Code | Meaning | HTTP equivalent | +|---|---|---| +| 0.01 -- 0.04 | Request methods | GET, POST, PUT, DELETE | +| 2.01 | Created | 201 | +| 2.02 | Deleted | 204 | +| 2.04 | Changed | 204 | +| 2.05 | Content | 200 | +| 4.00 | Bad Request | 400 | +| 4.04 | Not Found | 404 | +| 4.05 | Method Not Allowed | 405 | +| 5.00 | Internal Server Error | 500 | + +### Options + +Options carry everything HTTP would put in a header: Content-Format, Uri-Path, Uri-Query, Max-Age, ETag, and so on. Rather than naming each option in the wire format, CoAP encodes an **option number** and relies on **delta encoding** -- each option's header carries the difference from the previous option's number, not the absolute number -- because options must be transmitted in ascending numeric order. This keeps small option numbers (the common ones) to a single nibble of overhead. + +Options are also marked **critical** or **elective** by convention (odd option numbers are critical, even are elective): if a recipient does not understand a critical option, it must reject the message; an unrecognised elective option can simply be ignored. This is how CoAP extensions (Observe, Block-wise, both discussed below) stay backward-compatible -- they are defined as options, and an endpoint that predates them either understands the option number as critical and correctly rejects an exchange it cannot support, or ignores it if defined as elective. + +```dot +digraph option_handling { + rankdir=TB; + node [shape=box]; + recv [label="Unrecognised option received"]; + parity [label="Option number odd or even?", shape=diamond]; + crit [label="Critical\n(odd)\nReject the message"]; + elec [label="Elective\n(even)\nIgnore and continue"]; + recv -> parity; + parity -> crit [label="odd"]; + parity -> elec [label="even"]; +} +``` + +## 5. Observe: subscribe instead of poll + +Plain CoAP is request/response, same as HTTP. The **Observe** option (RFC 7641) turns a single GET into a standing subscription: the client sets the Observe option (value 0) on a GET, and the server, instead of returning one response, keeps the "request" open and pushes a new response every time the observed resource's state changes -- each notification reusing the same Token, with an increasing Observe sequence number so the client can discard stale, out-of-order notifications. + +```mermaid +sequenceDiagram + participant Client + participant Server + Client->>Server: CON GET /sensor/temp, Observe=0 (Token=0x5) + Server-->>Client: ACK 2.05 Content, Observe=12, value=21.4 (Token=0x5) + Note over Server: value changes + Server->>Client: CON 2.05 Content, Observe=13, value=22.1 (Token=0x5) + Client-->>Server: ACK + Note over Server: value changes again + Server->>Client: CON 2.05 Content, Observe=14, value=21.9 (Token=0x5) + Client-->>Server: ACK +``` + +This is the mechanism LwM2M's Information Reporting interface builds on directly: an LwM2M "Observe" on a resource is, at the wire level, exactly this CoAP Observe relationship. + +> **Note:** Observe relationships are soft state. A server that reboots forgets all its subscriptions; a well-behaved client re-subscribes when it notices notifications have stopped (or on any transport-level reset, such as an RST for a stale MID). + +## 6. Block-wise transfer + +UDP datagrams -- and NB-IoT's radio frames beneath them -- have practical size limits well below what a firmware image or a bulk data payload needs. **Block-wise transfer** (RFC 7959) splits a large payload across multiple CoAP messages, each carrying a Block1 (request body) or Block2 (response body) option that encodes a block number, a "more blocks follow" flag, and the block size (a power of two from 16 to 1024 bytes). + +```mermaid +sequenceDiagram + participant Client + participant Server + Client->>Server: GET /firmware, Block2=(num=0, size=512, more=0) + Server-->>Client: 2.05 Content, Block2=(num=0, size=512, more=1), 512 bytes + Client->>Server: GET /firmware, Block2=(num=1, size=512, more=0) + Server-->>Client: 2.05 Content, Block2=(num=1, size=512, more=1), 512 bytes + Note over Client,Server: continues until more=0 +``` + +Each block is its own independent CoAP request/response (its own MID, though typically the same Token), so any single block can be retransmitted on its own without re-sending the whole transfer -- important on a lossy radio link where re-sending an entire firmware image because of one dropped datagram would be wasteful. This is the mechanism behind LwM2M firmware push/pull over the Firmware Update object. + +The Block1/Block2 option value itself is a single packed field (1-3 bytes depending on how large NUM needs to be), not three separate options: + +```wavedrom +{reg: [ + {bits: 3, name: 'SZX'}, + {bits: 1, name: 'M'}, + {bits: 20, name: 'NUM'} +], config: {bits: 24, lanes: 1}} +``` + +`SZX` encodes the block size as a power of two (0 = 16 bytes, up to 6 = 1024 bytes), `M` is the "more blocks follow" flag, and `NUM` is the block number -- all three squeezed into as few bytes as the current NUM value needs, which is why the option's wire length varies as a transfer progresses past block 15, then block 4095, and so on. + +## 7. Security: CoAP over DTLS + +CoAP's security is provided by DTLS (Datagram TLS) underneath it, not by CoAP itself -- `coaps://` on port 5684 is CoAP running inside a DTLS session, structurally identical to how HTTPS is HTTP inside TLS. LwM2M defines four security modes for this channel (NoSec, PSK, RPK, Certificate); the certificate chain, key formats, and handshake mechanics are covered in *Certificates and Keys: A Deep Explainer* and are not repeated here -- the relevant point for CoAP itself is simply that DTLS sits directly below it in the stack, unaware of CoAP's message semantics, so everything above (Message IDs, Tokens, Observe, Block-wise) works identically whether or not DTLS is present. + +## 8. CoAP compared to alternatives + +| | CoAP | MQTT | HTTP/1.1 | +|---|---|---|---| +| Transport | UDP (usually) | TCP | TCP | +| Model | Request/response + Observe | Publish/subscribe | Request/response only | +| Header overhead | ~4 bytes + options | ~2 bytes + variable | Hundreds of bytes (text) | +| Connection state | None required | Persistent broker connection | Persistent or per-request | +| Broker/server needed | No (direct client-server) | Yes (broker) | No | +| Multicast | Native support | No | No | +| Typical use | Device management, sensors | Telemetry, messaging | Web | + +CoAP's advantage over MQTT for device management specifically is that it needs no broker and no persistent connection -- a device can go dormant between NB-IoT PSM windows and still be addressable by URI when it wakes, whereas MQTT needs its TCP session and subscriptions re-established. This is the main reason LwM2M is built on CoAP rather than MQTT. + +## 9. Why this matters for LwM2M + +Everything LwM2M does maps directly onto the mechanisms above: + +- LwM2M **Read/Write/Execute** operations are CoAP GET/PUT-or-POST/POST +- LwM2M **Observe/Notify** is CoAP's Observe option, unmodified +- LwM2M **firmware push and pull** use Block-wise transfer +- LwM2M's four **security modes** are DTLS configurations sitting below an otherwise unchanged CoAP layer +- LwM2M's compact **object/resource path** model (`/3/0/1`) is simply the Uri-Path option, structured by convention + +The LwM2M deep dive builds on this document rather than re-deriving it -- if a CoAP mechanism looks unfamiliar while reading that document, it is explained here. diff --git a/content/guides/deep-dives/docs-diagrams-and-architecture-as-code.md b/content/guides/deep-dives/docs-diagrams-and-architecture-as-code.md new file mode 100644 index 0000000..ac7c7f5 --- /dev/null +++ b/content/guides/deep-dives/docs-diagrams-and-architecture-as-code.md @@ -0,0 +1,239 @@ +--- +title: Docs, diagrams and architecture as code +--- + +# Docs, diagrams and architecture as code + +## 1. Overview + +Three related practices, one underlying idea: treat documentation, diagrams, and architecture models as plain-text artefacts that live in version control and are built into publishable output by a pipeline, rather than edited by hand in a proprietary tool. + +- **Docs as code** -- prose treated like source code +- **Diagrams as code** -- visuals generated from a textual definition +- **Architecture as code** -- the architecture model itself expressed as a structured, queryable, version-controllable definition, from which diagrams and prose are derived + +Alongside these, there's a fourth thing worth keeping distinct: **architecture frameworks and methodologies** (TOGAF, ArchiMate, Zachman, SysML, Team Topologies, and similar). These describe *what* to model, *how* to categorise it, or *how to think about* a system's structure. They are not, in themselves, code -- most predate the "as code" movement and most of their common tooling is GUI-driven or repository-based rather than text-and-diff based. Some can be *implemented* using as-code tooling (an ArchiMate model can be authored in Archi and stored as versionable XML; a Wardley Map can be written as text), but the framework itself is a way of thinking, not an artefact format. + +This document surveys the landscape in each category -- what exists and what it's for -- rather than prescribing a specific pipeline. + +```mermaid +graph TD + F[Architecture Frameworks & Methodologies
TOGAF, ArchiMate, SysML, Team Topologies] -.informs.-> A + A[Architecture as Code] --> B[Diagrams as Code] + B --> C[Docs as Code] +``` + +> **Note:** the dotted line matters. Frameworks inform *what* an architecture model should contain; they don't dictate that the model must be text-based and version-controlled. That's a separate, tooling-level choice. + +## 2. Docs as Code: The Landscape + +### 2.1 The movement and its philosophy + +"Docs as code" originated as a community position, not a single tool. The **Write the Docs** community is its main gathering point, and the underlying argument is: documentation quality improves when writers use the same tools, workflows, and review discipline as developers (version control, pull requests, CI, issue tracking), rather than a separate wiki or CMS with no review trail. + +Related philosophies worth knowing: + +- **README-driven development** -- write the README (and usage docs) before the code, as a design step +- **Docs-like-code vs docs-in-code** -- a distinction between documentation stored alongside source (in-repo Markdown) versus documentation *generated from* source (docstrings, code comments) -- the two are complementary, not competing +- **Minimum viable documentation** -- an argument (associated with Google's internal engineering practices) for documenting only what's load-bearing, avoiding documentation debt from over-specification + +### 2.2 Structuring frameworks (what content goes where, not code themselves) + +These frameworks answer "how should the documentation set be organised", independent of markup or tooling. Like the architecture frameworks discussed later, they are conceptual templates rather than artefact formats. + +| Framework | Origin/domain | Core idea | +| --- | --- | --- | +| **Diátaxis** | Web/software docs (Django docs origin) | Splits all documentation into four types: tutorials, how-to guides, reference, and explanation -- each with different writing rules | +| **arc42** | Software architecture (German-speaking enterprise/embedded world) | A 12-section template for architecture documentation specifically (context, constraints, building blocks, runtime, deployment, risks, decisions, glossary) | +| **DITA (Darwin Information Typing Architecture)** | Technical publishing, aerospace/defence, regulated industries | An OASIS XML standard for topic-based authoring: content is written as reusable "topics" (concept, task, reference) and assembled into different outputs via "maps" -- this one *does* double as a genuine as-code format, since DITA content is XML and version-controllable, unlike the other frameworks in this table | +| **Divio documentation system** | General software docs | A close relative/precursor of Diátaxis, same four-part split | +| **Google Developer Documentation Style Guide** | Style, not structure | A widely referenced house style (voice, terminology, formatting conventions) rather than a document skeleton | +| **Microsoft Writing Style Guide** | Style, not structure | Microsoft's equivalent; commonly used as a base for corporate style guides | + +### 2.3 Markup and authoring formats + +| Format | Character | Typical use | +| --- | --- | --- | +| **CommonMark / GitHub Flavored Markdown (GFM)** | Lightweight, widely supported | General docs, READMEs, wikis | +| **MyST Markdown** | Markdown extended with reStructuredText-like directives | Technical/scientific docs needing cross-references, admonitions, and Sphinx integration without full reST | +| **AsciiDoc** | More expressive than Markdown (native tables, includes, conditionals, cross-references) | Larger technical manuals, books, arc42 templates are often authored in AsciiDoc | +| **reStructuredText (reST)** | Python ecosystem standard, verbose syntax | Sphinx-based documentation, especially Python project docs | +| **DITA XML** | Structured, topic-based, verbose | Enterprise/regulated technical publishing with heavy reuse requirements | +| **Textile** | Lightweight, older | Largely superseded by Markdown, still seen in legacy systems | + +### 2.4 Static site generators and publishing tooling + +| Tool | Language/ecosystem | Notable for | +| --- | --- | --- | +| **MkDocs** (+ Material theme) | Python | Simple, fast, popular for technical/engineering docs; Material theme adds strong navigation and search | +| **Sphinx** | Python | The Python ecosystem standard; strong cross-referencing, extension ecosystem (autodoc for code-derived docs) | +| **Docusaurus** | JavaScript/React | Versioned docs, i18n, strong for open-source project documentation | +| **Antora** | JavaScript, AsciiDoc-based | Purpose-built for multi-repository documentation sets -- pulls docs from many repos into one site, common in arc42/enterprise contexts | +| **Hugo** | Go | General-purpose static site generator, very fast builds, used for docs and general sites alike | +| **Jekyll** | Ruby | Older, GitHub Pages' original native generator | +| **Docsify / VuePress / GitBook** | JavaScript | Lighter-weight alternatives, GitBook has a hosted SaaS offering | +| **mdBook** | Rust | Common for Rust-ecosystem docs and technical books | +| **Zola** | Rust | Single-binary static site generator, no dependency management overhead | +| **Read the Docs** | Hosting platform | Not a generator itself -- hosts Sphinx/MkDocs output with automatic versioned builds triggered from git | + +### 2.5 API documentation as code + +A specialised branch, since API references are usually generated from a machine-readable contract rather than written by hand. + +| Standard/tool | Purpose | +| --- | --- | +| **OpenAPI Specification (formerly Swagger)** | The dominant standard for describing REST APIs as a YAML/JSON contract | +| **AsyncAPI** | Equivalent standard for event-driven/async APIs (message queues, webhooks) | +| **Redoc** | Renders an OpenAPI spec into a polished reference site | +| **Swagger UI** | Interactive OpenAPI explorer, "try it out" style | +| **Slate / Stoplight** | Alternative API doc renderers with a more narrative, three-column layout | + +### 2.6 Doc quality and linting + +| Tool | Checks | +| --- | --- | +| **Vale** | Prose style linting (configurable to enforce a house style guide, catches passive voice, banned terms, etc.) | +| **markdownlint** | Markdown formatting/syntax consistency | +| **alex** | Catches insensitive or inconsiderate phrasing | +| **write-good / textlint** | General prose quality heuristics | + +## 3. Diagrams as Code: The Landscape + +### 3.1 General-purpose diagramming languages + +| Tool | Underlying model | Strengths | +| --- | --- | --- | +| **Graphviz (DOT)** | Directed/undirected graphs, automatic layout | The foundational layout engine; many higher-level tools compile down to DOT | +| **Mermaid** | Text syntax rendered client-side or via CLI | Flowcharts, sequence, class, state, ER, Gantt, and C4-ish diagrams; renders natively in GitHub, GitLab, Notion | +| **PlantUML** | Text syntax, Java-based renderer | Strong UML support (all diagram types), mature ecosystem, C4-PlantUML extension | +| **D2** (Terrastruct) | Text syntax, modern renderer | More visually polished default output than Mermaid/PlantUML, good for architecture-style diagrams | +| **Nomnoml** | Lightweight text syntax | Simple UML-ish class/box diagrams | +| **Ditaa** | ASCII-art to diagram | Converts hand-drawn-looking ASCII boxes into cleaner rendered diagrams | +| **svgbob** | ASCII-art to SVG | Similar niche to Ditaa, Rust-based | + +### 3.2 UML-specific tooling + +- **PlantUML** covers the full UML spec (class, sequence, activity, state, component, deployment, use case) +- **yUML** -- a simpler, URL-based UML sketching tool, good for quick embeds +- **UMLet** -- has a text/DSL mode alongside its GUI + +### 3.3 Cloud and infrastructure diagrams + +| Tool | Approach | +| --- | --- | +| **Diagrams** (Python library, "diagrams" on PyPI) | Python code with provider-specific icon sets (AWS, Azure, GCP, Kubernetes) renders architecture diagrams -- popular because the diagram source is executable Python | +| **Cloudcraft** | SaaS, isometric AWS architecture diagrams, some automation from live AWS accounts | +| **AWS CDK / Terraform visualisation plugins** | Not diagramming tools per se, but some IaC tooling can generate a dependency graph as a diagram from the deployed infrastructure definition | + +### 3.4 Sequence, timing, and network-family diagrams + +| Tool | Diagram type | +| --- | --- | +| **WaveDrom** | Digital timing diagrams (bit/signal waveforms) -- relevant in embedded/hardware contexts | +| **js-sequence-diagrams** | Lightweight sequence diagrams, precursor to Mermaid's sequence support | +| **blockdiag family** (blockdiag, seqdiag, actdiag, nwdiag, rackdiag) | A Python-based family covering block diagrams, sequence, activity, network, and rack diagrams from similar-looking text syntax | + +### 3.5 Data model and entity-relationship diagrams + +| Tool | Notes | +| --- | --- | +| **dbdiagram.io / DBML** | A dedicated markup language (Database Markup Language) for ER diagrams, popular for quick schema sketches | +| **SchemaSpy** | Generates ER diagrams and documentation directly from a live database's metadata (reverse-engineering rather than hand-authored) | +| **Mermaid ER diagrams** | Built into Mermaid, adequate for simpler schemas | + +### 3.6 Whiteboard-style diagrams and Wardley Maps + +| Tool | Purpose | +| --- | --- | +| **Excalidraw** | Primarily a GUI whiteboard tool, but supports a JSON/text scene format that can be scripted or templated | +| **Eraser.io** | SaaS with a text-to-diagram DSL alongside a whiteboard canvas | +| **OnlineWardleyMaps** | A genuine text syntax for **Wardley Mapping** -- plots components on a value chain against an evolution axis (genesis to commodity); this is one of the few strategy-oriented approaches with a real as-code implementation, unlike most other strategic frameworks (see section 5) | + +### 3.7 Academic/embedded-adjacent tooling + +| Tool | Context | +| --- | --- | +| **TikZ/PGF** | LaTeX-native diagramming, common in academic papers and formal specifications | +| **Asymptote** | A vector graphics language, sometimes used for precise technical illustration in academic/embedded documentation | + +## 4. Architecture as Code: The Landscape + +This section covers tools where the architecture model itself is a text (or otherwise diffable) artefact that generates views, as opposed to frameworks that describe what a model should contain (covered separately in section 5). + +### 4.1 C4 model implementations + +| Tool | Character | +| --- | --- | +| **Structurizr DSL** | The reference implementation of the C4 approach -- one model, multiple derived views (context, container, component, deployment, dynamic); plain text, version-controllable | +| **C4-PlantUML** | A set of PlantUML macros/includes that let you draw C4-style diagrams within existing PlantUML tooling, without adopting a separate DSL or model -- closer to diagrams-as-code than a true single-model system, since each view is drawn independently rather than derived from one shared model | +| **LikeC4** | A TypeScript-based alternative to Structurizr DSL, open source, with a VS Code extension and its own web viewer; genuinely text-first | +| **IcePanel** | A browser-based, GUI-first C4 modelling tool (SaaS); the model lives in IcePanel's own store rather than a text file, so it sits closer to a hosted model repository than a strict "as code" artefact, though it can export data | + +### 4.2 Decision records and service catalogues as code + +| Tool/format | Notes | +| --- | --- | +| **Architecture Decision Records (ADRs)** | Short Markdown/text documents capturing one decision, its context, and consequences -- genuinely as-code, since they're plain text files reviewed like any other change | +| **Michael Nygard's original ADR format** | Title, Status, Context, Decision, Consequences -- the template most others derive from | +| **MADR (Markdown Architecture Decision Records)** | A more structured template, adds sections for considered options and decision drivers | +| **adr-tools** | A small CLI for creating and numbering ADRs following a consistent naming convention | +| **log4brains** | Generates a browsable, searchable static site from a folder of ADRs | +| **Backstage** (Spotify, now CNCF) | Each service/component described by a `catalog-info.yaml` file -- architecture-as-code at the "who owns what, what depends on what" level, aggregated across repos into a browsable catalogue | + +### 4.3 Infrastructure as code (adjacent, often conflated) + +Worth distinguishing clearly: these describe the *deployed infrastructure*, not the *software architecture*, though they're frequently mentioned in the same breath. + +| Tool | Scope | +| --- | --- | +| **Terraform** | Cloud-agnostic infrastructure provisioning via HCL | +| **AWS CloudFormation** | AWS-native infrastructure provisioning via YAML/JSON | +| **Pulumi** | Infrastructure provisioning using general-purpose programming languages rather than a DSL | +| **AWS CDK** | Infrastructure defined in TypeScript/Python/etc., synthesises to CloudFormation | + +> **Important:** infrastructure-as-code tools describe what gets *deployed*. Architecture-as-code tools (Structurizr, LikeC4) describe how the *system is structured and reasoned about*. A deployment view in Structurizr might reference the same environment that Terraform provisions, but the two serve different audiences and rarely share a single source file. + +## 5. Architecture Frameworks and Methodologies (Not Code) + +These are ways of thinking about architecture, not artefact formats. None of them require a text-based, version-controlled model, though some can be implemented that way with the right tooling. They're included here because they're commonly discussed alongside architecture-as-code, and it's worth being clear about which is which. + +### 5.1 Enterprise architecture frameworks + +| Framework/tool | Character | +| --- | --- | +| **TOGAF** | The dominant enterprise architecture methodology; defines an Architecture Development Method (ADM) and a broad set of artefact types -- a process and a set of deliverable categories, not a modelling notation or a code format | +| **ArchiMate** (language) + **Archi** (open-source tool) | A modelling *notation* for expressing TOGAF-style enterprise architecture across business, application, and technology layers. Archi stores models as `.archimate` XML, which is technically version-controllable -- but the authoring workflow is GUI-first (drag-and-drop modelling), and diffs on the underlying XML are not human-readable the way a Structurizr DSL diff is. Best thought of as "model-based" rather than "as code" | +| **Zachman Framework** | A classification grid (who/what/when/where/why/how across stakeholder perspectives) for organising enterprise architecture artefacts -- a taxonomy, not a notation or a tool | + +### 5.2 Systems engineering + +| Framework/tool | Character | +| --- | --- | +| **SysML** (Systems Modelling Language) | A UML dialect extended for systems engineering -- requirements, behaviour, structure, and parametrics for systems that include hardware, not just software | +| **MBSE tooling** (e.g. Cameo Systems Modeler, Rational Rhapsody) | Commercial Model-Based Systems Engineering tools built around SysML, common in aerospace, defence, and automotive; models typically live in a proprietary repository with GUI editing, not plain text -- SysML v2 introduces a textual notation, which may close this gap over time | + +### 5.3 Organisational and strategic frameworks + +| Framework | Character | +| --- | --- | +| **Team Topologies** | A framework for organisational design (stream-aligned, platform, enabling, complicated-subsystem teams) -- describes how teams should be shaped around a system's architecture; no code artefact of its own, though team-boundary diagrams are often drawn using the diagrams-as-code tools in section 3 | +| **Domain-Driven Design (DDD) context maps** | Diagrams showing bounded contexts and their relationships (partnership, customer/supplier, anticorruption layer, etc.) -- a modelling *technique*, typically rendered with general-purpose diagrams-as-code tools (Mermaid, PlantUML) rather than a dedicated DSL | +| **Event Storming** | A workshop technique producing a timeline of domain events, usually captured live as sticky notes -- not code, though the output is sometimes transcribed afterwards into a diagram for permanence | +| **Wardley Mapping** | The one exception in this section with a genuine as-code implementation (see section 3.6) -- included here too because the mapping *technique* itself (reasoning about value chains and evolution) is the methodology, while the text syntax is just one way of producing the artefact | + +## 6. How the Categories Compare + +| | Docs as code | Diagrams as code | Architecture as code | Frameworks/methodologies | +| --- | --- | --- | --- | --- | +| **What it is** | Narrative content | A rendered visual | A structured, diffable model | A way of thinking about structure | +| **Is it itself code?** | Yes | Yes | Yes | No -- may be *implemented* using as-code tooling, but isn't code by definition | +| **Primary audience** | Anyone consuming the documentation | Anyone reading a specific view | Architects, and tools/queries that consume the model | Architects and stakeholders reasoning about the system | +| **Typical source format** | Markdown, AsciiDoc, reST, DITA | Mermaid, PlantUML, D2, DOT, DBML | Structurizr DSL, LikeC4, ADRs, `catalog-info.yaml` | Documents, workshops, GUI model repositories | +| **Reuse across outputs** | Low | Medium (one source, multiple render formats) | High (one model, many derived views) | N/A -- describes intent, not an artefact | + +## 7. Choosing Between Approaches (Considerations, Not a Recipe) + +- **Regulated/embedded/defence contexts** tend to gravitate toward heavier tooling for documentation reuse (DITA) and systems modelling (SysML/MBSE), often alongside enterprise architecture governance (TOGAF/ArchiMate) mandated at organisational level -- these frameworks and their tooling are frequently GUI/repository-based rather than plain-text, which is a genuine tooling trade-off, not an oversight +- **Web/product engineering teams** more often favour lightweight text formats (Markdown, Mermaid, Structurizr/LikeC4) and static site generators, prioritising low friction and diffability over formal traceability +- **Enterprise architecture governance** (TOGAF/ArchiMate) and **software architecture-as-code** (Structurizr/LikeC4) address different altitudes and aren't substitutes -- larger organisations often run both, connected loosely (an ArchiMate application layer element might correspond to a Structurizr software system) rather than through a single shared tool +- **Strategic framing approaches** (Wardley Mapping, Team Topologies, DDD context maps) are almost always additive to whichever architecture-as-code tool is in use -- they explain *why* the structure exists, and only Wardley Mapping among them has a mainstream text-based implementation diff --git a/content/guides/deep-dives/git-internals.md b/content/guides/deep-dives/git-internals.md new file mode 100644 index 0000000..dc372c4 --- /dev/null +++ b/content/guides/deep-dives/git-internals.md @@ -0,0 +1,369 @@ +--- +title: Git internals +--- + +# Git internals + +## Overview + +Git is often described as a "version control system", but its core is simpler and stranger than that phrase suggests: it is a content-addressable object store, with a directed acyclic graph (DAG) of commits layered on top. Understanding those two ideas -- content addressing and the DAG -- explains almost everything else: why hashes look the way they do, why branches are so cheap, and what actually happens during a merge. + +## What Is a Commit? + +A commit is a snapshot, not a diff. Each commit points at a tree object recording the complete state of every file at that moment, plus a pointer to its parent commit(s) and metadata (author, committer, timestamp, message). Git computes differences on demand, by comparing two snapshots, when you run something like `git diff` or `git log -p`. A diff is never stored inside a commit; it's derived at the point you ask for it. + +Getting from a changed file to a commit goes through an intermediate step: the staging area (also called the index). Running `git add` doesn't create a commit; it copies the current state of a file into the staging area, which is Git's draft of the *next* tree object. `git commit` then takes whatever is in the staging area, writes it as a tree, and wraps it in a new commit object. This is why partial commits are possible: you can `git add` only some of your changes and commit just those, leaving the rest unstaged. + +```mermaid +graph LR + WD["Working directory\n(files you edit)"] -- "git add" --> SA["Staging area / index\n(draft of next tree)"] + SA -- "git commit" --> REPO["Repository\n(new commit object)"] +``` + +## Adding, Changing and Deleting Files + +Git tracks a file's state relative to the last commit as one of: untracked (new, unknown to Git), modified (tracked but changed since the last commit), or staged (copied into the index, ready to be part of the next commit). What actually happens in the object database depends on which of these three actions you take, and none of them ever edit an existing object in place. + +### Added (new file) + +The file starts untracked. Staging it (`git add`) computes a blob hash from its full content and adds an entry for that path to the staged tree. The next commit's tree gains that entry; nothing in the object database referenced the file before this point. + +### Changed (existing file) + +The file shows as modified in the working directory. Staging it hashes the new content, producing a new blob -- the old blob is untouched and remains referenced by every earlier commit that had it. The next commit's tree entry for that path points at the new blob instead of the old one, and every tree above it on the path to the root also changes, since a tree's own hash depends on the hashes of everything it contains. + +> **Note:** the new blob holds the file's complete new content, not a diff against the old version. Git has no "old blob plus changed lines" object; each blob is a full, standalone snapshot of one file's content, with a header (`blob \0`) prepended before hashing, then compressed for storage. Compact storage comes later, when Git packs similar objects together in a packfile; that's an on-disk optimisation, not a change to the logical model. + +### Deleted + +Staging a deletion (`git rm`, or `git add` after removing the file) removes that path's entry from the staged tree entirely. The next commit's tree simply has no entry for that path. The blob for the old content is not deleted; it still sits in the object database and is still referenced by every earlier commit that had the file, so checking out an older commit brings it straight back. + +```mermaid +graph LR + subgraph "Added" + A1["No blob"] -- "git add" --> A2["New blob\n(new tree entry)"] + end + subgraph "Changed" + B1["Blob v1\n(still referenced by\nolder commits)"] -- "git add" --> B2["Blob v2\n(new tree entry\nreplaces v1's)"] + end + subgraph "Deleted" + C1["Blob\n(still referenced by\nolder commits)"] -- "git add" --> C2["No tree entry\n(blob untouched)"] + end +``` + +The common thread across all three: nothing is ever edited in place. Every add, change or delete produces a new blob and/or a new set of tree objects, never modifies an old one, and only becomes permanent once wrapped in a new commit. + +## The Commit DAG + +Every commit points to one or more parent commits (except the very first commit, which has none). Because a commit can never point forward in time to something that doesn't exist yet, and history only ever grows backwards through parent links, the structure is acyclic -- there are no loops. + +```mermaid +graph RL + C1[C1: initial commit] --> C2[C2] + C2 --> C3[C3] + C3 --> C4[C4] + C3 --> C5[C5] + C4 --> C6[C6: merge] + C5 --> C6 +``` + +A few things to notice: + +- Arrows point *backwards*, from child to parent, because a commit is created after its parent and records that parent's hash at creation time. +- `C3` has two children (`C4` and `C5`), which happens when a branch is created at that point and both lines of development continue independently. +- `C6` has two parents (`C4` and `C5`), which is what a merge commit is -- a single commit that records two prior histories as both being its ancestors. + +The DAG is the entire "shape" of a repository's history. Branches, tags and `HEAD` are just labelled pointers into this graph; they don't change its shape, they just mark positions within it. + +## What a Hash Actually Represents + +Git's object database stores four kinds of objects, and every one of them is identified by the SHA-1 (or SHA-256, on newer repositories) hash of its own content. This is what "content-addressable" means: the name of an object is derived mathematically from what's inside it, not assigned by a human or a counter. + +```dot +digraph git_objects { + rankdir=LR; + node [shape=box, fontname="Helvetica"]; + + commit [label="commit\n(hash: a1b2c3...)\n- tree\n- parent(s)\n- author\n- message"]; + tree [label="tree\n(hash: d4e5f6...)\n- entries: mode, type, hash, name"]; + blob1 [label="blob\n(hash: 111aaa...)\ncontents of file A"]; + blob2 [label="blob\n(hash: 222bbb...)\ncontents of file B"]; + subtree [label="tree\n(hash: 333ccc...)\nsubdirectory"]; + blob3 [label="blob\n(hash: 444ddd...)\ncontents of file C"]; + + commit -> tree; + tree -> blob1; + tree -> blob2; + tree -> subtree; + subtree -> blob3; +} +``` + +The consequences of this design are what give Git its distinctive behaviour: + +- A `blob` is just file contents. If two files anywhere in the repository (or across its whole history) have identical contents, they are stored as the *same* blob, referenced from multiple places. Renaming a file with no content change doesn't create a new object. +- A `tree` is a directory listing: a set of names, modes and the hashes of the blobs or subtrees they point to. If a directory's contents are unchanged between two commits, both commits' trees can point at the exact same tree object. +- A `commit` object is small: it holds a pointer to one tree (the complete snapshot of the repository at that moment), pointers to zero or more parent commits, and metadata (author, committer, timestamp, message). +- Because the hash is computed from content, changing anything -- a single character in a file, the commit message, the author's name, the parent pointer -- produces a completely different hash. This is also why hashes are how Git detects corruption: if content and hash ever disagree, something is wrong. + +The diagram below makes deduplication concrete: two commits where only one file changed between them end up sharing every object except the one blob that differs, the tree that contains it, and the two commit objects themselves. + +```mermaid +graph TB + Commit1["Commit 1"] --> Tree1["Tree (root)"] + Commit2["Commit 2"] --> Tree2["Tree (root, new)"] + + Tree1 --> BlobA["Blob: file-a.txt\n(unchanged)"] + Tree2 --> BlobA + Tree1 --> BlobB1["Blob: file-b.txt\n(old content)"] + Tree2 --> BlobB2["Blob: file-b.txt\n(new content)"] +``` + +`file-a.txt` didn't change, so both commits' trees point at the exact same blob. `file-b.txt` changed, so it gets a new blob, which forces a new tree (since the tree's own content, and therefore its hash, includes the list of blob hashes it points to), which forces a new commit. Nothing else in the repository needs to be duplicated. + +So when people say "commit `a1b2c3`", they mean: the unique fingerprint of one exact snapshot of the whole repository at one point in the DAG, plus the metadata describing how it got there. + +### Long hash versus short hash + +There is only one hash per object, either 40 hex characters (SHA-1) or 64 (SHA-256, on newer repositories). The abbreviated hashes you see in `git log --oneline` or `git show` are not a separate identifier; they're a display shorthand. Git starts with 7 characters and checks whether that prefix uniquely identifies one object in the current repository; if there's a collision, it uses more characters until the prefix is unique. Internally, Git always expands a short hash back to the full one before using it. + +> **Note:** the short hash is not stable across all repositories or forever. As a repository grows, a prefix that was unique yesterday can collide with a new object tomorrow, at which point Git will start showing more characters for it. + +## Packing: How Objects Are Actually Stored + +Everything so far describes the logical model: every object (blob, tree, commit, tag) is a full, independent, content-addressed unit. Physically, Git stores objects in one of two forms, and it's worth being clear that switching between them never changes an object's hash or content, only how many bytes it takes up on disk. + +A newly created object starts as a loose object: one file per object, named after its hash, held in `.git/objects/`, compressed individually with zlib. This is simple but wasteful, since many blobs in a repository's history are near-identical (a file changed by a few lines still produces a wholly new blob). + +Periodically -- during `git gc`, when pushing or fetching over a threshold, or when `git repack` is run directly -- Git bundles loose objects into a packfile. Inside a pack, Git finds similar objects and stores most of them as a delta against a chosen base object, rather than storing their full content again. An accompanying index file (`.idx`) records where each object's data starts inside the pack, so looking up an object by hash is still fast even though the objects themselves are no longer one-file-each. + +```mermaid +graph LR + subgraph "Loose objects" + L1["objects/ab/cdef...\n(full content, zlib)"] + L2["objects/12/3456...\n(full content, zlib)"] + L3["objects/78/90ab...\n(full content, zlib)"] + end + + subgraph "After git gc / repack" + P["pack-xyz.pack\n(delta-compressed objects)"] + I["pack-xyz.idx\n(hash -> offset lookup)"] + end + + L1 --> P + L2 --> P + L3 --> P + P --- I +``` + +> **Important:** delta compression here is purely a storage-layer trick for making the pack file smaller; it's unrelated to the diffs `git diff` shows you, and it doesn't change what a blob logically contains. A packed blob still decompresses back to the exact same full content, and still has the exact same hash, as it did when loose. + +## Branches Are Just Pointers + +A branch is not a container for commits. It's a small file containing a single 40-(or 64-)character hash: the commit it currently points to. `HEAD` is similarly just a pointer, usually to a branch name, which in turn points to a commit. + +```mermaid +graph RL + C1[C1] --> C2[C2] + C2 --> C3[C3] + main["main"] -.-> C3 + HEAD["HEAD"] -.-> main +``` + +Creating a branch is cheap for exactly this reason: it's writing one line to one small file, not copying any commit data. When a new commit is made on a branch, Git: + +1. writes the new commit object, with its parent set to whatever the branch currently points to +2. moves the branch pointer forward to the new commit's hash + +```mermaid +graph RL + C1[C1] --> C2[C2] + C2 --> C3[C3] + C3 --> C4[C4: new commit] + main["main"] -.-> C4 + HEAD["HEAD"] -.-> main +``` + +Note that `C3` still exists in the object database; only the pointer moved. This is also why deleting a branch is safe in the short term (the commits are only garbage-collected once nothing references them and enough time has passed) but can make history hard to find later, since the label pointing at it is gone. + +> **Note:** unreferenced commits aren't deleted immediately. Git periodically packs loose objects into packfiles for storage efficiency, and a separate garbage collection pass (`git gc`) removes objects that are both unreachable from any branch, tag or `reflog` entry, and older than a grace period (a couple of weeks, by default). The `reflog` in particular is what makes "deleted" commits recoverable for a while: it's a local log of everywhere `HEAD` has pointed, independent of the DAG's own reachability. + +> **Important:** it's possible to check out a commit directly, by hash, rather than a branch name. `HEAD` then points straight at that commit instead of at a branch, a state Git calls "detached HEAD". New commits made in this state still get written to the object database, but no branch pointer moves to follow them, so they become unreachable and eligible for garbage collection the moment you check out something else, unless you create a branch to point at them first. + +## Tags + +A tag is another kind of pointer into the DAG, but with a different intent to a branch: it marks a specific point permanently (typically a release), rather than tracking the tip of ongoing work. Git supports two kinds. + +A lightweight tag is exactly like a branch pointer: a name pointing directly at a commit hash, with nothing else attached. It never moves on its own (nothing commits "onto" a tag), but it's just as easy to move or delete as a branch. + +An annotated tag is its own object in the database, with its own hash, distinct from the commit it points to. It carries a tagger name, timestamp, message, and optionally a GPG signature, which is why annotated tags are the recommended choice for releases: the tag itself is a verifiable, dated record, not just a label. + +```mermaid +graph RL + C5[C5] --> C4[C4] + v1["v1.0.0\n(annotated tag object)"] -.-> C5 + main["main"] -.-> C4 +``` + +## Remote-Tracking Branches + +A remote-tracking branch such as `origin/main` is a third category of pointer, easy to confuse with a local branch because it looks similar in `git branch -a` and `git log`. It records where a branch was on a given remote, last time Git talked to it (`fetch`, `pull`, `push`). It is not a pointer you commit onto directly; it only moves when Git talks to the network. + +```mermaid +graph RL + C1[C1] --> C2[C2] + C2 --> C3[C3] + C3 --> C4[C4: local-only commit] + main["main (local)"] -.-> C4 + origin["origin/main (remote-tracking)"] -.-> C3 +``` + +Here, `main` has moved on with a local commit that hasn't been pushed yet, so `origin/main` still points at the last commit Git knows the remote had. Running `git push` would move `origin/main` forward to match; running `git fetch` updates `origin/main` to wherever the remote server's `main` actually is, without touching your local `main` or your working directory at all. + +## Merging + +When two branches have diverged -- each has commits the other doesn't -- Git needs a way to bring them back together. There are two common outcomes, depending on the shape of the DAG at the time. + +### Fast-forward merge + +If the branch being merged in is a direct descendant of the current branch (no divergence at all), Git can simply move the pointer forward. No new commit is created. + +```mermaid +graph RL + C1[C1] --> C2[C2] + C2 --> C3[C3] + feature["feature (before)"] -.-> C3 + main["main (before and after)"] -.-> C2 +``` + +After merging `feature` into `main` here, `main` simply becomes another pointer at `C3`. The history is linear; nothing was combined because there was nothing to combine. + +### Three-way merge (merge commit) + +If both branches have new commits since they diverged, Git performs a three-way merge, comparing: + +- the common ancestor commit (where the branches split) +- the tip of the current branch +- the tip of the branch being merged in + +```mermaid +graph RL + Base[Base: common ancestor] --> A1[A1] + A1 --> A2[A2: main tip] + Base --> B1[B1] + B1 --> B2[B2: feature tip] + A2 --> M[Merge commit] + B2 --> M +``` + +For each file, Git compares the version at the base with the versions on each branch: + +- if only one side changed a file relative to the base, that side's version wins automatically +- if both sides changed the same file in the same way, or one side didn't touch it, it resolves cleanly +- if both sides changed the *same lines* differently, that's a conflict, and a human has to choose the result + +Once every file is resolved, Git creates a new commit -- the merge commit -- whose tree reflects the resolved result and whose parents are both `A2` and `B2`. That single commit is what makes the DAG show two lines of history converging, as in the very first diagram of this document. + +### The two outcomes side by side + +The difference between the two comes down to a single question: does the current branch's tip already sit on the line of history leading to the other branch's tip? If yes, fast-forward; if no, a merge commit is required. + +```mermaid +graph TB + subgraph "Fast-forward: main is an ancestor of feature" + FF1[C1] --> FF2[C2] + FF2 --> FF3[C3: feature tip] + FFmain["main"] -.-> FF2 + end + + subgraph "Merge commit: histories diverged" + M1[Base] --> M2[A: main tip] + M1 --> M3[B: feature tip] + M2 --> M4[Merge commit] + M3 --> M4 + end +``` + +On the left, `main` only needs to move; there is no divergence to reconcile. On the right, both sides added commits the other doesn't have, so a new commit is the only way to represent "both of these histories are now part of the same line". + +## Rebase, Briefly + +Rebase is the other common way to reconcile diverged branches, and it produces a different shape of history to a merge rather than a different result. Instead of creating a merge commit that has two parents, rebase rewrites the commits on one branch so they appear to have started from the other branch's current tip, one at a time, as if they'd been written there from the start. + +```mermaid +graph RL + Base[Base] --> A1[A1] + A1 --> A2[A2: main tip] + B1prime["B1'"] --> A2 + B2prime["B2': feature tip, rebased"] --> B1prime +``` + +Compare this to the three-way merge diagram: there, `Base`, `A1`/`A2` and `B1`/`B2` all remain in the DAG, joined by a new merge commit. Here, `B1` and `B2` are replaced entirely by new commits `B1'` and `B2'`, which have the same changes but different parents, and therefore different hashes. This is why rebase is described as rewriting history: the original `B1` and `B2` still exist in the object database until garbage-collected, but no branch points at them any more, and anyone else who already had a copy of them now has a different, incompatible set of commits to yours. That's the standard reason for the advice not to rebase commits that have already been pushed and shared. + +## What's Actually in a Repository Directory + +Everything described so far, objects, the DAG, branches, tags, remote-tracking branches, packing, lives inside a single `.git` directory at the root of a working copy. That directory is the repository; the rest of the files you edit are just a checked-out snapshot of one tree object. + +```mermaid +graph TB + Git[".git/"] --> Objects["objects/\nloose objects, by hash prefix"] + Git --> Pack["objects/pack/\npackfiles + .idx files"] + Git --> Refs["refs/heads/\none file per local branch"] + Git --> RefsRemote["refs/remotes/\norigin/main etc"] + Git --> RefsTags["refs/tags/\nlightweight tag pointers"] + Git --> PackedRefs["packed-refs\nrefs bundled into one file"] + Git --> Head["HEAD\ncurrent branch or commit"] + Git --> Index["index\nthe staging area"] + Git --> Config["config\nrepo-local settings, remotes"] + Git --> Logs["logs/\nreflog: history of ref movements"] + Git --> Hooks["hooks/\nscripts run on git events"] +``` + +A few points worth calling out directly: + +- `objects/` holds loose objects, each in a subdirectory named after the first two hash characters, plus `objects/pack/` for packfiles and their `.idx` companions, as covered above. +- `refs/heads/main` is literally a text file containing the 40- (or 64-) character hash `main` currently points to. `refs/remotes/origin/main` is the same idea for a remote-tracking branch. Once a repository has many refs, Git may bundle them into a single `packed-refs` file for efficiency, functionally equivalent to lots of small files. +- `HEAD` is a text file too, normally containing something like `ref: refs/heads/main` (a pointer to a branch, itself a pointer to a commit) rather than a commit hash directly; in detached HEAD state, it holds the commit hash itself. +- `index` is the binary file backing the staging area described earlier: the draft tree for the next commit. +- `logs/` holds the reflog: a local, time-ordered record of everywhere `HEAD` and each branch have pointed, which is how `git reflog` can recover commits that no ref currently points at. +- `hooks/` contains scripts (disabled by default, just example files) that Git can run automatically at points like before a commit or after a merge. + +None of this is exposed as part of the DAG or the object model; it's the filesystem scaffolding that makes the DAG, the pointers, and the working directory all work together as one repository. + +## Comparison with Other Version Control Systems + +Git's design choices stand out more clearly against other systems, past and present. + +### RCS (legacy, file-level, no networking) + +RCS predates the idea of a shared repository. It manages version history for a single file at a time, on a single machine, using reverse deltas (the latest version is stored in full, older versions as diffs back from it). There's no concept of a commit spanning multiple files, no branching model beyond simple file-level revision trees, and no client-server or distributed operation. Tools like CVS were built on RCS-like storage to add multi-file, networked repositories, but inherited its per-file delta model. + +### Subversion (centralised, directory-versioned) + +Subversion (SVN) versions the whole repository tree, closer to Git's snapshot model than RCS's per-file deltas, but it is centralised: there's one authoritative repository on a server, and every commit requires network access to it. History is linear and identified by an incrementing global revision number rather than a content hash, so two people's local checkouts never disagree about what revision 4021 is. Branches and tags are directory copies within the same tree (`svn copy`), which is a lightweight operation on the server but conceptually heavier than a movable pointer, and merge tracking has to be layered on top explicitly rather than falling out of a DAG. + +### Mercurial (distributed, comparable model) + +Mercurial (`hg`) is the closest cousin to Git: distributed, snapshot-based, with commits identified by content hash and a DAG of changesets. The main differences are more in default workflow and tooling philosophy than in underlying model. Mercurial favours a simpler, more restricted command set and historically discouraged history rewriting (no built-in rebase until an optional extension), whereas Git exposes rewriting commands (`rebase`, `commit --amend`, `filter-branch`) as first-class, and treats branches as far more disposable, cheap, short-lived pointers by convention. + +### Perforce (centralised, file-locking, large binary assets) + +Perforce (P4) is centralised like Subversion but built for a different problem: very large repositories, including large binary assets (game art, media), where full distributed clones of the whole history would be impractical. It uses a central server as the single source of truth, supports exclusive file locking (useful for binary files that can't be merged line-by-line), and organises work through "changelists" rather than a DAG of hash-identified commits. It scales well for huge monorepos with big binaries, a case where Git historically struggles without extensions such as Git LFS. + +### The underlying distinction + +The dividing line that matters most is centralised versus distributed, and content-hash-identified versus sequential-number-identified: + +- centralised systems (SVN, Perforce) need a server to define what "the current state" is, and history is a single line of numbered revisions +- distributed, hash-based systems (Git, Mercurial) let every clone hold the full history and compute the same hash for the same content independently, so two people can work offline and later reconcile via merge, with no central authority needed to agree on what came first + +## Putting It Together + +The reason this design holds together is that all four ideas are really one idea viewed from different angles: + +- content-addressing (hashes) makes objects immutable and easy to deduplicate +- the DAG is simply what you get when immutable commits point at their immutable parents +- a branch is a cheap, movable label on a point in that graph +- a merge is a new commit added to the DAG with two parents instead of one, created after resolving the differences between the histories those parents represent + +Everything else in Git -- rebasing, cherry-picking, `reflog`, tags -- is built from these same primitives: objects addressed by content, and pointers that move around a graph of them. diff --git a/content/guides/deep-dives/index.md b/content/guides/deep-dives/index.md new file mode 100644 index 0000000..bc3685f --- /dev/null +++ b/content/guides/deep-dives/index.md @@ -0,0 +1,23 @@ +--- +title: Deep dives +--- + +# Deep dives + +Long explanations of single technologies, for when a [primer](../../primers/index.md) is not enough. Each one goes underneath the topic: how it works, why it was designed that way, and where it catches people out. + +| Deep dive | Covers | +| --- | --- | +| [CBOR](cbor.md) | The major types with worked examples, canonical CBOR, tags, and how it compares with the alternatives | +| [CoAP](coap.md) | The Constrained Application Protocol: messages, methods, observe, block transfer and security | +| [LwM2M](lwm2m.md) | The roles, the object model, the four interfaces, payload encodings, bootstrapping, security and firmware update | +| [NB-IoT and cellular fundamentals](nb-iot-and-cellular.md) | How cellular IoT works: NB-IoT and LTE-M, power saving, coverage and the signal diagnostics a module reports | +| [Certificates and keys](certificates-and-keys.md) | Keys against certificates: confidentiality, public-key cryptography, integrity with hashes and MACs, and identity | +| [RTOS internals](rtos-internals.md) | Tasks, scheduling, synchronisation and how the common real-time kernels compare | +| [SysVars: a schema-driven system variable store](sysvars.md) | One shared, schema-defined data model for firmware state, generated into C | +| [MISRA C in depth](misra-c.md) | Its history, rule categories, deviations and tooling, and what changed in MISRA C:2023 and 2025 | +| [Static analysis](static-analysis.md) | The levels of analysis, soundness against completeness, the tool landscape, and where it fits among other quality practices | +| [Git internals](git-internals.md) | Objects, hashes, the commit graph, branches and how merges work underneath | +| [Docs, diagrams and architecture as code](docs-diagrams-and-architecture-as-code.md) | The tools and approaches for keeping documentation, diagrams and architecture models in Git | + +[[CAPTION:Table]] The deep dives. diff --git a/content/guides/deep-dives/lwm2m.md b/content/guides/deep-dives/lwm2m.md new file mode 100644 index 0000000..6e9f78d --- /dev/null +++ b/content/guides/deep-dives/lwm2m.md @@ -0,0 +1,236 @@ +--- +title: LwM2M +--- + +# LwM2M + +## Purpose and scope + +This document explains Lightweight M2M (LwM2M) at a mechanism level: the object model, the four interfaces, bootstrapping, and how firmware update fits into the protocol. It is the third of three companion documents -- *CoAP: A Deep Explainer* covers the transport LwM2M is built on, and *CBOR: A Deep Explainer* covers the payload encoding LwM2M uses for structured data -- and assumes both as read. Where a mechanism here is really a CoAP or CBOR mechanism wearing an LwM2M label, this document says so and points back rather than re-deriving it. It assumes general familiarity with embedded/firmware concepts but not prior exposure to device management protocols. + +## 1. What problem LwM2M solves + +A fleet of IoT devices in the field needs to be: registered so a server knows they exist, configured and reconfigured over their lifetime, monitored for health and connectivity, and updated with new firmware -- all over a link that may be high-latency, low-bandwidth, and only intermittently available (a device on NB-IoT PSM may only be reachable for seconds at a time). LwM2M (specified by OMA SpecWorks, currently at version 1.2) is a device management protocol built specifically for that environment, in contrast to protocols built for always-on, high-bandwidth links. + +| | LwM2M | TR-069 (CWMP) | Custom MQTT-based | +|---|---|---|---| +| Transport | CoAP/UDP (usually) | SOAP/HTTP | MQTT/TCP | +| Designed for | Constrained radio (cellular IoT, LPWAN) | Fixed-line CPE (routers, gateways) | Whatever the implementer chose | +| Data model | Standardised Object/Resource registry | Standardised but XML/SOAP-heavy | Ad hoc | +| Typical payload overhead | Very low (CoAP + CBOR/TLV) | High (XML) | Depends | +| Bootstrapping | Standardised | Standardised (ACS) | Ad hoc | + +LwM2M's contribution over a hand-rolled MQTT scheme is a standardised **Object/Resource model** -- so "battery level" or "firmware version" means the same thing, at the same path, on every conformant client and server, rather than being reinvented per project. + +## 2. Architecture: three roles + +```mermaid +flowchart LR + D[LwM2M Client on device] <-->|CoAP/DTLS| S[LwM2M Server] + D <-->|CoAP/DTLS, bootstrap only| B[Bootstrap Server] +``` + +- **LwM2M Client** -- runs on the device. Holds the Object/Resource data, responds to server operations, initiates registration. +- **LwM2M Server** -- the device management platform. Reads/writes resources, observes them, issues Execute operations, receives registration updates. +- **Bootstrap Server** -- a distinct, more privileged role used only to provision a device with its Server connection details and security credentials, before the device has ever spoken to its actual Server. A device may not need one on every connection (see section 6) but the role always exists conceptually. + +A single device typically has a relationship with one Bootstrap Server and one (or occasionally more) LwM2M Server; the security details of each relationship are held in the client's own Security object (Object ID 0), described in section 8. + +## 3. The object model + +LwM2M structures all device data into a strict four-level hierarchy: + +```mermaid +flowchart TB + A["Object (e.g. Device = Object 3)"] --> B["Object Instance (e.g. /3/0)"] + B --> C["Resource (e.g. Firmware Version = Resource 3)"] + C --> D["Resource Instance (only for multi-instance resources)"] +``` + +- **Object** -- a defined data structure with an Object ID, describing one area of device functionality (Device, Connectivity Monitoring, Firmware Update, and so on). Object IDs 0-10 and a growing standardised range are reserved by OMA SpecWorks; vendors may define custom objects starting from ID 10000 upwards for functionality the standard registry does not cover. +- **Object Instance** -- a concrete instantiation of an Object. Most Objects support multiple instances (e.g. multiple Connectivity Monitoring instances for a multi-bearer device); the instance number is the second path segment. +- **Resource** -- a single data point or executable action within an Object Instance, identified by a Resource ID defined in that Object's specification. +- **Resource Instance** -- only exists where a Resource itself is multi-instance (an array-valued resource); otherwise the Resource IS the value, with no further nesting. + +This gives every piece of data in the system a stable path, exactly analogous to a filesystem path or a URI: + +``` +/3/0/3 + | | | + | | +-- Resource 3 (Firmware Version) + | +----- Object Instance 0 + +-------- Object 3 (Device) +``` + +That path is, at the wire level, the CoAP Uri-Path option from the companion CoAP document -- an LwM2M Read of `/3/0/3` is a CoAP GET with Uri-Path segments `3`, `0`, `3`. + +### Standard Objects (illustrative, not exhaustive) + +| Object ID | Name | Purpose | +|---|---|---| +| 0 | LwM2M Security | Server URIs, security mode, keys/certificates per server relationship | +| 1 | LwM2M Server | Lifetime, binding mode, notification storing behaviour | +| 3 | Device | Manufacturer, model, firmware version, reboot/factory-reset Execute resources | +| 4 | Connectivity Monitoring | Signal strength, bearer type, cell ID | +| 5 | Firmware Update | Package delivery, update trigger, update state/result | +| 6 | Location | GPS coordinates, timestamp | +| 7 | Connectivity Statistics | Data volume sent/received | + +Vendors extending this registry for device-specific functionality (custom telemetry, application-specific configuration, and so on) follow the same four-level model on custom Object IDs -- the mechanism does not change, only which IDs are in use. + +## 4. The four interfaces + +LwM2M defines exactly four logical interfaces between Client and Server. Each is a distinct set of operations, though all of them are, underneath, CoAP request/response or Observe exchanges. + +```mermaid +flowchart TB + A[Bootstrap] --> E[Client provisioned with Server details + keys] + B[Registration] --> F[Server knows the Client exists and what Objects it supports] + C["Device Management and Service Enablement"] --> G[Server can Read/Write/Execute/Create/Delete resources] + D["Information Reporting"] --> H[Server observes resources, gets pushed notifications on change] +``` + +### Bootstrap + +Covered in detail in section 6. + +### Registration + +The Client initiates a **Register** operation (CoAP POST to `/rd`) once it has its Server connection details, sending its Endpoint Name, a requested **Lifetime**, its supported Binding mode (which transport bindings it will accept -- UDP, SMS, non-IP, etc), and an Object/Object Instance list describing what it supports. The Server replies with a Location path identifying this registration. + +Before the Lifetime expires, the Client must send an **Update** (a lighter-weight CoAP POST to the Location path from registration) to keep the registration alive; if it doesn't, the Server considers the Client's registration stale and treats it as offline. A clean disconnect uses **De-register**. + +```mermaid +sequenceDiagram + participant Client + participant Server + Client->>Server: Register (Endpoint, Lifetime, Objects) + Server-->>Client: 2.01 Created, Location=/rd/abc123 + Note over Client,Server: periodically, before Lifetime expires + Client->>Server: Update (Location=/rd/abc123) + Server-->>Client: 2.04 Changed +``` + +Lifetime is a direct trade-off against battery and airtime: a short Lifetime means the Server notices a dropped device quickly but the Client spends more radio time re-registering; a long Lifetime saves battery but means a genuinely dead device may look "registered" for longer than it has actually been reachable. + +### Device Management and Service Enablement + +This is the interface most people mean when they say "LwM2M operations" -- the Server acting on a specific resource path: + +| Operation | CoAP method | Meaning | +|---|---|---| +| Read | GET | Retrieve current value(s) | +| Write | PUT or POST | Set value(s) | +| Write-Attributes | PUT | Set Observe notification conditions (min/max period, thresholds) | +| Execute | POST | Trigger an action with no persistent value (e.g. Reboot) | +| Create | POST | Create a new Object Instance | +| Delete | DELETE | Remove an Object Instance | +| Discover | GET | Retrieve the Object/Resource structure supported | + +Each of these is literally the CoAP method named in the table, sent to the resource's Uri-Path, with the payload (where present) encoded in one of the content formats described in section 5. + +### Information Reporting + +This is CoAP's Observe option (see the CoAP companion document), used unmodified: the Server sends a Read with the Observe option set, and the Client keeps pushing new values as they change, governed by the Write-Attributes set on that resource (minimum period between notifications, maximum period forcing a notification even if unchanged, and value-change thresholds). This is how a Server monitors, say, signal strength or a sensor reading continuously without polling. + +## 5. Data formats (payload encodings) + +The Device Management operations above need a way to encode a resource's value (or several resources' values) into the CoAP payload. LwM2M supports several Content-Format options, negotiated the same way any CoAP Content-Format is: + +| Format | Use case | +|---|---| +| Plain Text | A single Resource, primitive type, human-readable | +| Opaque | A single Resource, raw binary (e.g. a firmware package) | +| TLV | Multiple Resources/Instances in one compact binary payload, LwM2M-specific encoding | +| SenML JSON | Multiple Resources, text-based, timestamped, human-readable | +| SenML CBOR | Multiple Resources, binary, timestamped, most compact | + +TLV was LwM2M's original compact binary format, defined before CBOR was standardised; SenML CBOR has since become the preferred compact format for new deployments because it reuses a general IETF standard (see the CBOR companion document) rather than an LwM2M-specific scheme, and because SenML's record structure natively carries a timestamp per value, useful for Information Reporting notifications that batch several readings together. + +Unlike SenML CBOR, TLV is an LwM2M-specific byte format rather than a general one, so it is worth showing its Type byte structure directly: + +```wavedrom +{reg: [ + {bits: 3, name: 'Length (if Length Type = 00)'}, + {bits: 2, name: 'Length Type'}, + {bits: 1, name: 'ID Length'}, + {bits: 2, name: 'Type of ID'} +], config: {bits: 8, lanes: 1}} +``` + +Reading this: the top 2 bits say what kind of thing this TLV entry is (Object Instance, Resource, multi-Resource, or Resource with a value); the next bit says whether the Identifier that follows is 8 or 16 bits; the next 2 bits say whether the Length that follows is encoded directly in the remaining 3 bits of this same byte, or needs its own following 8/16/24-bit field. A short Resource value (a battery percentage, say) can fit Type, Identifier, and Length into 2-3 bytes total; a large value (a firmware Opaque blob) spills its length into the following bytes rather than this one. + +A SenML CBOR payload is, at the byte level, exactly what the CBOR document describes: an array of maps, each map using short integer keys (by SenML convention, `0` for base name, `2` for value, `6` for time, and so on) rather than the human-readable string keys used in that document's worked examples -- the byte-level rules (major types, additional info, map encoding) are identical, only the key/field convention is LwM2M/SenML-specific. + +## 6. Bootstrapping + +A device fresh from manufacture has no Server relationship yet -- bootstrapping is how it gets one. LwM2M defines several bootstrap modes: + +- **Factory Bootstrap** -- Security and Server object instances are provisioned at manufacture time, baked into the device. No bootstrap exchange happens at all in the field. +- **Client-Initiated Bootstrap** -- the device, on first boot (or after a factory reset), contacts a pre-configured Bootstrap Server, which then Writes the Security and Server object instances the device needs, then instructs the device (via a Bootstrap-Finish Execute) to proceed to Registration with its real Server. +- **Server-Initiated Bootstrap** -- the Bootstrap Server, rather than waiting to be contacted, pushes bootstrap data to a device it already knows about, e.g. to rotate credentials or point it at a new Server. +- **Smartcard Bootstrap** -- bootstrap data is provisioned via SIM/smartcard rather than over the air, relevant where the SIM provisioning process already exists for network attach. + +```mermaid +sequenceDiagram + participant Client + participant Bootstrap as Bootstrap Server + participant Server as LwM2M Server + Client->>Bootstrap: Request Bootstrap + Bootstrap->>Client: Write Security object (Server 0 credentials) + Bootstrap->>Client: Write Server object (Lifetime, binding) + Bootstrap->>Client: Execute Bootstrap-Finish + Client->>Server: Register +``` + +The Bootstrap Server's own connection to the Client uses its own entry in the Security object (a distinct Object Instance from the one describing the eventual LwM2M Server), so the bootstrap channel can use a different security mode or credential set than the operational one it is provisioning. + +## 7. Security + +Object 0 (Security) holds, per Server relationship (one Object Instance per Server, including the Bootstrap Server itself), the Server URI, which of four security modes applies, and the corresponding keys: + +| Mode | Mechanism | +|---|---| +| 0 -- Pre-Shared Key (PSK) | DTLS PSK cipher suites; a shared secret per device | +| 1 -- Raw Public Key (RPK) | DTLS with a bare public key, no certificate chain | +| 2 -- Certificate | Full X.509 certificate chain, DTLS certificate-based handshake | +| 3 -- NoSec | No DTLS at all -- plain CoAP, generally only acceptable for lab/test | + +The certificate formats, chain validation, and key types behind modes 1 and 2 are covered in *Certificates and Keys: A Deep Explainer* and are not repeated here; from LwM2M's perspective, the Security object simply selects which DTLS configuration the underlying CoAP layer should use for that Server relationship, exactly as described in the CoAP companion document's security section. + +## 8. Firmware Update (Object 5) + +Firmware Update is worth walking through in detail because it exercises almost every mechanism covered above and in the two companion documents. + +Object 5 exposes (among others) a **Package** resource (Opaque, for pushing the firmware binary directly via Write), a **Package URI** resource (a URL the Client can pull the package from), an **Update** Execute resource (triggers the actual flash/apply step), and **State**/**Update Result** resources the Server can Read or Observe to track progress. + +### Push delivery + +```mermaid +sequenceDiagram + participant Server + participant Client + Server->>Client: Write /5/0/Package (firmware binary, Block-wise) + Note over Client: State -> Downloaded + Server->>Client: Execute /5/0/Update + Note over Client: applies firmware, reboots + Client->>Server: Register (post-reboot) + Server->>Client: Read /5/0/Update Result +``` + +Because a firmware image is far larger than a single UDP datagram, the Write of the Package resource uses CoAP Block-wise transfer (see the CoAP companion document) -- each block is its own CoAP exchange, individually retransmittable, which matters on a lossy NB-IoT link where re-sending an entire image because of one dropped block would be wasteful of both airtime and battery. + +### Pull delivery + +Alternatively, the Server Writes a URI to Package URI rather than the binary itself, and the Client fetches the package on its own schedule (over CoAP, or another protocol such as HTTP, depending on what the URI scheme and the Client's capabilities support) -- useful where the Client wants to control exactly when it spends the airtime, rather than the Server pushing at a time that may not suit the device's power/connectivity window. + +### State machine + +The State and Update Result resources track progress through a small state machine (roughly: Idle, Downloading, Downloaded, Updating), with Update Result recording the outcome (success or a specific failure reason) once an Execute /Update has run. A Server observing these resources (Information Reporting again) sees firmware rollout progress across a fleet without polling each device. + +## 9. Practical considerations on constrained links + +- **Payload size vs radio link.** Every LwM2M exchange is ultimately a CoAP exchange, so the CE-level/repetition trade-offs from the NB-IoT explainer apply directly -- a large Write or a firmware Block still costs proportionally more airtime and battery on a device in a poor-coverage CE level. +- **Lifetime vs battery.** A short Registration Lifetime keeps the Server's view of "online" accurate but costs more periodic Update traffic; this is a deployment-level tuning decision, not a protocol default to leave unexamined. +- **Queue Mode binding.** For devices that sleep between radio windows (PSM), the Server cannot simply send a request whenever it likes -- Queue Mode binding means the Server queues operations and only delivers them when the Client next wakes and re-registers/updates, rather than assuming the Client is always reachable. +- **Custom Objects.** Anything the standard registry does not cover -- device-specific configuration, application data -- follows the same four-level Object/Instance/Resource/Resource-Instance model on a custom Object ID; the mechanism never changes, only which IDs and Resource definitions are in use for a given deployment. diff --git a/content/guides/deep-dives/misra-c.md b/content/guides/deep-dives/misra-c.md new file mode 100644 index 0000000..7fa5fc3 --- /dev/null +++ b/content/guides/deep-dives/misra-c.md @@ -0,0 +1,258 @@ +--- +title: MISRA C in depth +--- + +# MISRA C in depth + +## 1. What MISRA C Is and Why It Exists + +MISRA C is a set of software development guidelines for the C language, produced by the MISRA Consortium (Motor Industry Software Reliability Association). The goal is not "good style" in the abstract -- it is removing constructs from C that are ambiguous, undefined, unspecified, or implementation-defined, so that the same source code behaves identically across compilers, targets, and optimisation levels. + +C was never designed for safety-critical use. It trusts the programmer completely: pointer arithmetic, implicit conversions, undefined behaviour on overflow, and a preprocessor that can rewrite the language itself. MISRA C constrains this trust. It defines a *subset* of C that is safe to reason about statically, both by a human reviewer and by a static analysis tool. + +> **Note:** MISRA C is a coding guideline, not a language variant. Code that violates MISRA C still compiles and runs -- MISRA C is about eliminating behaviours that are technically legal C but practically dangerous. + +> **Important:** The MISRA C document itself is copyrighted and commercially licensed, not freely reproducible. A rule number and one-line summary from a static analyser's report is not the guideline -- the actual document contains the rationale, the exact wording of the requirement, compliant/non-compliant code examples, and cross-references to related rules. A team relying purely on a shared PDF, a colleague's memory, or tool tooltips will misapply rules and mis-scope deviations. Every developer who writes code that has to be MISRA compliant should have their own licensed copy (or paid-for organisational access) of the current edition, both to understand *why* a rule exists -- essential for writing sound deviation rationale -- and because MISRA's own licensing terms require it for compliant use in a professional context. + +## 2. History and Origins + +MISRA originated in the UK automotive sector. Two of its founding members, Ford and Land Rover, discovered independently that they were each developing in-house coding guidelines for embedded C used in engine control units. MISRA C originated when two of the original members of the MISRA project, Ford and Land Rover, became aware they were working separately on coding guidelines for embedded C. This led to a collaborative, cross-industry standard rather than two competing internal ones. + +Timeline of major editions: + +```mermaid +timeline + title MISRA C Evolution + 1998 : MISRA-C:1998 (First edition) + 2004 : MISRA-C:2004 (Second edition) + 2012 : MISRA C:2012 (Third edition) : Directives introduced + 2016 : Amendment 1 (AMD1) : Security guidelines added + 2019 : MISRA C:2012 3rd ed, 1st revision + 2020 : Amendment 2 (AMD2) : C11 core support + 2022 : Amendment 3 (AMD3) : C11/C18 features + 2023 : MISRA C:2012 Amendment 4 (AMD4) : Multithreading and atomics + 2023 : MISRA C:2023 : Consolidated edition + 2025 : MISRA C:2025 : Incremental refinement +``` + +Key structural change: for the first two editions (1998, 2004), every guideline was a "Rule". With the publication of MISRA C:2012 a new category of Guideline was introduced -- the Directive, whose compliance is more open to interpretation, or relates to process or procedural matters. This Rule/Directive split still exists today. + +MISRA C has since spread well beyond its automotive origins. Although originally specifically targeted at the automotive industry, MISRA C has evolved as a widely accepted model for best practices in sectors including automotive, aerospace, telecom, medical devices, defence, railway, and others. It is also referenced by other standards -- for example, ISO 26262-6:2018 Part 6 cites MISRA C:2012, and the AUTOSAR General Software Specification requires that BSW modules written in C shall conform to MISRA C. + +## 3. Why Use It: Pros and Cons + +### Pros + +- **Undefined/unspecified behaviour elimination.** Most rules target specific clauses of the ISO C standard that are undefined, unspecified, or implementation-defined -- the exact bugs that are invisible in testing but appear on a different compiler, target, or optimisation flag. +- **Portability.** A MISRA-compliant module behaves the same on GCC, IAR, or Keil, which matters when you have firmware families sharing a codebase across silicon vendors. +- **Auditable process, not just style.** MISRA Compliance:2020 defines Guideline Enforcement Plans (GEP) and Deviation Records, giving you a documented trail for functional safety assessors (ISO 26262, IEC 61508, DO-178C-adjacent audits). +- **Tooling maturity.** Every major static analyser (LDRA, Parasoft, Perforce QAC/Klocwork, Polyspace, PC-lint/PC-lint Plus, Cppcheck premium) has first-class MISRA support, so adoption doesn't require inventing your own checker. +- **Industry lock-in effect.** If you supply automotive, aerospace, medical, or rail customers, MISRA compliance is frequently a contractual gate, not a nice-to-have. + +### Cons + +- **False sense of safety.** MISRA compliance proves the *absence of certain classes of undefined behaviour*. It says nothing about whether the logic is correct, whether the requirements are right, or whether concurrency design is sound. +- **Volume and friction.** 200+ guidelines is a lot to internalise. New team members hit a wall of unfamiliar restrictions (no recursion, no dynamic allocation in some profiles, restricted pointer arithmetic) that can feel arbitrary until the rationale is understood. +- **Legacy codebase cost.** Retrofitting MISRA onto an existing non-compliant codebase can produce thousands of violations, most low-risk. This is exactly why baselining and risk-based prioritisation exist as a stated best practice -- teams "set baselines" and "prioritize violations based on risk" rather than fixing everything at once. +- **Tooling divergence.** Not every tool implements every rule identically, particularly the more "undecidable" ones requiring manual review (for example Directive 1.1 on implementation-defined behaviour documentation). Two compliant tool reports for the same codebase can disagree. +- **Deviation overhead.** Legitimate deviations (raw pointer arithmetic in a memory-mapped register driver, for instance) need formal justification, review, and sign-off, which is process overhead some teams resent. + +## 4. Alternatives and How They Relate + +MISRA C is not the only C/C++ coding standard, and it is common to combine it with others rather than choose one exclusively. + +| Standard | Focus | Relationship to MISRA C | +|---|---|---| +| CERT C | Security vulnerabilities (buffer overflow, integer overflow, format string attacks) | Complementary -- CERT C is security-first, MISRA C is safety/portability-first. Significant overlap but different emphasis. | +| CWE/SANS Top 25 | Catalogue of common weakness types, not a style guide | Used to cross-map static analysis findings; not a coding standard to write against directly. | +| JPL C Coding Standard | NASA/JPL, derived partly from MISRA, adds "Power of Ten" rules (no recursion, no dynamic memory after init, bounded loops) | Stricter subset in places, aimed at flight software rather than general embedded/automotive. | +| Barr Group Embedded C Coding Standard | Practical embedded C style guide, less formal than MISRA | Lighter-weight, no formal deviation/compliance framework -- more a house style than an audit standard. | +| AUTOSAR C++14 | Automotive C++ guidelines pre-2023 | Largely superseded by MISRA C++:2023 for coding rules, though AUTOSAR still governs broader architecture/toolchain concerns. | +| MISRA C++:2023 | The C++ sibling standard | Same governing body, developed jointly with AUTOSAR, targets C++17. | + +> **Important:** MISRA C and CERT C are frequently applied together in practice, particularly where a project needs both functional safety sign-off and security hardening (for example, connected embedded devices like NB-IoT gas meters, where an attacker on the network is a realistic threat model alongside classic safety hazards). + +## 5. Rule Categories and Classification + +MISRA guidelines are classified along several independent axes, and a given rule carries values on all of them simultaneously. + +### 5.1 Guideline type + +- **Directive** -- guidance where compliance requires judgement, process, or information outside the source text itself (for example, "document all implementation-defined behaviour relied upon"). +- **Rule** -- guidance that is, in principle, fully decidable from the source code alone. + +### 5.2 Decidability + +- **Decidable** -- a tool can determine compliance with certainty from the translation unit(s) in scope. +- **Undecidable** -- compliance cannot be fully automated; manual review is required even with best-in-class tooling. + +### 5.3 Enforcement category (severity) + +- **Mandatory** -- no deviation permitted, ever. Violating a Mandatory rule means the code is not MISRA compliant, full stop. +- **Required** -- compliance is expected; deviations are permitted but must be formally recorded and justified. +- **Advisory** -- best practice, non-compliance does not need a formal deviation record, though good practice is to still note it. + +### 5.4 Scope + +- **Single translation unit** -- checkable by looking at one `.c` file (with its includes) in isolation. +- **System** -- requires whole-programme/link-time analysis (for example, checking that a function is only ever called with values in a certain range across the entire codebase). + +```mermaid +flowchart TD + A[MISRA Guideline] --> B{Directive or Rule?} + B --> C[Directive] + B --> D[Rule] + A --> E{Decidable?} + E --> F[Decidable] + E --> G[Undecidable] + A --> H{Enforcement} + H --> I[Mandatory] + H --> J[Required] + H --> K[Advisory] + A --> L{Scope} + L --> M[Single Translation Unit] + L --> N[System] +``` + +## 6. Deviations + +A deviation is the formal mechanism for permitting a documented violation of a Required or Advisory guideline (Mandatory guidelines cannot be deviated from at all). + +The framework, defined in MISRA Compliance:2020, distinguishes two related artefacts: + +- **Deviation Record** -- a project-specific, instance-level justification: which guideline, which line(s) of code, why the violation is safe in this context, and who approved it. +- **Deviation Permit** -- a pre-approved, reusable justification for a recurring pattern (for example, "pointer arithmetic on a memory-mapped peripheral register block is permitted under Rule 18.4 across this driver layer"), so every instance doesn't need a fresh record. + +A typical deviation record captures: + +1. The guideline number and short description +1. The specific file/function/line(s) affected +1. The technical rationale for why the violation does not introduce the risk the rule exists to prevent +1. Any compensating control (additional review, test coverage, runtime check) +1. Approval sign-off (name, role, date) + +> **Warning:** A deviation record that just says "required for performance" or "legacy code" without a technical safety argument is not a deviation -- it is an unmanaged violation with paperwork attached. Auditors will reject it. + +Guideline Enforcement Plans (GEP) and Guideline Compliance Summaries (GCS) sit above individual deviations: the GEP states, per guideline, how it will be enforced (tool, manual review, or a combination) and to what confidence; the GCS is the final report showing compliance status guideline-by-guideline, including deviation counts. + +## 7. Tooling + +Static analysis is the practical enforcement mechanism for the majority of rules, with manual review reserved for undecidable guidelines. + +Commonly used MISRA-capable tools: + +- **LDRA tool suite** -- long-standing MISRA working group contributor, same-day support for new releases +- **Parasoft C/C++test** -- claims full guideline coverage including MISRA C++:2023 +- **Perforce QAC / Klocwork** -- widely used in automotive supply chains +- **Polyspace (MathWorks)** -- pairs static analysis with formal/abstract interpretation, useful for provable absence of runtime errors, not just MISRA text-matching +- **PC-lint Plus** -- lighter-weight, common in smaller embedded teams +- **Cppcheck (premium/MISRA add-on)** -- open-source core with a commercial MISRA rule add-on +- **IAR C-STAT** -- static analysis bundled directly into IAR Embedded Workbench, useful where the toolchain is already IAR-based since it needs no separate build integration + +Practical tooling guidance that recurs across vendor material: + +1. **Know your version.** Rule numbers are not stable across editions -- a tool flagging "Rule 21.2" against MISRA C:2023 will not necessarily mean the same thing as the equivalent numbered rule elsewhere, particularly given MISRA C:2025 permanently retiring renumbered rule identifiers rather than reusing them. +1. **Continuous checking beats periodic sweeps.** Catching a violation at commit time is cheaper than during a pre-release compliance sweep across a mature codebase. +1. **Baseline legacy code.** Suppress/accept existing violations as a known baseline, then gate new code strictly, rather than blocking all delivery until a legacy backlog is cleared. +1. **Risk-weight the backlog.** Not all violations carry equal risk; triage by guideline severity (Mandatory > Required > Advisory) and by the safety criticality of the affected module. +1. **Fix tool version and configuration.** Because some guidelines are undecidable or tool-interpretation-dependent, the specific tool version used for compliance claims should be fixed and documented, not left to float. + +> **Note:** MISRA explicitly treats AI-generated code the same as handwritten code for compliance purposes -- there is no exemption or separate rule set for machine-authored source. + +### 7.1 Enforce compliance at code review, not after the fact + +A static analysis report that nobody is required to act on is decoration, not compliance. MISRA compliance only means something operationally if it is a gate, not a report: + +- New/changed code should be run through the static analyser as part of CI, and the pull request should fail (or be flagged for explicit override) on any new Mandatory or Required violation. +- Reviewers should treat a new, unexplained violation the same way they'd treat a failing unit test -- it blocks merge until it's fixed or a deviation is raised and approved. +- Deviations should be proposed and reviewed *in* the pull request (linking to the Deviation Record or Permit), not retrofitted weeks later during a separate "compliance sweep". +- Baseline violations in legacy code are tracked separately and don't block unrelated changes, but any line a PR *touches* is fair game for enforcement -- this is how legacy debt gets paid down incrementally rather than all at once or never. + +Checking compliance only at a release milestone turns MISRA into an audit exercise that generates a large, late, demoralising backlog. Checking it at review time turns it into a habit. + +### 7.2 Why tools cannot check everything + +Not every guideline is mechanically decidable from source text alone, even with the best commercial analyser. A few guidelines that require human judgement no tool can fully substitute for: + +- **Directive 1.1** -- that all implementation-defined behaviour relied upon is documented and understood. A tool cannot know what the developer understands, only what's written down. +- **Directive 4.14** -- that values received from external sources are validated before use. A tool can confirm *some* check exists on a variable, but not whether it's the *correct* check for that value's valid range, or whether it covers every path that consumes the value. +- **Directive 4.1** -- that run-time failures are minimised, covering things like resource exhaustion and hardware fault handling. Whether a given design adequately "minimises" failure is a judgement call, not a syntactic property. + +This is exactly why Directives (as opposed to Rules) exist as a distinct category, and why a Guideline Enforcement Plan has to state, per guideline, whether it's enforced by tool, by manual review, or both. + +## 8. MISRA C:2012 to MISRA C:2023: What Changed + +MISRA C:2012 was extended over a decade through four amendments before being folded into a single consolidated edition. + +- **AMD1 (2016)** -- added new security-focused guidelines, bringing the total to 156 rules and 17 directives (173 guidelines). +- **AMD2 (2020)** -- added two new rules covering C11 core functionality, for 175 guidelines total. +- **AMD3 (2022)** -- added 24 new rules and 1 new directive covering C11/C18 features previously blocked outright by Rule 1.4, bringing the total to 200 guidelines. +- **AMD4 (March 2023)** -- added Rules and Directives for multithreading and atomic types, plus clarifications aligning guidance with how developers actually use the language. + +In May 2023 MISRA published MISRA C:2023 (MISRA C Third edition, Second revision), which incorporates Amendments 2 to 4 and Technical Corrigendum 2, and adds support for C11 and C17 language features. In other words, MISRA C:2023 is not a from-scratch rewrite -- it is the four amendments plus corrigenda, consolidated into one authoritative document so teams no longer need to track a base edition plus a stack of separate amendment PDFs. + +## 9. MISRA C:2023 to MISRA C:2025: What Changed + +MISRA C:2025 was released two years after MISRA C:2023 and, although a less substantive update than its predecessor, built on the efficiency of MISRA C:2023 with policy changes, enhancements, and new guidelines. It was announced at the Embedded World event in Nuremberg, Germany, in March 2025. + +Headline numbers: MISRA C:2023 had 221 guidelines; MISRA C:2025 contains 224 guidelines in total, with 4 new rules introduced, 2 rules removed, 1 rule disapplied by default, 3 rules renumbered, and 13 rules modified. + +The substantive changes: + +### New rules + +- Four new rules address tentative definitions in headers (8.18), external declarations inside source files (8.19), implicit pointer-to-NULL comparisons (11.11), and reading inactive union members (19.3). +- Rule 19.3 in particular loosens a previously blanket prohibition: it eases the use of unions by separating consistent use of one variant member from the stricter, genuinely unsafe case of changing the active member and reading a different one, with regular "safe" reads now advisory rather than forbidden outright. + +### Retired/relaxed rules + +- The long-standing single-point-of-exit rule (historically Rule 15.5, restricting multiple return statements per function) is now **disapplied**. MISRA no longer expects compliance with it, although other safety standards such as IEC 61508 and ISO 26262 may still require similar practices in specific project contexts. +- For the first time, the release introduces deleted and disapplied rules as distinct categories, and rule numbers that are deleted will not be reused in future versions -- so a tool report referencing an old rule number can never be ambiguous about which edition's rule it meant. + +### Extended/clarified rules + +- Pointer conversion rules (11.3, 11.4, 11.6) now account for the intptr_t family, giving more headroom for working with integers and the integer representation of addresses -- relevant for anything doing pointer tagging or low-level memory-mapped access, which was previously blocked under a blanket address-manipulation prohibition. +- Directive 4.6, which requires size- and signedness-indicating typedefs in place of basic numerical types, has had floating point types removed from its scope, because their characteristics extend beyond just size. +- Example code that previously included `` purely to obtain the definition of `NULL` has been updated to use ``, which is universally available across implementations. +- Developers now have increased flexibility to terminate switch clauses (for example with return or continue), reducing audit friction. + +### Practical effect for legacy compliance work + +By removing obsolete rules, the standard reduces compliance overhead that previously bogged down development teams -- for example, certain restrictions on declaration placement have been relaxed when they don't affect safety, making refactoring simpler. This reorganisation effort is explicitly framed as laying groundwork for the next version rather than being an end state in itself. + +> **Note:** MISRA C:2025 does not drop C90/C99 support. It continues to cover C90, C99, and C11/C18, so a project pinned to an older C standard is not forced onto newer language features by upgrading the guideline edition. + +## 10. Key Rules Worth Knowing + +A non-exhaustive set of rules that come up constantly in practice, useful as a mental checklist rather than a substitute for the full text. Guidelines that are **new in MISRA C:2025** are marked explicitly; everything else marked "(Required)"/"(Mandatory)" with no version tag has been carried forward from MISRA C:2023 (or earlier) unchanged in substance. + +- **Rule 1.3 (Required)** -- no undefined or critical unspecified behaviour anywhere in the programme. *Carried forward from MISRA C:2023.* +- **Rule 2.2 (Required)** -- no dead code with no effect on programme behaviour. *Carried forward from MISRA C:2023.* +- **Rule 8.4/8.5/8.6 family** -- consistent, single declarations for objects and functions across translation units. *Carried forward from MISRA C:2023.* +- 🆕 **Rule 8.18 -- NEW IN MISRA C:2025** -- restricts tentative definitions in headers, closing ambiguity about whether a declaration is acting as a definition. +- 🆕 **Rule 8.19 -- NEW IN MISRA C:2025** -- restricts external declarations inside source files, for the same reason as 8.18. +- **Rule 9.1 (Mandatory)** -- no read of an object with automatic storage duration before it has been initialised. *Carried forward from MISRA C:2023.* +- **Rule 10.x series** -- the "essential type model": implicit conversions between essentially different types (char, signed, unsigned, float, bool) are restricted, catching a large class of silent truncation/sign-extension bugs. *Carried forward from MISRA C:2023.* +- **Rule 11.3/11.4/11.6** -- restrictions on casting between pointer types and between pointers and integers. *Carried forward from MISRA C:2023, but the scope was extended in MISRA C:2025 to accommodate the intptr_t family.* +- 🆕 **Rule 11.11 -- NEW IN MISRA C:2025** -- no implicit comparison of a pointer to NULL; comparisons must be explicit. +- **Rule 14.4 (Required)** -- the controlling expression of an if/iteration statement shall have essentially Boolean type. *Carried forward from MISRA C:2023.* +- ⚠️ **Rule 15.5 -- DISAPPLIED IN MISRA C:2025** -- formerly restricted multiple return points per function (the "single point of exit" rule); MISRA no longer expects compliance with it at all, though downstream safety standards (IEC 61508, ISO 26262) may still require similar practice on specific projects. +- **Rule 17.2 (Required)** -- no recursion, direct or indirect. *Carried forward from MISRA C:2023.* +- **Rule 18.1/18.4 family** -- restrictions on pointer arithmetic, particularly arithmetic that could run off the end of an array. *Carried forward from MISRA C:2023.* +- 🆕 **Rule 19.3 -- NEW IN MISRA C:2025** -- restricts reading a union member other than the one last written, but with an exception carved out for consistent, safe single-member use (this is a loosening as well as a new rule -- see Section 9). +- **Rule 21.x series** -- restrictions or outright bans on standard library functions with well-known undefined-behaviour footguns (`gets`, unchecked `sprintf`, certain `` allocation patterns). *Carried forward from MISRA C:2023.* +- **Directive 1.1 (Required)** -- any implementation-defined behaviour the programme's output depends on must be documented and understood. *Carried forward from MISRA C:2023.* This is the canonical example of an undecidable Directive requiring manual review rather than tooling alone (see Section 7.2 for a worked example of an undecidable guideline). +- **Directive 4.6** -- use of sized/signedness-explicit typedefs (`uint16_t`, `int32_t`) in place of `int`, `short`, `long`. *Carried forward from MISRA C:2023, but narrowed in MISRA C:2025 to exclude floating point types from its scope.* + +> **Note:** MISRA C:2025 also renumbered three existing guidelines into more appropriate sections and deleted two rules outright (folding their intent into other guidelines). Deleted/renumbered rule numbers are permanently retired and will not be reused, so an old rule number in a legacy tool report always refers unambiguously to the pre-2025 guideline. + +## 11. Summary + +- MISRA C exists to remove undefined, unspecified, and implementation-defined behaviour from C, not to enforce a house style. +- It splits guidelines into Directives (judgement/process) and Rules (code-derivable), each further tagged Mandatory/Required/Advisory and Decidable/Undecidable. +- MISRA C:2023 consolidated a decade of amendments (AMD1 through AMD4) into one document; MISRA C:2025 is a deliberately incremental refinement adding 4 rules, removing 2, disapplying 1 (single point of exit), and clarifying roughly a dozen more. +- Deviations are the formal, auditable mechanism for permitted violations of Required/Advisory rules; Mandatory rules admit no deviation. +- Compliance is a tooling-plus-process exercise: static analysis handles decidable rules, manual review and documentation (Guideline Enforcement Plans, Deviation Records) handle the rest -- and some guidelines (Directive 4.14 being the clearest example) can never be fully mechanised, because a tool can confirm a check exists without knowing whether it's the *right* check. +- Compliance only works as a gate if it's enforced at code review/CI time on every change, not as a one-off pre-release sweep against a growing backlog. +- The guideline text itself is licensed, not freely reproducible -- every developer writing MISRA-relevant code needs their own copy of the current edition, not just a tool's one-line rule summaries. +- MISRA C is commonly paired with CERT C for security, and sits alongside stricter derivatives like the JPL Power of Ten rules for the most safety-critical domains. diff --git a/content/guides/deep-dives/nb-iot-and-cellular.md b/content/guides/deep-dives/nb-iot-and-cellular.md new file mode 100644 index 0000000..16eb6cf --- /dev/null +++ b/content/guides/deep-dives/nb-iot-and-cellular.md @@ -0,0 +1,377 @@ +--- +title: NB-IoT and cellular fundamentals +--- + +# NB-IoT and cellular fundamentals + +## Purpose and scope + +This document explains how cellular LPWAN technologies work at a mechanism level, and what it actually takes to get a device from a radio design onto a live network. It starts with the physical layer, moves through the technology choices and their trade-offs, and finishes with the practical layer most explainers skip: SIM provisioning, APN configuration, certification, and spectrum. It assumes familiarity with general embedded/firmware concepts but not prior cellular radio knowledge. + +## 1. Where NB-IoT sits in the LPWAN landscape + +NB-IoT (Narrowband IoT) is a 3GPP-standardised Low Power Wide Area Network (LPWAN) technology, first specified in **Release 13** (2016) and extended through subsequent releases. It is one of three cellular-based LPWAN variants: + +| Technology | Bandwidth | Typical data rate | Mobility | Voice | Typical use | +|---|---|---|---|---|---| +| NB-IoT | 180 kHz | ~20-250 kbit/s | Stationary/low mobility | No | Metering, sensors | +| LTE-M (Cat-M1) | 1.4 MHz | ~1 Mbit/s | Full mobility, handover | Optional (VoLTE) | Asset tracking, wearables | +| Sigfox/LoRaWAN | Non-3GPP, unlicensed spectrum | Very low (<1 kbit/s effective) | Stationary | No | Ultra-low-power sensors | + +NB-IoT trades throughput and mobility support for **deep coverage penetration** (it can close a link at roughly 20 dB better Maximum Coupling Loss than GPRS, which matters for meters in basements, metre boxes, or dense masonry) and **very low modem power draw**, both of which are the deciding factors for gas metering. + +> **Note:** NB-IoT does not support handover between cells while in an active data session. A device that moves between cells will drop back to idle and perform cell reselection, not a seamless handover. This is irrelevant for a fixed metre, but worth remembering if debugging odd reconnection behaviour on the network side. + +## 2. NB-IoT deployment modes + +NB-IoT reuses existing LTE spectrum in one of three ways. The network operator chooses this, not the device: + +- **Standalone** -- deployed in previously unused (often refarmed GSM) spectrum, using a full 200 kHz channel. No LTE carrier alongside it. +- **Guard-band** -- deployed in the unused guard-band spectrum between two LTE carriers. +- **In-band** -- deployed within an existing LTE carrier's resource blocks, coexisting with normal LTE traffic on a reserved Physical Resource Block (PRB). + +The deployment mode is transparent to the application layer but affects link budget and interference, which is why the same hardware can show materially different signal quality across UK network operators. + +## 3. NB-IoT physical layer basics + +### Channel structure + +NB-IoT occupies a single 180 kHz channel (one LTE PRB), split into: + +- **Downlink**: OFDMA, 15 kHz subcarrier spacing, 12 subcarriers -- structurally identical to one LTE PRB. +- **Uplink**: SC-FDMA, either 15 kHz spacing (multi-tone) or 3.75 kHz spacing (single-tone, for extreme coverage cases). + +Modulation is deliberately simple -- QPSK only, no adaptive higher-order modulation -- because the design goal is coverage robustness, not spectral efficiency. + +### Repetition and coverage + +The core mechanism that gives NB-IoT its coverage advantage is **repetition**. Rather than transmitting a message once at high power, the modem retransmits the same data block up to 2048 times (downlink) or 128 times (uplink) at low power, and the receiver combines the repeated copies to recover a signal that would otherwise be below the noise floor. This is a direct power/time trade: better coverage costs more airtime and more battery, which is why CE level matters so much for battery life modelling and, later in this document, for latency. + +```mermaid +flowchart LR + A[Data block] --> B[Repeat N times] + B --> C[Transmit at low power] + C --> D[Receiver combines repeats] + D --> E[Recovered signal above noise floor] +``` + +## 4. NB-IoT Coverage Enhancement (CE) levels + +The network classifies each device into a **CE level** based on measured downlink signal quality, and this determines how many repetitions it will use: + +| CE Level | Typical use case | Repetition behaviour | +|---|---|---| +| CE0 | Normal coverage | Minimal/no repetition | +| CE1 | Extended coverage | Moderate repetition | +| CE2 | Extreme coverage (deep indoor, basements) | Maximum repetition | + +A metre in a cellar or dense-brick metre box will commonly sit at CE1 or CE2. This is diagnostic gold: if a device sits persistently at CE2, that is the underlying reason for slower registration and higher battery drain, not a fault condition in itself. + +## 5. Signal quality parameters + +These parameters are defined once here because the underlying radio measurements are identical across NB-IoT and LTE-M; only the typical values and how actively the network uses them differ, which is called out where relevant in Section 8. You will see them reported by modem AT commands (e.g. u-blox `AT+NUESTATS`, `AT+CESQ`) or network-side diagnostics. + +### RSRP (Reference Signal Received Power) + +The average power of the LTE/NB-IoT reference signal across resource elements, measured in dBm. This is the primary indicator of raw signal strength -- how strong the signal is, independent of interference. + +| RSRP (dBm) | Interpretation | +|---|---| +| > -80 | Excellent | +| -80 to -90 | Good | +| -90 to -100 | Fair | +| -100 to -110 | Poor (typical CE1 territory) | +| < -110 | Very poor (typical CE2 territory) | + +### RSRQ (Reference Signal Received Quality) + +A ratio of RSRP to total received power (RSSI), adjusted for the number of resource blocks, expressed in dB. RSRQ captures **interference and congestion**, not just strength -- a device can have strong RSRP but poor RSRQ if the cell is congested or there is strong co-channel interference. Typical usable range is roughly -3 dB (excellent) to -20 dB (very poor). + +### RSSI (Received Signal Strength Indicator) + +The total wideband received power, including signal, interference and noise, in dBm. RSSI alone is of limited diagnostic value because it doesn't distinguish wanted signal from noise -- it's mainly used as an input to derive RSRQ. + +### SNR / SINR (Signal-to-Noise / Signal-to-Interference-plus-Noise Ratio) + +Expressed in dB, this is the ratio of wanted signal power to the combined noise and interference floor. It is the parameter that most directly predicts whether a given modulation and coding scheme will decode successfully. Low SNR with high RSRP is a classic sign of interference (e.g. in-band deployment sharing a carrier with heavy LTE traffic) rather than a coverage problem, and points you towards a different fix (operator/channel investigation) than low RSRP does (physical placement). + +> **Important:** RSRP tells you the signal is *there*. SNR tells you whether it's *usable*. A troubleshooting flow that only looks at RSRP will misdiagnose interference-limited sites as coverage-limited ones. + +### CQI (Channel Quality Indicator) + +A 4-bit index (0-15) the device reports to the network summarising the highest modulation/coding scheme it can reliably decode, derived internally from SINR. On NB-IoT this is of limited relevance, since modulation is fixed at QPSK and the network manages link quality mainly via repetition count rather than adaptive modulation. On LTE-M, by contrast, CQI is actively used to drive adaptive modulation and coding, since LTE-M supports higher-order schemes -- see Section 8. + +### ECL (Extended Coverage Level) + +The u-blox/vendor-facing name for the device's own view of its CE level (0, 1 or 2), reported alongside RSRP/SNR in `AT+NUESTATS`. Functionally the same concept as CE level above, just reported at the device rather than inferred network-side. + +## 6. Power-saving mechanisms + +Battery life for a 15-year gas metre deployment is dominated by two things: how often the modem wakes to transmit, and how long it stays reachable for incoming pages after each transmission. NB-IoT and LTE-M define two complementary mechanisms to minimise the latter, and these same mechanisms are usually the real constraint on end-to-end responsiveness discussed in Section 11. + +### RRC states + +```mermaid +stateDiagram-v2 + [*] --> RRC_Idle + RRC_Idle --> RRC_Connected: Data to send/receive + RRC_Connected --> RRC_Idle: Release (inactivity timer) + RRC_Idle --> PSM: T3324 (active timer) expires + PSM --> RRC_Connected: Periodic TAU / mobile-originated data +``` + +- **RRC Connected**: active data transfer, highest power draw. +- **RRC Idle**: registered, not transmitting, but still monitoring paging channel at normal DRX intervals (typically ~1.28-2.56 s cycle) -- moderate power draw. +- **PSM (Power Saving Mode)**: the modem is unreachable and effectively powered down at the radio level, drawing near-zero current, while remaining registered on the network without needing to re-attach. + +### PSM (Power Saving Mode) + +PSM is negotiated with the network via two timers exchanged at attach/TAU (Tracking Area Update): + +- **T3324 (Active Timer)**: how long the device remains reachable (in RRC Idle, monitoring paging) after finishing a data transaction, before entering PSM. +- **T3412 (Periodic TAU timer, extended in NB-IoT as T3412-ext)**: the maximum interval before the device must wake and re-register with the network, regardless of whether it has data to send. This can be extended to hours or days, which is the mechanism that permits multi-year battery life. + +While in PSM the device is **not reachable for mobile-terminated (network-initiated) traffic at all** -- if the network needs to reach the device, it must wait until the device wakes on its own schedule. This is the key architectural constraint PSM imposes on any mobile-terminated command design. + +```mermaid +sequenceDiagram + participant Device + participant Network + Device->>Network: Data transfer (RRC Connected) + Network-->>Device: Ack, release to Idle + Note over Device: T3324 running (reachable, DRX paging) + Device->>Device: T3324 expires + Note over Device: PSM (unreachable, minimal current) + Device->>Device: T3412 expires + Device->>Network: Wake, periodic TAU +``` + +### eDRX (Extended Discontinuous Reception) + +eDRX addresses the middle ground: it extends the *paging cycle* while the device remains in RRC Idle (reachable), rather than going fully unreachable as in PSM. Instead of checking for pages every ~1.28-2.56 s, the device negotiates a much longer eDRX cycle (up to ~3 hours for NB-IoT, up to ~44 minutes for LTE-M in idle mode, longer again for LTE-M's connected-mode eDRX variant), waking briefly at each cycle to check a Paging Time Window (PTW) for incoming pages, then sleeping again. + +| | PSM | eDRX | +|---|---|---| +| Reachable for mobile-terminated traffic | No | Yes, at each eDRX wake window | +| Power draw | Lowest | Low, but higher than PSM | +| Wake interval | Only own schedule (T3412) or MO data | Configurable eDRX cycle (device negotiates) | +| Typical use | Devices that only report periodically (metre reads) | Devices needing occasional network-initiated contact without full PSM latency | + +> **Note:** PSM and eDRX are not mutually exclusive; a device can use eDRX for the RRC Idle period before it drops into PSM. Network support for both, and the actually-granted timer values, depend on the operator -- the device requests values, the network can grant shorter ones, so firmware should always read back the granted timers rather than assuming the requested values were accepted. + +## 7. NB-IoT cell selection and mobility + +NB-IoT does not support handover. Instead, a device performs **cell (re)selection**: it camps on the strongest suitable cell based on RSRP/RSRQ thresholds, and only re-evaluates this when idle (not mid-transaction). For a fixed metre this happens rarely after initial deployment, but is worth checking if a device is unexpectedly bouncing between two similarly-sited cells (a "ping-ponging" pattern visible as repeated re-registration in network logs), which usually indicates the reselection hysteresis parameters are too tight for a borderline signal location rather than a device fault. + +## 8. LTE-M (Cat M1/M2) in depth + +LTE-M is the other 3GPP LPWA variant, also first specified in Release 13. Everything in Sections 2-7 has a direct LTE-M counterpart; this section works through each in turn. + +### 8.1 Deployment mode + +Where NB-IoT can be deployed standalone, in-band, or in a guard-band, LTE-M's 1.4 MHz channel requires six contiguous PRBs, which in practice means it is **almost always deployed in-band**, occupying a reserved slice of an existing LTE carrier (5, 10, 15 or 20 MHz). Standalone or guard-band deployment is not a realistic option for LTE-M in the way it is for NB-IoT, simply because there isn't usually six contiguous PRBs of otherwise-unused spectrum to place it in outside an existing carrier. + +### 8.2 Physical layer + +LTE-M uses the same OFDMA downlink / SC-FDMA uplink structure as NB-IoT, with 15 kHz subcarrier spacing, but across 6 PRBs (1.4 MHz) rather than NB-IoT's single 180 kHz PRB. The bandwidth difference is the direct cause of most of the throughput and latency advantage over NB-IoT. + +The other structural difference is modulation: LTE-M supports **adaptive modulation and coding**, including higher-order schemes such as 16QAM, rather than NB-IoT's fixed QPSK. This is why CQI (Section 5) is actively used on LTE-M -- the network uses it to select the best modulation/coding combination for current conditions, trading spectral efficiency against robustness dynamically, rather than relying on repetition alone. + +### 8.3 Repetition and CE Modes + +LTE-M uses the same repetition principle as NB-IoT but names it differently and caps it lower: + +| CE Mode | Typical use case | Repetition behaviour | +|---|---|---| +| CE Mode A | Normal coverage | Minimal/no repetition, full adaptive modulation available | +| CE Mode B | Extended coverage | Heavy repetition, falls back to more robust, lower-order modulation | + +LTE-M's maximum repetition count is lower than NB-IoT's, which is the direct mechanism behind its slightly worse maximum coupling loss (~156 dB vs NB-IoT's ~164 dB): it simply cannot push repetition as far, in exchange for the throughput and latency it gains from the wider channel and adaptive modulation. + +### 8.4 Signal quality in practice + +RSRP, RSRQ, RSSI and SNR are measured and interpreted identically to NB-IoT (Section 5) -- same definitions, same rough thresholds. The practical difference is that CQI-driven adaptive modulation means a marginal LTE-M link degrades more gracefully than NB-IoT: rather than only falling back to more repetitions, it can also fall back to a more robust modulation scheme at the cost of throughput, giving the network two independent levers instead of NB-IoT's one. + +### 8.5 Power-saving mechanisms + +PSM and eDRX work identically in mechanism to Section 6 (same T3324/T3412 timers, same RRC state machine), since PSM was in fact specified for LTE-M first and then extended to NB-IoT. The one practical difference worth noting is that LTE-M's idle-mode eDRX cycle is capped shorter (~44 minutes maximum) than NB-IoT's (~3 hours), and LTE-M additionally supports **connected-mode DRX (C-DRX)** with more granular short/long cycle configuration, reflecting its design bias towards devices that need to stay more readily reachable than a typical NB-IoT metre. + +### 8.6 Mobility and handover + +This is the fundamental behavioural difference from NB-IoT. Where NB-IoT devices only ever perform idle-mode cell reselection, an LTE-M device in RRC Connected can be **handed over between cells without dropping the session**, using the same mechanism as ordinary LTE: + +1. The network configures the device with measurement reporting rules (e.g. an A3 event: "neighbour cell is offset dB stronger than serving cell for a defined time"). +2. The device continuously measures RSRP/RSRQ of neighbouring cells while connected and sends a measurement report when a configured event triggers. +3. The network decides whether to hand over, prepares the target cell in advance (via the operator's core/X2 or Xn interface between base stations), and sends the device an RRC reconfiguration message with the new cell's parameters. +4. The device retunes to the target cell and resumes the session, with a brief (tens of milliseconds) interruption rather than a full re-attach. + +This is what makes LTE-M suitable for genuinely mobile assets (vehicle tracking, portable equipment) in a way NB-IoT is not, and it is also the mechanism that underpins LTE-M's optional **VoLTE** support, which relies on maintaining a continuous session across cell boundaries in exactly the way a moving phone call does on ordinary LTE. + +## 9. LTE Cat 1 and Cat 1bis + +LTE Cat 1 is not an IoT-optimised technology at all -- it is one of the original LTE UE categories from 3GPP Release 8, predating NB-IoT and LTE-M by several years. It has become relevant to IoT deployments mainly as the **replacement for 2G/3G fallback modules** as operators shut down legacy networks, because it runs on standard LTE infrastructure with no special network-side configuration required. + +Mechanically, Cat 1 uses a full LTE carrier (up to 20 MHz), full-duplex FDD, and originally required 2-antenna MIMO reception. **Cat 1bis** is a later variant that drops the MIMO requirement (single receive antenna), reducing module cost and board area to be closer to NB-IoT/LTE-M module pricing while keeping the same throughput -- this is the variant most commonly used in new IoT hardware designs today rather than original Cat 1. + +Cat 1 has **no coverage-enhancement/repetition mechanism**. It gets whatever coverage a normal LTE cell provides, which is materially worse in deep-indoor or basement scenarios than NB-IoT or LTE-M CE Mode B. PSM and eDRX support exist in later 3GPP releases but are less consistently implemented in Cat 1 modules and networks than in NB-IoT/LTE-M, since Cat 1's original design assumptions were smartphone-like (mains-powered or frequently-charged), not multi-year battery operation. + +## 10. Cross-technology comparison + +| Parameter | NB-IoT | LTE-M (Cat M1) | LTE Cat 1 / Cat 1bis | +|---|---|---|---| +| Channel bandwidth | 180 kHz | 1.4 MHz | Up to 20 MHz | +| Duplex | Half-duplex | Half or full-duplex FDD | Full-duplex FDD | +| Modulation | Fixed QPSK | Adaptive, up to 16QAM | Full LTE adaptive modulation | +| Mobility/handover | No (reselection only) | Yes | Yes | +| Peak throughput (DL/UL) | ~127/159 kbit/s (typ.) | ~1 Mbit/s / 1 Mbit/s | ~10 Mbit/s / 5 Mbit/s | +| Latency (typical, good coverage) | 1.6-10 s | Tens of ms to ~1 s | 10-50 ms | +| Max coupling loss (deep coverage) | ~164 dB | ~156 dB | Standard LTE (~142-146 dB) | +| Voice (VoLTE) | No | Optional | Yes | +| PSM / eDRX | Yes, core design feature | Yes, core design feature | Later releases, inconsistent support | +| Typical module cost | Lowest | Low-medium | Medium (lower with Cat 1bis) | +| Typical use case | Static, deep-coverage, high-density (metering, sensors) | Mobile/tracked assets, wearables, telematics | 2G/3G fallback replacement, higher-throughput IoT, non-battery-critical devices | + +> **Note:** The latency figures above are for the network round trip once the modem has already attached and is transmitting; they do not include the wake-up time out of PSM, which for a sleeping device can dominate total end-to-end latency regardless of technology (see Section 11). + +### Pros and cons summary + +**NB-IoT** + +- Pros: deepest coverage penetration, lowest module cost and power draw, ideal for static high-density deployments like metering. +- Cons: half-duplex only, no mobility support, highest latency, lowest throughput, not suitable for voice or firmware-over-the-air updates of any significant size. + +**LTE-M** + +- Pros: supports mobility/handover and voice, materially lower latency and higher throughput than NB-IoT, graceful degradation via adaptive modulation as well as repetition, still supports PSM/eDRX for multi-year battery life. +- Cons: larger/costlier module than NB-IoT, slightly worse deep-indoor coverage than NB-IoT, higher power draw per transaction. + +**LTE Cat 1 / Cat 1bis** + +- Pros: highest throughput and lowest latency of the three, full duplex, voice support, runs on any standard LTE network with no special configuration, natural replacement for legacy 2G/3G fallback hardware. +- Cons: no coverage enhancement mechanism (worst deep-coverage performance of the three), least consistent PSM/eDRX support, highest power draw and typically higher module cost than NB-IoT (Cat 1bis narrows this gap). + +## 11. When latency actually matters + +The three technologies above were all designed around periodic, delay-tolerant reporting. A different class of device -- an alarm panel that must guarantee cloud delivery within a second, for instance -- has a different constraint, and it's worth being clear about where that constraint actually lives. + +For this kind of requirement, **LTE Cat 1/Cat 1bis kept in a normally-connected (or short-cycle idle) state** is the practical choice among the technologies covered here: sub-1s delivery is routinely achievable on a healthy LTE link, whereas NB-IoT's repetition-based coverage mechanism and multi-second wake cycles make a *guaranteed* sub-1s bound difficult to promise, even though typical-case latency can be lower. Beyond Cat 1, 5G RedCap and URLLC network slices push latency down further still where the operator supports them, and private 5G, Wi-Fi, or wired Ethernet with TSN remove the public RAN from the picture entirely if the device doesn't need wide-area mobility. + +The detail that matters more than the technology choice, though, is this: **PSM and eDRX are latency's real enemy, not the air interface**. A device sitting in PSM is simply unreachable until it wakes on its own schedule, and a device using a long eDRX cycle only checks for pages at each window -- so a "guaranteed under 1 second" requirement effectively rules out PSM and long eDRX for the alarm path, regardless of which radio technology carries it. In practice this means keeping the device in RRC Connected or a short-cycle Idle state (at a battery cost), or using mobile-originated signalling only, where the device itself decides when to raise the alarm rather than waiting to be paged. This is a general sizing point rather than something to over-engineer around; most alarm-class designs solve it by simply accepting the higher standby power draw of staying reachable, rather than trying to force PSM/eDRX to hit a hard latency guarantee. + +## 12. Getting a device onto a network: SIM, eSIM and IMSI + +Everything above assumes the device can already authenticate to a network. That authentication is anchored in the SIM. + +A **SIM** stores the device's **IMSI** (International Mobile Subscriber Identity -- a unique identifier built from Mobile Country Code, Mobile Network Code, and a subscriber number) along with a secret key used to authenticate against the home network's core. The physical form factor is largely irrelevant to how any of this works; what matters is who can reprogram it and how. + +- **Removable SIM**: standard, replaceable, but physically inaccessible once a metre or sensor is sealed and deployed for a decade or more. +- **eSIM (eUICC)**: a soldered, non-removable chip that can still have its network profile **remotely reprovisioned** over the air via an SM-DP+ server, without physical access to the device. This is the practical reason eSIM matters for long-life IoT: it decouples "which operator this device is on" from "can someone get to the device," which matters a great deal once a device is behind a metre box for fifteen years. +- **iSIM**: takes this a step further by integrating the SIM function directly into the modem/SoC silicon rather than as a separate chip, saving board space and cost at high volume. + +**Multi-IMSI SIMs** store several IMSI profiles on one physical or embedded SIM and can switch between them, typically to select whichever home network gives the best signal at a given site, or as a fallback if the primary network becomes unavailable. + +**Roaming** lets a device operate on a network other than its home network via commercial roaming agreements between operators, which is how many "global" IoT SIMs work in practice -- a single IMSI with a home network in one country, roaming permanently onto local networks elsewhere. Two things are worth flagging: some countries restrict *permanent* roaming for security or data-sovereignty reasons, which can block a roaming-only SIM strategy in those markets; and roaming signalling is sometimes routed back via the home network's core before reaching local breakout to the internet, which adds a fixed latency and routing cost on top of anything discussed in Section 11. + +## 13. APN configuration + +The **APN (Access Point Name)** is the configuration that tells the operator's core network how to connect the device's session to an external network, and it is where a lot of the practical security and routing decisions for a fleet actually get made: + +- A **public APN** routes the device out to the public internet, typically with a shared or dynamically-allocated IP, the simplest option but the least controlled. +- A **private APN** keeps device traffic off the public internet entirely, routing it via a dedicated tunnel or leased connection straight into the customer's own network (e.g. a utility's back-office platform). This is the common choice for metering and other critical infrastructure, since it removes public internet exposure as an attack surface for the device fleet. + +The APN is provisioned against the SIM/IMSI, not the device hardware, which is why changing an APN is usually a SIM/network-side operation rather than a firmware change. + +## 14. Certification + +Before a device can legally be sold and connected, it has to clear two largely independent certification hurdles. + +### Network/conformance certification + +- **GCF (Global Certification Forum)**: certifies a device against 3GPP conformance test cases, and is required or strongly preferred by most operators worldwide before they will allow a device onto their network. +- **PTCRB**: functionally equivalent certification used mainly for North American operators. +- **Operator-specific approval**: many operators layer their own additional testing on top of GCF/PTCRB before formally approving a specific module/firmware combination for their network, particularly for critical infrastructure like metering. + +### Regulatory (radio equipment) certification + +- **RED (Radio Equipment Directive, 2014/53/EU)**: the EU legal framework a radio device must satisfy before it can carry a CE mark and be placed on the market, covering spectrum use, EMC, and health/safety. The UK equivalent post-Brexit uses the same technical standards under a UKCA mark, overseen by Ofcom. +- **FCC**: the equivalent US regulatory certification. +- **ISED (formerly Industry Canada/IC)**: the equivalent Canadian regulatory certification. + +A device intended for a specific market needs both the relevant network/conformance certification (GCF/PTCRB, plus operator approval) and the relevant regulatory certification (RED/UKCA, FCC, ISED, etc.) for that market before deployment. + +## 15. Spectrum and bands + +Band choice is an operator/regulatory decision, but it directly shapes coverage, since **lower frequencies propagate further and penetrate buildings better** than higher ones, for a given transmit power. + +| Technology | Common UK/EU bands | Notes | +|---|---|---| +| NB-IoT | B20 (800 MHz), B8 (900 MHz), B3 (1800 MHz) | Sub-1 GHz bands (B20/B8) preferred for deepest coverage | +| LTE-M | B20, B8, B3, B1 (2100 MHz) | Similar low-band preference to NB-IoT for coverage-critical deployments | +| LTE Cat 1/Cat 1bis | Whatever LTE bands the operator has deployed (B1, B3, B7, B20, B28, etc.) | No special low-band preference, since there's no repetition mechanism relying on it | +| 5G RedCap | Existing 5G NR bands, e.g. n78 (3.5 GHz mid-band), n1, n3, n28 | Early RedCap deployment is mid-band first; sub-1 GHz NR coverage bands will follow as operators roll them out | + +Exact bands and their availability vary by country and operator, so this table should be read as "what to expect," not as a substitute for checking the specific target network's band plan. + +## 16. Practical diagnostics (device-side) + +A typical u-blox NB-IoT module exposes signal diagnostics via AT commands: + +```text +AT+NUESTATS ++NUESTATS: +"Signal power",-8814 +"Total power",-8760 +"TX power", 240 +"TX time", 3120 +"RX time", 9840 +"Cell ID", "1A2B3C" +"ECL", 1 +"SNR", 31 +"EARFCN", 3684 +"PCI", 215 +"RSRP", -9812 +"PagingUERxs", 12 +``` + +Fields of interest for troubleshooting: + +- **Signal power / RSRP**: raw strength (see thresholds in Section 5). +- **SNR**: interference/usability (see Section 5) -- typically reported as (actual dB + 20) x 4 by convention on some modules, so check the specific module's datasheet for the scaling before comparing raw values across vendors. +- **ECL**: current coverage enhancement level (0/1/2) -- directly correlates to repetition count and therefore to transmission time and battery draw for that session. + +> **Warning:** Different vendors scale and offset SNR and RSRP differently in their AT command output. Always confirm the conversion formula in the specific module's AT command manual before comparing raw register values across hardware revisions or suppliers. + +## 17. Glossary + +| Term | Meaning | +|---|---| +| APN | Access Point Name; operator-side configuration determining how a device's data session is routed | +| Cat 1 / Cat 1bis | LTE UE Category 1; full-throughput LTE variant used for 2G/3G fallback replacement (Cat 1bis: single-antenna, lower-cost variant) | +| Cat M1 / M2 | LTE-M UE categories; M1 ~1 Mbit/s, M2 ~4 Mbit/s with wider bandwidth | +| CBRS | Citizens Broadband Radio Service; US shared spectrum band commonly used for private LTE/5G | +| C-DRX | Connected-mode DRX; short/long paging-style cycle configuration while in RRC Connected, used by LTE-M | +| CE Level / CE Mode | Coverage Enhancement level (NB-IoT: 0/1/2) or mode (LTE-M: A/B); determines repetition count | +| CQI | Channel Quality Indicator | +| DRX | Discontinuous Reception | +| eSIM / eUICC | Embedded, non-removable SIM that can be remotely reprovisioned with a new network profile | +| FDD | Frequency Division Duplex; separate uplink/downlink frequencies allowing full-duplex operation | +| GCF | Global Certification Forum; 3GPP conformance certification widely required by operators | +| IMSI | International Mobile Subscriber Identity; unique subscriber/device identifier used for network authentication | +| iSIM | SIM function integrated directly into modem/SoC silicon rather than a separate chip | +| MCL | Maximum Coupling Loss; link budget measure of deep-coverage capability | +| MIMO | Multiple Input Multiple Output; multi-antenna transmission/reception | +| PSM | Power Saving Mode; fully unreachable, minimal power state | +| PTCRB | North American equivalent of GCF conformance certification | +| PTW | Paging Time Window; wake window within an eDRX cycle | +| RED | Radio Equipment Directive (EU); regulatory certification for radio devices (UK equivalent: UKCA) | +| RedCap | Reduced Capability; 3GPP Release 17 5G NR device class positioned between LTE-M/Cat 1 and full 5G | +| RRC | Radio Resource Control (connection state machine) | +| RSRP | Reference Signal Received Power | +| RSRQ | Reference Signal Received Quality | +| RSSI | Received Signal Strength Indicator | +| SNR/SINR | Signal-to-(Interference-plus-)Noise Ratio | +| T3324 | Active timer (RRC Idle duration before PSM) | +| T3412 | Periodic TAU timer (maximum time before mandatory wake) | +| TAU | Tracking Area Update | +| TSN | Time-Sensitive Networking; Ethernet standards providing deterministic low-latency delivery | +| URLLC | Ultra-Reliable Low-Latency Communication; 5G NR network feature/slice for single-digit-ms latency | diff --git a/content/guides/deep-dives/rtos-internals.md b/content/guides/deep-dives/rtos-internals.md new file mode 100644 index 0000000..63378cc --- /dev/null +++ b/content/guides/deep-dives/rtos-internals.md @@ -0,0 +1,268 @@ +--- +title: RTOS internals +--- + +# RTOS internals + +## Overview + +A real-time operating system is often described as "a small OS for embedded devices", but that undersells what actually makes it different. The core idea is simpler and more specific: an RTOS exists to make the timing of execution predictable, not just the outcome. A general-purpose OS like Linux optimises for throughput and fairness across many competing processes; an RTOS optimises for a task meeting its deadline, even if that means starving lower-priority work indefinitely. Understanding that single goal explains almost everything else: why scheduling is priority-based rather than fair, why context switches are measured in microseconds, and why synchronisation primitives carry so much ceremony around priority. + +Throughout, FreeRTOS is used as the worked example, since its API is what most embedded engineers meet first. The final section broadens out to compare FreeRTOS, Zephyr, Eclipse ThreadX, PX5 RTOS and SEGGER embOS as five complete kernels, rather than feature by feature. + +## What Is a Task + +A task (also called a thread, though RTOS literature tends to say "task") is an independent thread of execution with its own stack, its own saved register state, and a priority. Unlike a process on a general-purpose OS, tasks on most RTOSes share a single flat memory space; there is no virtual memory, no per-task page table, and usually no memory protection unless the silicon has an MPU and the RTOS is configured to use it. + +Each task is represented internally by a Task Control Block (TCB): a small structure holding the task's saved stack pointer, its priority, its current state, and a pointer to its stack memory. The TCB is what the scheduler actually manipulates; the task's code and data are just memory the TCB points at. + +```mermaid +graph LR + TCB["Task Control Block\n- stack pointer\n- priority\n- state\n- stack base/size"] --> Stack["Task's own stack\n(saved registers when not running)"] + TCB --> Code["Task function\n(shared code, not per-task)"] +``` + +A task's stack is not scratch space in the way a single-threaded programme's stack is. Every task has its own, sized at creation time, and that size has to be estimated correctly: too small and a deep call chain or interrupt landing on that stack corrupts adjacent memory silently; too large and RAM, often the scarcest resource on a microcontroller, is wasted per task. + +## Task States + +A task is always in exactly one of a small number of states, and the scheduler's whole job is deciding which ready task actually runs. + +```mermaid +graph LR + Ready["Ready\n(runnable, waiting for CPU)"] + Running["Running\n(currently on the CPU)"] + Blocked["Blocked\n(waiting on semaphore, queue, delay, etc.)"] + Suspended["Suspended\n(explicitly paused, not scheduled)"] + + Ready -- "scheduler picks it" --> Running + Running -- "preempted or yields" --> Ready + Running -- "blocks on resource" --> Blocked + Blocked -- "resource becomes available" --> Ready + Running -- "vTaskSuspend" --> Suspended + Suspended -- "vTaskResume" --> Ready +``` + +Only one task is ever in the Running state on a single core. Everything else waiting to run sits in Ready; everything waiting on something (a timer, a mutex, data on a queue) sits in Blocked and consumes no CPU time at all while there. A blocked task costs nothing, so it is normal to have many tasks, most of them blocked most of the time, waiting for their particular event. + +## The Scheduler + +The scheduler decides, at every scheduling point, which Ready task becomes the Running task. Almost all RTOSes used in industrial and consumer embedded work use priority-based preemptive scheduling as the default: the highest-priority Ready task always runs, and it runs immediately, interrupting a lower-priority task if necessary. + +This is a deliberately different goal to a general-purpose OS scheduler. Linux's completely fair scheduler tries to give every process a proportionate share of CPU time over some window; an RTOS scheduler makes no such promise. A low-priority task can be starved indefinitely if higher-priority tasks are always Ready, and that is treated as correct behaviour, not a bug, because the entire premise is that priority reflects real deadline urgency. + +```mermaid +graph TB + SP["Scheduler runs\n(interrupt returns, task blocks,\npriority changes, or a tick occurs)"] + SP --> Check{"Highest-priority\nReady task ==\ncurrently Running task?"} + Check -- "Yes" --> NoOp["No context switch"] + Check -- "No" --> Switch["Context switch to\nhighest-priority Ready task"] +``` + +A preemptive scheduler can interrupt a running task at any scheduling event, without that task's cooperation. A cooperative scheduler only switches tasks when the running task explicitly yields, which is simpler to reason about but means one badly behaved task can block everything else indefinitely. Most modern RTOSes default to preemptive, but many also support cooperative or "run to completion" scheduling for specific low-priority tasks. + +When two or more Ready tasks share the same priority, most schedulers apply round-robin time slicing between them: each gets a fixed slice before being rotated to the back of the queue for that priority level. This only ever happens within a priority level; a lower-priority task never gets a slice while a higher-priority task is Ready. + +## Context Switching + +A context switch is the mechanical act of saving one task's CPU register state to its own stack and restoring another's. On a typical Cortex-M target, this is a fixed, small sequence: push the remaining general-purpose registers not already saved by the hardware exception entry, save the stack pointer into the outgoing task's TCB, load the stack pointer from the incoming task's TCB, and pop its registers back off. This cost, multiplied by however often it happens, is pure overhead the application never sees as useful work. + +```mermaid +graph LR + A["Task A running\n(registers live in CPU)"] --> B["Save Task A's registers\nto Task A's stack"] + B --> C["Load Task B's saved\nregisters from Task B's stack"] + C --> D["Task B running\n(registers live in CPU)"] +``` + +This is also why an RTOS cares so much about worst-case interrupt latency and context-switch time as headline numbers: they are the tax paid on every single scheduling decision, and a real-time deadline has to account for that tax on top of the actual work. + +## Interrupts and Deferred Processing + +An Interrupt Service Routine (ISR) runs outside the normal task scheduling entirely; it preempts whatever task is running regardless of that task's priority, because interrupts sit above the RTOS's own priority scheme at the hardware level. The rule that keeps a system predictable is: keep the ISR short. An ISR that does substantial work delays every task in the system, including higher-priority ones, for as long as it runs. + +The standard pattern is to defer the real work out of the ISR and into a task: + +```mermaid +graph LR + HW["Hardware event"] --> ISR["ISR:\nread minimal data,\nclear interrupt flag,\nsignal a task"] + ISR -- "semaphore give / queue send /\ntask notification" --> Task["Deferred handler task\n(does the real work,\nscheduled normally)"] +``` + +In FreeRTOS, the signalling step is deliberately one of the cheapest calls available, `xSemaphoreGiveFromISR()` or a direct-to-task notification via `xTaskNotifyGiveFromISR()`, precisely because it happens inside an ISR, where every cycle counts double. + +## Synchronisation Primitives + +Tasks running concurrently need to coordinate: protect shared data, signal that an event happened, or pass data between each other. The primitives look similar on the surface but solve different problems, and FreeRTOS's API makes a good concrete reference for how they differ mechanically. + +### Mutex + +A mutex protects a shared resource by enforcing mutual exclusion: only the task that locked it can unlock it, and most RTOS mutexes support priority inheritance specifically because ownership is a well-defined concept. Use a mutex when the question is "only one task at a time may touch this data or hardware peripheral". + +In FreeRTOS, a mutex is created with `xSemaphoreCreateMutex()`. It starts full (available immediately) and tracks ownership internally: only the task that called `xSemaphoreTake()` successfully may call `xSemaphoreGive()` to release it, and FreeRTOS will not release a mutex on behalf of a task that never took it. This ownership tracking is the entire mechanism priority inheritance depends on, covered below. + +```c +SemaphoreHandle_t xMutex = xSemaphoreCreateMutex(); + +if (xSemaphoreTake(xMutex, portMAX_DELAY) == pdTRUE) { + /* critical section: only this task may proceed */ + xSemaphoreGive(xMutex); +} +``` + +### Binary semaphore + +A binary semaphore has no concept of ownership; any task can give it, and any task can take it. It is a signalling mechanism, not a protection mechanism: the classic use is an ISR giving a semaphore to wake a waiting task. Using a semaphore to protect shared data instead of a mutex is a common mistake, because without ownership there is no priority inheritance, and priority inversion can recur. + +In FreeRTOS, this is `xSemaphoreCreateBinary()`, which starts empty (must be given once before it can be taken). + +```mermaid +graph TB + subgraph "Binary semaphore: no owner" + BS1["ISR: xSemaphoreGiveFromISR()"] -.-> BS2["Task B: xSemaphoreTake()"] + end + subgraph "Mutex: owner tracked" + M1["Task A: xSemaphoreTake()\n(A recorded as holder)"] --> M2["...critical section..."] + M2 --> M3["Task A: xSemaphoreGive()\n(only A may do this)"] + end +``` + +> **Important:** `xSemaphoreCreateBinary()` never provides priority inheritance, even used in a mutex-like pattern (one task always gives, another always takes back). If a shared resource needs protection and priority inversion is a concern, the fix is `xSemaphoreCreateMutex()`, not extra care with a binary semaphore. + +### Counting semaphore + +A counting semaphore generalises the binary case to a count of available resources (buffer slots, pool entries), decremented on take and incremented on give, with tasks blocking when the count reaches zero. In FreeRTOS this is `xSemaphoreCreateCounting(uxMaxCount, uxInitialCount)`. + +### Queue + +A queue passes actual data between tasks, copying fixed-size items in and out, with tasks able to block on either end. This is the standard way to hand data from an ISR or producer task to a consumer task without them sharing memory directly. In FreeRTOS, `xQueueCreate(uxQueueLength, uxItemSize)` creates the queue, and `xQueueSend()` / `xQueueReceive()` (or their `FromISR` variants) move items with an optional blocking timeout. A queue of length one holding no meaningful data is, internally, exactly how FreeRTOS builds a binary semaphore; both semaphores and mutexes sit on top of the same underlying queue structure. + +```mermaid +graph LR + subgraph "Mutex: protects data, has ownership" + M1["Task locks"] --> M2["...uses resource..."] --> M3["Same task unlocks"] + end + subgraph "Semaphore: signals an event, no ownership" + S1["ISR or Task A gives"] -.-> S2["Task B takes\n(different task)"] + end + subgraph "Queue: moves data, blocks either end" + Q1["Producer sends item"] --> Q2["Queue buffer"] --> Q3["Consumer receives item"] + end +``` + +### Mailboxes and lightweight signalling + +Some kernels add a further primitive, often called a mailbox or postbox, for cases a full queue is comparatively expensive for: handing a single value or wake-up signal directly to one specific task. FreeRTOS's version is the task notification, which lives inside the receiving task's own TCB rather than as a separate kernel object, so there is nothing to allocate or size: + +```c +xTaskNotifyGive(xTaskHandle); /* ISR or task: send */ +ulTaskNotifyTake(pdTRUE, portMAX_DELAY); /* task: receive, clearing count */ +``` + +Giving a task notification is measurably faster and uses less RAM than either a binary semaphore or a queue of length one. The trade-off is that a task notification can only ever have one recipient, whereas a queue can be read by any number of consumer tasks that happen to call receive on it. How other kernels implement this same idea, and how their terminology diverges, is covered in the comparison section. + +## Priority Inversion and Priority Inheritance + +Priority inversion occurs when a high-priority task is blocked waiting for a resource held by a low-priority task, and an unrelated medium-priority task, which needs no such resource, preempts the low-priority task and runs freely. The high-priority task ends up waiting on the medium-priority one indirectly, which is exactly backwards from what the priority scheme was meant to guarantee. + +```mermaid +graph TB + subgraph "Without priority inheritance" + L1["Low-priority task\nholds mutex"] --> L2["Medium-priority task\npreempts it (unrelated work)"] + L2 --> L3["High-priority task blocked,\nwaiting on mutex the whole time"] + end +``` + +This is not a hypothetical for firmware with mixed-criticality tasks: a low-priority logging task holding a mutex over a shared I2C bus, preempted by an unrelated medium-priority housekeeping task, can leave a high-priority sensor-read task blocked far longer than its deadline allows, even though the housekeeping task has nothing to do with the I2C bus at all. + +The standard fix is priority inheritance: when a high-priority task blocks on a mutex held by a lower-priority one, the RTOS temporarily raises the holder's priority to match the highest-priority waiter, for exactly as long as it holds the mutex. + +FreeRTOS's mutex implementation includes this automatically; it is not an opt-in feature or a separate API. When `xSemaphoreTake()` is called on a mutex already held by a lower-priority task, FreeRTOS calls its internal `vTaskPriorityInherit()` on the holder, temporarily raising its priority to match the blocked, higher-priority caller. When the holder calls `xSemaphoreGive()`, FreeRTOS calls `vTaskPriorityDisinherit()`, restoring the holder's original priority (or the next-highest priority still waiting, if more than one task was blocked). + +```mermaid +graph TB + subgraph "With priority inheritance" + H1["Low-priority task holds mutex,\nboosted to high priority by vTaskPriorityInherit()"] --> H2["Medium-priority task cannot preempt:\nboosted holder now outranks it"] + H2 --> H3["Holder finishes critical section quickly,\nxSemaphoreGive() triggers vTaskPriorityDisinherit()"] + H3 --> H4["High-priority task's xSemaphoreTake() unblocks"] + end +``` + +> **Note:** priority inheritance bounds the inversion; it does not eliminate the wait entirely. The high-priority task still waits for the critical section to finish, which is correct and unavoidable -- the point is that the wait is now bounded by the holder's own critical-section length, not by however long an unrelated medium-priority task chooses to run. + +> **Warning:** `vTaskPrioritySet()` called manually on a task that also participates in mutex-protected sections can interact awkwardly with inheritance bookkeeping if not done carefully; in most application code it is better to leave priority management to the inheritance mechanism rather than adjusting priorities directly around mutex use. + +Because priority inheritance depends on the RTOS knowing who the holder is, in FreeRTOS it only exists for `xSemaphoreCreateMutex()` (and the recursive variant, `xSemaphoreCreateRecursiveMutex()`). It categorically does not exist for `xSemaphoreCreateBinary()` or `xSemaphoreCreateCounting()`, because neither tracks ownership. Other kernels make this optional, add a second mechanism entirely, or support a different protocol (priority ceiling) alongside inheritance; the comparison section covers each approach. + +## Memory: Static Versus Dynamic Allocation + +Most RTOSes offer a choice between allocating tasks, queues, and semaphores statically (fixed at compile time, memory reserved in `.bss`) or dynamically (via the RTOS's own heap, usually a purpose-built allocator rather than the standard C library one). Static allocation is deterministic and avoids fragmentation entirely, at the cost of having to size everything up front; dynamic allocation is more flexible but reintroduces the classic embedded worry of heap fragmentation over long uptimes. + +> **Note:** this is a large part of why safety-critical and long-uptime firmware, including metering firmware, tends to default to static allocation wherever the RTOS supports it, reserving dynamic allocation for genuinely variable-sized, short-lived cases. + +## Tickless Idle + +An RTOS normally runs a periodic tick interrupt (in FreeRTOS, `configTICK_RATE_HZ`, commonly 1000 Hz) to drive time-slicing and to wake tasks whose delays have expired. On a battery- or PSM-constrained device, waking the CPU on every tick even when every task is blocked is wasted energy; that periodic wake-up is the entire cost tickless idle exists to remove. + +In FreeRTOS, with `configUSE_TICKLESS_IDLE` set to `1`, the idle task, instead of just spinning, calls the port-specific `portSUPPRESS_TICKS_AND_SLEEP()`. This function: + +1. calculates the number of tick periods until the next task is due to wake, from the delayed-task list +1. reprograms the SysTick (or an equivalent low-power timer) to fire once, after that calculated interval, rather than on every tick +1. puts the CPU into a low-power sleep mode (`WFI` on Cortex-M, or a deeper stop mode on some parts) for that interval +1. on waking, either because the timer fired or because an interrupt occurred early, corrects the tick count for time actually slept and resumes normal ticking + +```mermaid +graph TB + Idle["Idle task runs:\nno Ready task above idle priority"] --> Calc["Calculate ticks until next\ntask wake-up (delayed list)"] + Calc --> Reprogram["Reprogram timer to fire once\nafter that interval"] + Reprogram --> Sleep["Enter low-power sleep (WFI)"] + Sleep -- "timer fires or\nan interrupt occurs" --> Correct["Correct tick count for\ntime actually slept"] + Correct --> Resume["Resume normal tick-driven scheduling"] +``` + +> **Important:** an early wake caused by an unrelated interrupt still requires the tick count correction step, because real time passed during the sleep regardless of why the CPU woke. Skipping this step is a common source of subtly wrong delays and timeouts in a tickless configuration, since every blocked task's remaining delay is computed from the tick count. + +Tickless idle only ever engages when the idle task is the highest-priority Ready task, meaning no other task, mutex holder or otherwise, has anything left to do. A task incorrectly left runnable (spinning on a semaphore instead of blocking with a timeout) will prevent tickless idle from ever engaging, which tends to show up as unexpectedly high idle power draw rather than a scheduling bug. Other kernels take different positions on how central tickless operation is to the design, covered below. + +## Comparison with Other Execution Models + +The simplest alternative to an RTOS is no scheduler at all: a single `while (1)` loop that calls each subsystem's handling function in turn, with interrupts doing the minimum and setting flags the loop polls. This is entirely deterministic and has zero context-switch overhead, but every piece of work shares the same implicit priority (loop order), and a slow iteration in one subsystem delays every other subsystem's next check by exactly that long. It scales poorly as the number of independent concerns grows, which is the usual trigger for moving to an RTOS. + +Linux's default scheduler (CFS) optimises for fairness and throughput across processes that mostly don't have hard deadlines; it has virtual memory, per-process address space isolation, and a scheduler tick and context-switch cost orders of magnitude larger than a typical RTOS. Linux does have real-time scheduling classes (`SCHED_FIFO`, `SCHED_RR`) that borrow the RTOS priority model, and `PREEMPT_RT` patches push determinism further, but the underlying kernel still carries far more machinery than an RTOS kernel, which is often only a few thousand lines of code in total. + +The underlying distinction: a superloop has no scheduler at all, so priority does not exist as a concept. An RTOS has a scheduler whose entire purpose is enforcing priority order deterministically, at the cost of task and primitive bookkeeping. A general-purpose OS has a scheduler whose entire purpose is fairness and throughput across untrusted, isolated processes, with determinism as an optional extra. + +## Five RTOS Kernels Compared + +Everything above describes the general model, with FreeRTOS as the worked example. FreeRTOS, Zephyr, Eclipse ThreadX, PX5 RTOS, and SEGGER embOS all implement the same underlying model: priority-based preemptive scheduling, round-robin time slicing at equal priority, and mutex/semaphore/queue primitives. At the scheduling-algorithm level, all five are close to identical, so that is not a meaningful axis of comparison. What actually differs is each kernel's design ethos: what it is optimising for as a product, and why a team would pick it. Each is summarised as a whole below. + +### FreeRTOS + +FreeRTOS's ethos is minimalism and ubiquity. The kernel itself is small, the primitive set stays deliberately narrow (semaphores and mutexes are both implemented as special cases of the same queue mechanism, rather than as separate machinery), and priority inheritance on mutexes is automatic and non-optional, so there is nothing to configure to get correct behaviour by default. + +Its key benefit is reach: FreeRTOS is MIT licensed and ported across a huge range of MCU vendors and toolchains, so most vendor SDKs already ship a working port. For a team that wants the smallest possible dependency and the widest possible hardware choice, without needing a rich built-in subsystem stack, FreeRTOS is usually the path of least resistance. + +### Zephyr + +Zephyr's ethos is "batteries included": it positions itself less as a bare kernel and more as a full embedded platform, with networking stacks, Bluetooth, USB, filesystems and device-tree-based hardware description built in alongside the scheduler. Governed by the Linux Foundation and Apache 2.0 licensed, it is designed for teams building genuinely connected products who would otherwise have to integrate several separate libraries on top of a bare-metal kernel themselves. + +Its key benefit is breadth: one codebase and build system (`west`, CMake, device tree) spans a very wide range of silicon, with the connectivity and driver subsystems most IoT products need already present. The trade-off is a larger footprint and a steeper learning curve than a bare scheduler, which is the price of that breadth. + +### Eclipse ThreadX + +ThreadX's ethos is proven, certified determinism at very small size. It has one of the longest deployment histories of any RTOS, widely cited at around ten billion devices, mostly accumulated while it was a commercial product (Express Logic, then Microsoft's Azure RTOS) before being open sourced under the Eclipse Foundation. It also offers preemption-threshold, a scheduling-level tool distinct from priority inheritance, giving it a genuinely different option for controlling preemption that most other kernels do not offer. + +Its key benefit is maturity: an extremely long track record in the field, a deep history of safety certifications, and a small footprint despite that history, making it a common default choice for safety-critical or certification-heavy programmes. + +### PX5 RTOS + +PX5's ethos is portability through standards. Rather than a proprietary API, it is built natively around POSIX pthreads, so application code written against PX5 shares far more directly with Linux-side code than a proprietary-API kernel would allow. It is a newer product from the original author of ThreadX and Nucleus, aimed squarely at safety-certified use. + +Its key benefit is a combination that is unusual to find together: POSIX-standard code portability, one of the smallest footprints and fastest documented API/context-switch times of the five, run-time integrity checking (Pointer/Data Verification), and safety certifications including IEC 61508 SIL 4 and ISO 26262 ASIL D. The trade-off is a much shorter field track record than ThreadX, given how recently it launched. + +### SEGGER embOS + +embOS's ethos is commercial simplicity backed by strong tooling. It is a straightforward, general-purpose kernel whose biggest differentiator is not the kernel itself but its integration with SEGGER's own ecosystem: J-Link debug probes, SystemView real-time tracing, and Embedded Studio. Priority inheritance and idle-time power saving are both built in by default rather than requiring configuration. + +Its key benefit is the debugging and tracing experience for teams already using SEGGER hardware, plus a straightforward commercial licensing model. The trade-off is that its differentiation is mostly at the tooling layer rather than the kernel design itself, since the underlying scheduling model is unremarkable next to the other four. + +> **Note:** none of these differences make one kernel strictly better; they reflect different target markets and different things each vendor chose to optimise for. FreeRTOS trades on ubiquity and vendor support, Zephyr on subsystem breadth for connected IoT, ThreadX on deployment scale and certification history, PX5 on POSIX portability and footprint, and embOS on tool integration. The right choice depends on which of those trade-offs matters for the specific product, not on which kernel implements the underlying scheduling and synchronisation model "more correctly" -- at that level, all five are close to identical. diff --git a/content/guides/deep-dives/static-analysis.md b/content/guides/deep-dives/static-analysis.md new file mode 100644 index 0000000..eb052bd --- /dev/null +++ b/content/guides/deep-dives/static-analysis.md @@ -0,0 +1,200 @@ +--- +title: Static analysis +--- + +# Static analysis + +## Purpose and scope + +This document explains static analysis as a technique and a tool category: what it actually does, the different levels of rigour tools operate at, and where it sits alongside compiler warnings, code review, and testing in an overall code quality picture. It specifically addresses a common conflation -- treating "MISRA compliant" and "statically analysed" as the same thing -- and covers why static analysis matters for de-risking software security specifically, not just general code quality. It is a companion document to *MISRA C: A Deep Dive*, which covers one specific rule set that static analysis tools are commonly used to check; this document covers the technique itself, independent of any one rule set. + +## 1. What static analysis actually is + +Static analysis means examining source code (or, for some tools, compiled binaries or intermediate representations) **without executing it**, to find properties that are true or false regardless of what input the programme eventually receives. This is the defining contrast with dynamic analysis -- testing, fuzzing, sanitizers -- which observes the programme's actual behaviour while it runs, on specific inputs. + +```mermaid +flowchart LR + A[Source code] --> B[Static analysis: examine without running] + A --> C[Compile and run] + C --> D[Dynamic analysis: observe actual execution] + B --> E["Finds: properties true for all possible executions the tool can reason about"] + D --> F["Finds: bugs actually triggered by the specific inputs tested"] +``` + +Neither approach subsumes the other. Static analysis can, in principle, reason about every code path including ones no test ever exercises; but it reasons about an *approximation* of the programme's behaviour, and can be wrong in both directions -- flagging something that can never actually happen (a false positive) or missing something that can (a false negative). Dynamic analysis is ground truth for whatever it actually observes, but only for the inputs and paths a test run happens to exercise. + +## 2. The levels of static analysis + +"Static analysis" is not one technique -- it is a spectrum of increasingly sophisticated approaches to the same underlying question (what can this code do?), each level trading analysis cost and false-positive rate for depth of reasoning. + +```mermaid +flowchart TB + A["Level 0: Compiler diagnostics\nsyntax, type checking, local warnings"] --> B["Level 1: Pattern/lint-based checkers\nsyntactic rule matching, single translation unit"] + B --> C["Level 2: Data-flow and control-flow analysis\ntraces values and paths through a function/program"] + C --> D["Level 3: Abstract interpretation\nsound, mathematically-grounded over-approximation of all possible states"] + D --> E["Level 4: Formal verification\nproves code matches a written specification"] +``` + +### Level 0: Compiler diagnostics + +Every compiler performs a basic level of static analysis as a side effect of compiling: type checking, detecting an uninitialised variable used before assignment in an obvious case, an unreachable `return`, a signed/unsigned comparison mismatch. This is fast (it runs on every build, for free) and shallow -- confined largely to a single translation unit, and limited to what the compiler's own front-end already computes while generating code. + +### Level 1: Pattern/lint-based checkers + +Tools such as a basic linter or Cppcheck's default mode look for known bad patterns syntactically -- a `strcpy` call (a classic buffer-overflow-prone function), a comparison of a value against itself, an assignment where a comparison was likely intended (`=` vs `==`). These checks are fast and easy to reason about, but they are pattern matches, not analyses of what the code actually does at runtime -- they don't trace a value's possible range or follow it across function boundaries. + +### Level 2: Data-flow and control-flow analysis + +This is where most commercial static analysis tools (Coverity, Klocwork, Perforce QAC) do their real work: building a control-flow graph of the programme and tracing how values flow through it -- can this pointer be NULL at this dereference, can this array index exceed its bounds, can this value reach this sink without being validated (taint analysis). This requires modelling the programme's possible states across function calls and, for whole-programme analysis, across translation units, which is considerably more expensive than Level 1 but catches genuine logic bugs Level 1 pattern matching cannot see. + +### Level 3: Abstract interpretation + +Tools such as Polyspace Code Prover or Astrée go further: rather than exploring representative paths, they compute a mathematically sound *over-approximation* of every value a variable could ever take, at every point in the programme, across all possible inputs. "Sound" here has a precise meaning (see section 5) -- if the tool says a runtime error is impossible, that is a proof, not a heuristic. The cost is a higher false-positive rate on legitimately safe code the abstraction is too coarse to fully understand, and significantly higher analysis time. + +### Level 4: Formal verification + +At the far end, tools like SPARK Ada, Frama-C, or CBMC do not just check for the *absence* of runtime errors (null derefs, overflows, out-of-bounds access) -- they prove that code satisfies a **written specification** of what it should do, using formal proof techniques (model checking, SMT solvers). This requires the specification to be written formally in the first place, which is a substantial upfront cost, and is typically reserved for the highest-assurance components of a system (a cryptographic primitive, a safety interlock) rather than applied wholesale. + +> **Note:** These levels are not mutually exclusive tiers you graduate through -- a mature pipeline typically runs several simultaneously (compiler warnings on every build, a Level 2 tool in CI, perhaps Level 3 on a safety-critical module), because each level catches a different, only partially overlapping, class of problem. + +## 3. A second axis: how faithfully the tool models your actual compiler and target + +Section 2 covered levels of *analytical depth* -- how far a tool reasons about programme behaviour. There is a second, largely independent axis worth covering explicitly: how faithfully the tool models the **specific compiler and target** the code will actually be built for, as opposed to a generic model of "C" or "C++" as a language standard describes it. + +```mermaid +flowchart TB + A["Generic syntactic parsing\nown lightweight parser, no real preprocessing\ne.g. basic regex/pattern linters"] --> B["Standard-conformant front end\nfull parser for the language standard\nbut generic assumptions about int width, endianness, etc\ne.g. many standalone MISRA/portability checkers"] + B --> C["Actual compiler front end\nplugs into or reuses your real compiler's parser and preprocessor\nsees exactly what it sees: macros, pragmas, intrinsics, extensions\ne.g. clang-tidy, GCC -fanalyzer, MSVC /analyze"] + C --> D["Target-aware modelling\nfaithfully models the actual MCU/ABI:\nint/pointer widths, alignment, endianness, calling convention, register layout\ne.g. Polyspace/Astrée configured for a specific cross-compiler and target"] +``` + +This matters specifically because a large fraction of what MISRA C (and C generally) worries about is **implementation-defined and unspecified behaviour** -- the size of `int`, signedness of `char`, struct padding and alignment, how a signed integer overflow actually behaves on a given ABI. These are, by definition, not fixed by the language standard -- they are fixed by the actual compiler, flags, and target architecture in use. A tool reasoning about generic "C" without reference to any of that can only ever tell you what is *portable* to worry about, not what your build *actually does*. + +- **A generic parser** (the shallow end) may not even fully tokenise the exact dialect your codebase uses -- vendor-specific keywords, compiler intrinsics, or unusual preprocessor macro patterns can trip up a lightweight standalone parser that was never built against your compiler's actual grammar. +- **A standards-conformant front end** parses correctly against the C/C++ standard itself, but still has to *assume* answers to implementation-defined questions (a common default: `int` is 32-bit, two's complement, little-endian) unless separately configured -- which is fine for portability-focused checks (exactly MISRA's original goal, see the MISRA companion document) but can silently produce wrong or irrelevant findings if your actual target differs from the tool's default assumptions. +- **Tools built directly on the real compiler's front end** -- clang-tidy and GCC's `-fanalyzer` are the clearest examples, since both reuse the actual Clang/GCC parser and preprocessor rather than a separate reimplementation -- see precisely what your real build sees: the same macro expansions, the same `#pragma` handling, the same compiler-specific extensions and intrinsics. This eliminates an entire class of false positive/negative caused by the analysis tool and the real compiler simply disagreeing about what a piece of code even parses to. +- **Target-aware modelling** goes one step further again: a tool like Polyspace or Astrée, correctly configured for your actual cross-compiler and MCU, doesn't just parse identically to your toolchain -- it reasons about value ranges, overflow, and undefined behaviour using your target's *actual* integer widths, alignment rules, endianness, and calling convention, rather than a generic default. This is the difference between "this could theoretically overflow on some C implementation" and "this overflows given the EFM32-class target and compiler flags this project actually ships with" -- a materially more precise (and more directly actionable) finding. + +> **Note:** this axis and the depth axis in section 2 combine, they don't replace each other. A tool can be deep (Level 3, abstract interpretation) but still configured with generic target assumptions if nobody has told it the real target's integer widths and calling convention -- in which case its mathematically sound proofs are sound proofs about the *wrong machine*. Getting the target configuration right is a prerequisite for the deeper levels' guarantees to actually mean what they claim about your shipped binary, not an optional refinement. + +## 4. A tool landscape at a glance + +Sections 2 and 3 named specific tools as they came up against each axis separately. Pulled into one table, against both axes at once: + +| Tool | Depth (section 2) | Front-end/target fidelity (section 3) | What it actually does | Licensing | +| --- | --- | --- | --- | --- | +| GCC / Clang `-Wall -Wextra` | Level 0 | Is the actual compiler | Syntax, type, and local flow warnings as a side effect of compiling | Free | +| Cppcheck (default/free mode) | Level 1 | Own standalone parser, generic assumptions | Pattern-based bug detection (known-bad constructs), no build integration needed | Free | +| clang-tidy | Level 1-2 (style checks plus some Clang Static Analyzer dataflow checks) | Actual Clang front end | Style/modernisation checks, plus genuine dataflow-based bug checks when the static analyzer checks are enabled | Free | +| GCC `-fanalyzer` | Level 2 | Actual GCC front end | Intraprocedural dataflow -- use-after-free, double-free, leaks, some null-deref paths | Free | +| Semgrep | Level 1 (pattern matching with light semantic awareness) | Own parser per language | Fast, rule-authorable pattern matching, popular for CWE-mapped security rules in CI | Free/commercial tiers | +| SonarQube/SonarCloud | Level 1-2 | Own parser per language | Broad code-quality plus security rule coverage across many languages, dashboarded over time | Free/commercial tiers | +| Coverity (Synopsys) | Level 2 | Build-captured, tuned to match your real compiler's view of the build | Whole-programme interprocedural dataflow and taint analysis | Commercial | +| Klocwork (Perforce) | Level 2 | Similar build-capture approach to Coverity | Whole-programme dataflow, MISRA/CERT rule checking built in | Commercial | +| Perforce QA·C / QAC | Level 2, MISRA-focused | Configurable to target compiler/architecture | Deep C/C++ dataflow with strong MISRA rule coverage specifically | Commercial | +| Parasoft C/C++test | Level 2 | Configurable to target | Dataflow analysis, MISRA/CERT checking, integrates with unit test/coverage tooling | Commercial | +| LDRA | Level 2 | Configurable to target | Dataflow analysis, MISRA/CERT, plus certification-focused coverage and traceability tooling | Commercial | +| CodeSonar (GrammaTech) | Level 2, moving toward Level 3 in places | Configurable to target | Whole-programme dataflow with some abstract-interpretation-style range analysis | Commercial | +| Polyspace Bug Finder | Level 2 (deliberately unsound, low-noise) | Configurable to compiler/target | Finds likely bugs with a low false-positive rate; the "find real bugs fast" counterpart to Code Prover | Commercial | +| Polyspace Code Prover | Level 3 | Configurable to compiler/target, used deliberately for fidelity | Sound abstract interpretation -- proves absence of specific runtime error classes (overflow, out-of-bounds, div-by-zero) given the configured target | Commercial | +| Astrée (AbsInt) | Level 3 | Very high target fidelity, built specifically for this | Sound abstract interpretation, widely used in aerospace/automotive for proving absence of runtime errors on real embedded targets | Commercial | +| Frama-C | Level 3 (EVA plugin) to Level 4 (WP plugin) | Configurable, ACSL specification-driven for the WP plugin | Open framework: value analysis (EVA) for abstract interpretation, or deductive proof (WP) against formally written ACSL contracts | Free/open source | +| CBMC | Level 4 (bounded model checking) | Compiles against the real C source | Proves assertions hold up to a bounded loop/recursion depth, or finds a concrete counterexample | Free/open source | +| SPARK Ada / GNATprove | Level 4 | Sound, Ada/SPARK language subset | Proves code satisfies formally written contracts; used in the highest-assurance rail/aerospace/security contexts | Community edition free, commercial support available | + +> **Note:** this table is illustrative of the landscape, not an endorsement or a procurement recommendation -- tool capability, pricing, and MISRA/target support all change over time, so treat specific product claims as something to verify against current vendor documentation before a tool selection decision, not something to take as fixed from this document. + +## 5. Soundness and completeness: the trade-off underneath every tool + +Two formal properties explain why static analysis tools behave so differently from each other, and why "the tool found nothing" never means "there is nothing to find": + +- **Sound** -- if the tool reports no issue of a given kind, that is a guarantee: no such issue exists (within the scope the tool analysed). A sound tool may still produce false positives (flagging something that cannot actually happen) because it over-approximates to stay safe. +- **Complete** -- if an issue actually exists, the tool will report it. A complete tool may produce false negatives if it is not also sound, by under-approximating to reduce noise. + +Almost no practical tool is both fully sound and fully complete for a language as permissive as C -- doing so runs into undecidability results from theoretical computer science (the general problem of "will this code ever divide by zero" is equivalent to the halting problem). Tool vendors therefore choose a point on this trade-off deliberately: Level 3 abstract-interpretation tools bias hard toward soundness (accepting more false positives to guarantee no false negatives for the specific error classes they target); Level 1/2 tools typically bias toward completeness-in-practice and low noise, accepting that some real bugs will be missed, in exchange for a report a human can actually act on. + +> **Important:** knowing which side of this trade-off a specific tool sits on changes how you should read its output. A sound tool's clean report on a specific property (no possible buffer overflow, say) is a genuine guarantee. An unsound tool's clean report is "we didn't find one," which is a much weaker claim, even though both may present as an equally confident-looking green checkmark. + +## 6. Where static analysis sits among the other quality practices + +Static analysis is one layer in a defence-in-depth model, not a replacement for the others. Each layer catches a different class of problem, and -- critically -- each layer has holes that the other layers are positioned to catch instead. + +```mermaid +flowchart LR + A[Compiler warnings] --> F[Shipped code] + B[Static analysis] --> F + C[Code review] --> F + D[Unit/integration testing] --> F + E[Fuzzing/dynamic testing] --> F +``` + +| Practice | What it catches | What it structurally cannot catch | +| --- | --- | --- | +| Compiler warnings | Local syntax/type issues, some uninitialised-use cases | Anything requiring cross-function or whole-programme reasoning | +| Static analysis (lint/dataflow/abstract interpretation) | Logic bugs, undefined behaviour, security-relevant patterns, all reachable paths within its model | Whether the requirements themselves are correct; design-level and architectural problems; anything outside its language/scope model | +| Code review | Design intent, requirement correctness, readability/maintainability, "is this the right approach", domain knowledge a tool has no model of | Subtle multi-step logic errors a reviewer's attention doesn't happen to trace through; scales poorly with reviewer fatigue | +| Unit/integration testing | Confirms specific behaviours work as intended, on the inputs actually tested | Anything outside the test cases written -- a test suite only proves the paths it exercises | +| Fuzzing/dynamic testing (ASan, UBSan, fuzzers) | Real, triggered runtime faults (memory corruption, crashes) on whatever inputs the fuzzer generates | Coverage is probabilistic, not exhaustive -- an unfuzzed path is simply unfuzzed | + +This is sometimes called the "Swiss cheese model" of defence in depth (borrowed from safety engineering): each layer has holes, but the holes in different layers are rarely in the same place, so a problem has to slip through several independent layers simultaneously to reach production. A team that runs only compiler warnings and code review, and calls that "quality assurance," is missing entire classes of problem (deep data-flow bugs, memory-safety issues under specific input combinations) that no amount of careful human review reliably catches, because humans are not good at manually tracing value ranges across dozens of call sites. + +## 7. MISRA and static analysis are not the same thing + +This is worth stating directly because the two get conflated constantly in casual conversation, usually as "we run static analysis, so we're MISRA compliant" or the reverse, "we're MISRA compliant, so static analysis is covered." + +```mermaid +flowchart LR + subgraph SA["Static analysis: a technique"] + A[MISRA C rule checking] + B[CERT C security rule checking] + C[General bug-finding: null derefs, leaks, races] + D[CWE-mapped security pattern detection] + end +``` + +**MISRA is a rule set** -- a specification of which C constructs are permitted, written down independently of any tool, whose rationale and correct application requires the licensed document itself (see the MISRA companion document). **Static analysis is a technique/tool category** that can be pointed at many different rule sets, MISRA being one of them. + +The conflation breaks in both directions: + +- **A static analysis tool running with no MISRA rule set enabled still finds real bugs** -- null pointer dereferences, memory leaks, uninitialised reads -- that have nothing to do with MISRA at all. Static analysis is useful, and used, in codebases that have never adopted MISRA. +- **Running a static analysis tool with MISRA rules enabled does not, by itself, make a codebase MISRA compliant.** As the MISRA companion document covers, a meaningful fraction of MISRA guidelines are classified **undecidable** -- Directive-level guidance requiring judgement, process, or information outside the source text (documenting implementation-defined behaviour relied upon, for instance) that no tool, however sophisticated, can fully automate. A tool's clean report covers only the **decidable, Rule-level** subset it implements; genuine MISRA compliance additionally requires the manual review, deviation records, and Guideline Enforcement Plan process the MISRA Compliance:2020 framework defines. +- **Different tools implement MISRA differently.** As the MISRA companion document notes, tooling divergence is a known, documented limitation -- two "MISRA-compliant" reports from two different tools on the same codebase can legitimately disagree, because the rule text itself does not always reduce to one unambiguous, universally-agreed check. + +The practically important consequence: a codebase can be "clean" under a specific static analysis tool's default configuration while still containing security vulnerabilities the tool's rule set was never configured to look for (see section 8), and a codebase can be fully MISRA-Rule-compliant by a tool's report while still failing genuine MISRA compliance on the undecidable Directives that report never checked. + +## 8. Why this matters for de-risking software security specifically + +Static analysis used for security purposes is usually called **SAST** (Static Application Security Testing) to distinguish the security-focused use case from general code-quality static analysis, though the underlying technique (sections 1-3) is identical -- the difference is which rule set/finding categories are enabled and how findings are triaged. + +The case for static analysis as a security practice, specifically, rests on a few properties dynamic testing and code review don't share: + +- **It runs before the code executes anywhere, including in a test environment.** A vulnerability class like a buffer overflow reachable only under a specific combination of inputs may never get exercised by a test suite or a fuzzer's finite run, but a data-flow-level static analyser reasons about the code path directly, independent of whether any test happens to trigger it. +- **It scales to the whole codebase on every commit,** where a human security review of every line, on every change, is not realistic at normal engineering velocity. +- **It maps directly onto known weakness catalogues.** Modern SAST tools tag findings against CWE (Common Weakness Enumeration) identifiers, which is what lets a finding be prioritised by known real-world severity and cross-referenced against the same taxonomy vulnerability disclosures and CVEs use. +- **The cost of finding a defect shifts dramatically depending on when it's found** -- the "shift left" argument. A buffer overflow found by static analysis in a pull request costs a code change. The same defect found after a firmware image has shipped to a fleet of field devices costs a FOTA campaign (see the LwM2M deep dive's Firmware Update section) across potentially thousands of units on constrained radio links, plus, if it was exploited before the fix shipped, an incident response and disclosure process. + +> **Important:** for connected embedded devices specifically, this isn't an abstract argument -- it's the direct rationale behind the Cyber Resilience Act's emphasis on "security by design" and vulnerability handling covered in the SBOM/CBOM companion document. Static analysis is one of the practical mechanisms that lets a "security by design" requirement mean something concrete in a development process, rather than a phrase in a compliance document with no corresponding engineering practice. + +Static analysis findings and SBOM/CBOM data are complementary, not overlapping: static analysis reasons about the code you wrote; SBOM/CBOM (see the companion document) inventories the components and cryptographic assets you depend on. A vulnerability in your own logic is a static analysis finding; a vulnerability in a third-party library version you shipped is an SBOM query. A fully secure development practice needs both, because neither one sees the other's blind spot. + +## 9. The wider static/dynamic analysis family + +Static analysis and SAST sit alongside several related but distinct categories, worth distinguishing clearly since the acronyms get used loosely: + +| Category | What it analyses | When it runs | +| --- | --- | --- | +| **SAST** (Static Application Security Testing) | Source code, without executing it | Build/CI time | +| **DAST** (Dynamic Application Security Testing) | A running application, from the outside, probing for exploitable behaviour | Against a deployed/staged instance | +| **IAST** (Interactive Application Security Testing) | Instrumented code observed while running real test traffic | During test execution, combining static instrumentation with dynamic observation | +| **SCA** (Software Composition Analysis) | Third-party/open-source dependencies specifically, matched against known vulnerability databases | Build time, or against an existing SBOM | +| **Fuzzing** | A running programme fed randomised/mutated inputs, often paired with sanitizers (ASan, UBSan) to detect memory corruption the moment it occurs | Dedicated fuzzing campaigns, continuous or periodic | + +SCA in particular is worth noting as the direct link back to the SBOM/CBOM companion document: SCA tooling is what actually populates and continuously checks an SBOM against newly disclosed CVEs, rather than the SBOM being a static document someone remembers to re-check manually. + +## 10. Practical considerations for running static analysis well + +- **Baselining on legacy code.** Turning on a serious static analysis tool against an existing, previously unanalysed codebase routinely surfaces thousands of findings. The practical approach -- exactly as the MISRA companion document describes for retrofitting a rule set -- is to baseline the existing findings, gate only *new* code against a "zero new findings" policy, and work through the backlog risk-prioritised rather than attempting to fix everything before shipping anything. +- **False positive management is a first-class cost, not an afterthought.** A tool with a high false-positive rate that nobody triages quickly trains engineers to ignore its output entirely -- at which point it provides zero value regardless of how sound its underlying analysis is. Suppression/waiver mechanisms need the same review discipline as MISRA deviation records: a documented reason, not a silent dismissal. +- **CI gating vs advisory reporting.** Deciding which findings block a merge (typically: anything new, above a severity threshold) versus which are surfaced as advisory-only is a policy decision that determines whether static analysis actually changes what ships, or becomes a report nobody reads. +- **Tool choice should match the level you actually need.** Running a Level 3 abstract-interpretation tool across an entire large codebase for every commit may be prohibitively slow; a common pattern is Level 1/2 tools in every CI run, with Level 3 reserved for specific safety- or security-critical modules where the higher assurance justifies the cost. +- **No single tool covers every level.** A realistic static analysis practice is usually a stack of tools (compiler warnings always on, a Level 2 dataflow tool in CI, MISRA-specific rule checking as its own configured pass, perhaps a Level 3 tool on the most critical modules) rather than one tool assumed to cover everything the term "static analysis" could mean. diff --git a/content/guides/deep-dives/sysvars.md b/content/guides/deep-dives/sysvars.md new file mode 100644 index 0000000..2aa6879 --- /dev/null +++ b/content/guides/deep-dives/sysvars.md @@ -0,0 +1,469 @@ +--- +title: "SysVars: a schema-driven system variable store" +--- + +# SysVars: a schema-driven system variable store + +## 1. Overview + +Section 2.2 of the companion document (*Layered Firmware Architecture, HAL, and RTOS*) introduced "data storage / persistence with callbacks" as an example of a vertical, cross-cutting service -- infrastructure that every horizontal layer of a firmware stack can call into directly, rather than a rung in the layered ladder itself. + +This document covers one well-proven way to build exactly that: a **system variable store** (often shortened to "sysvars"). It is a single, schema-defined data model that every layer can read, write, and subscribe to, with the C code, external API, and even a web UI all generated from one source of truth rather than hand-written and kept in sync manually. + +## 2. The Core Idea + +- Every piece of shared state (a config value, a live sensor reading, a radio setting) is declared once, in a JSON schema, not hand-coded as an ad hoc global or struct field +- A code generator turns that schema into C structs, typed accessor functions, and a registration table +- Any component (application, middleware, HAL driver) can **register a callback** against a specific variable; when that variable is set, every registered callback fires, so producers and consumers are decoupled -- the radio driver that sets a value has no idea who else cares about it +- The schema supports **arrays of instances** of the same structure, so "radio" can be declared once and instantiated as `radio[0]`, `radio[1]`, ... each with its own `frequency`, `mode`, and so on, without duplicating the definition +- Each field can carry **metadata**: help text, units, read-only/writable flags, whether it is exposed over an external API, whether it should appear in a web UI, and how -- and that metadata is what drives the generated API and UI, not separate hand-maintained code + +## 3. From Schema to Generated Artefacts + +```mermaid +graph TD + JSON["JSON Schema - single source of truth\nvariables, types, array/instance counts, help text, API/UI flags"] + + JSON --> Gen[Code Generator] + + Gen --> CStructs["Generated C structs + typed get/set accessors"] + Gen --> CB["Generated callback registration table"] + Gen --> API["Generated external API - REST/RPC surface"] + Gen --> WebUI["Generated Web UI forms - built from the same flags"] + Gen --> Docs["Generated reference docs - from the help text field"] +``` + +This is the same principle as any docs-as-code or schema-first approach: define structure once, generate every downstream artefact from it, so the C code, the API, the UI, and the documentation cannot silently drift apart from each other -- they are all outputs of the same build step, not independently maintained. + +## 4. Runtime Behaviour: Set, Notify, Multiple Subscribers + +```plantuml +@startuml +participant "Radio Driver" as Radio +participant "SysVar Store" as Store +participant "Comms Stack" as Comms +participant "Web UI Backend" as Web + +Comms -> Store: register_callback("radio[1].frequency", on_freq_changed_comms) +Web -> Store: register_callback("radio[1].frequency", on_freq_changed_web) + +Radio -> Store: set("radio[1].frequency", 868100000) +activate Store +Store -> Store: value changed, look up subscribers for this variable +Store -> Comms: on_freq_changed_comms(868100000) +Store -> Web: on_freq_changed_web(868100000) +deactivate Store +@enduml +``` + +Neither the radio driver nor the store itself needs to know how many subscribers exist or what they do with the value; new consumers can subscribe later without the producer changing at all. This is the same decoupling principle a message broker gives you at the system level, applied inside a single piece of firmware. + +## 5. Where It Sits in a Layered Architecture + +A sysvar store is exactly the kind of vertical service described in section 2.2: application code, middleware, and HAL drivers can all set and subscribe to variables directly, and it has its own lifecycle (typically initialised very early, since other init code may depend on default values already being populated). It is a specific, concrete implementation of the "Data Storage / Persistence - callbacks" node shown in the cross-cutting services diagrams earlier. + +## 6. Extending the Schema: Validation, Access Mode, Persistence, and Audit + +The schema is the natural place to declare more than just type and name. Each of the following can be a flag or field on the resource definition, enforced by the generated code rather than trusted to every caller: + +- **min / max / step**: the store rejects an out-of-range or non-step-aligned set before it ever reaches a callback, so range checking is written once, in the generator, instead of duplicated in every producer +- **access mode**: `read_only` (firmware sets it, generated API exposes no setter), `read_write` (normal config), or `write_only` (a command/trigger field, or a secret such as a PIN that should never be readable back out through the generated API) +- **persistent**: whether the value survives a reboot; flagged variables are written to flash-backed storage, ideally batched and committed atomically so a power loss mid-write cannot corrupt the structure +- **audit**: whether every change to this specific variable is appended to a change log (timestamp, old value, new value, and setter identity where available) -- deliberately opt-in per variable, since logging every RSSI update would flood the log while a tariff or PIN change is exactly what you want a durable trail for + +```json +{ + "objects": { + "radios": { + "help": "Radio subsystem settings", + "instances": 2, + "resources": { + "frequency": { + "type": "uint32", + "unit": "Hz", + "min": 863000000, + "max": 870000000, + "step": 100000, + "access": "read_write", + "persistent": true, + "audit": true, + "help": "Carrier frequency for this radio instance" + }, + "mode": { + "type": "enum", + "values": ["idle", "tx", "rx", "sleep"], + "access": "read_write", + "persistent": true, + "audit": true, + "help": "Current operating mode" + }, + "rssi": { + "type": "int16", + "unit": "dBm", + "access": "read_only", + "persistent": false, + "audit": false, + "help": "Last measured received signal strength" + }, + "reset_request": { + "type": "bool", + "access": "write_only", + "persistent": false, + "audit": true, + "help": "Write true to request a radio reset; always reads back false" + } + } + } + } +} +``` + +Because these flags live in the schema, the generated API and web UI can also honour them directly: a `write_only` field never appears with a visible current value in the UI, a `read_only` field renders with no input control, and the min/max/step values populate a slider or spinner control automatically rather than being re-declared by whoever builds the UI. + +## 7. Cardinality: Single vs Array, With a Bound + +The earlier examples used `"instances": N` to mean two different things at once: "this is an array" and "the array has exactly N elements". It is worth pulling those apart explicitly, and worth distinguishing a **fixed** array from a **dynamic** one: + +- **Single**: exactly one instance, always. There is no `[N]` in the path at all (`device.serial_number`, not `device[0].serial_number`). +- **Fixed array**: a known, unchanging count baked in at build time (three status LEDs on a board never becomes four without a hardware respin). The generator sizes a plain C array and there is no notion of instances being added or removed at runtime. +- **Dynamic array**: a build-time capacity ceiling (`max_instances`), with a separate resource tracking how many are actually active right now (paired peripherals, enabled radio channels). The generator still sizes a fixed-length C array of `max_instances` -- this is embedded firmware, not a language with dynamic allocation as the default -- but a live count guards how many of those slots are currently valid. + +```json +{ + "objects": { + "leds": { + "cardinality": "fixed_array", + "max_instances": 3, + "resources": { "state": { "type": "bool", "access": "read_write" } } + }, + "paired_devices": { + "cardinality": "dynamic_array", + "max_instances": 8, + "active_count_resource": "paired_device_count", + "resources": { "address": { "type": "string", "access": "read_only" } } + }, + "device": { + "cardinality": "single", + "resources": { "serial_number": { "type": "string", "access": "read_only" } } + } + } +} +``` + +Making cardinality explicit rather than inferring it from a number also makes the schema self-documenting: a reviewer can see at a glance whether `leds` can ever have a fourth instance added later without checking the generator's rules for what "instances" defaults to. + +## 8. Reusable Named Types + +Repeating `min`/`max`/`step`/`unit` inline on every resource that happens to be a frequency risks one copy drifting from another when a regulatory limit changes. A `types` block, defined once and referenced by name, closes that gap -- the same idea as a C `typedef` or a JSON Schema `definitions`/`$ref` block: + +```json +{ + "types": { + "frequency_hz": { + "base": "int32", + "unit": "Hz", + "min": 863000000, + "max": 870000000, + "step": 100000, + "help": "Sub-GHz ISM band frequency" + }, + "percentage": { + "base": "uint8", + "unit": "%", + "min": 0, + "max": 100, + "step": 1 + }, + "radio_mode": { + "base": "enum", + "values": ["idle", "tx", "rx", "sleep"] + } + }, + "objects": { + "radios": { + "cardinality": "fixed_array", + "max_instances": 2, + "resources": { + "frequency": { "type": "frequency_hz", "access": "read_write", "persistent": true, "audit": true }, + "mode": { "type": "radio_mode", "access": "read_write", "persistent": true, "audit": true } + } + } + } +} +``` + +A resource can still override a field from its named type locally (a specific radio instance with a narrower frequency range than the general type allows), but the common case -- every radio using the same band limits -- is declared once. If the band limits change, one edit to `types.frequency_hz` propagates everywhere the type is used, rather than requiring a find-and-replace across every object. + +## 9. Other Schema Considerations + +A handful of smaller fields are easy to skip early on and awkward to retrofit later, because they affect either the generated C struct layout or how old persisted data is interpreted after a firmware update: + +- **`max_length`** on string types: the generator needs a fixed buffer size (`char ssid[33]`), so this cannot be left implicit +- **`default`**: what a resource resets to on factory reset or before anything has explicitly set it, distinct from a hardcoded zero-initialised struct +- **`precision`**: for floating-point resources, how many decimal places are meaningful for display -- a UI control showing six decimal places of sensor noise is not more informative, just noisier +- **`deprecated`**: when retiring a resource, marking it deprecated rather than deleting it keeps its ID and storage offset reserved, so old persisted data is not silently reinterpreted as a different field's value after an update +- **schema `version`**: a version number on the whole schema (or per object), so the generator or the runtime can detect a mismatch between what's already in flash and what the current firmware expects, and run an explicit migration rather than misreading old data as the new layout + +## 10. Atomic Multi-Variable Transactions + +Setting `radio[1].frequency` and `radio[1].mode` as two separate calls means a subscriber can observe the frequency changed but the mode not yet updated, an inconsistent intermediate state. A transaction API closes this gap: stage several changes, validate them together, then apply and notify as one unit. + +```plantuml +@startuml +participant "Caller" as Caller +participant "SysVar Store" as Store +participant "Subscriber A" as SubA +participant "Subscriber B" as SubB + +Caller -> Store: begin_transaction() +activate Store +Caller -> Store: set("radio[1].frequency", 868100000) +note right of Store: staged, not yet applied\nor visible to readers +Caller -> Store: set("radio[1].mode", "tx") +note right of Store: staged, not yet applied + +Caller -> Store: commit() +Store -> Store: validate all staged values together +Store -> Store: apply all staged values atomically +Store -> SubA: on_change("radio[1].frequency", 868100000) +Store -> SubB: on_change("radio[1].mode", "tx") +deactivate Store +@enduml +``` + +A subscriber that cares about both fields still gets two callbacks, but only after both values are already consistent with each other, and the transaction boundary is the natural place to add cross-field validation that a single variable's min/max/step cannot express on its own (for example, rejecting a `mode="tx"` commit if `frequency` is outside the band permitted for transmission). + +## 11. Hierarchy: Nesting, Path Grammar, and Subscription + +Everything so far has used a flat structure: an object, optionally an instance index, and one of its resources (`radio[1].frequency`). Real schemas often need genuine nesting -- an object containing a sub-object, not just a scalar resource -- and it is worth having an explicit grammar for that rather than letting it emerge ad hoc. + +```json +{ + "objects": { + "radios": { + "cardinality": "fixed_array", + "max_instances": 2, + "resources": { + "frequency": { "type": "frequency_hz", "access": "read_write" } + }, + "objects": { + "antennas": { + "cardinality": "fixed_array", + "max_instances": 2, + "resources": { + "gain_db": { "type": "int8", "access": "read_only" } + } + } + } + } + } +} +``` + +This gives a path like `radios[1].antennas[0].gain_db` -- object, instance, nested object, nested instance, resource -- and the same grammar extends to any depth. Keeping the separator (`.` between object and sub-object, `[N]` for instance index) consistent regardless of nesting depth is what makes the wildcard expansion in the rest of this section, and the REST path mapping in section 12, mechanical rather than something the generator has to special-case. + +With that grammar in place, a callback can subscribe against a pattern instead of an exact path: + +- `radios[*].frequency` -- notify on a frequency change on any radio instance +- `radios[1].*` -- notify on any change under radio instance 1, including nested objects +- `radios[*].antennas[*].gain_db` -- notify on any antenna's gain changing, on any radio +- `radios.*` -- notify on any change anywhere under the radios object + +> **Important:** because the schema is fixed at build time, every concrete variable path the device will ever have is already known to the generator. This means wildcard subscriptions do not need a runtime pattern-matching engine at all -- the generator can expand each wildcard subscription into the exact set of concrete variable IDs it matches, at build time, and the runtime dispatch stays a plain lookup table. This keeps the embedded runtime cost identical to exact-path subscription; the flexibility is entirely a build-time convenience. + +```mermaid +graph LR + Sub["Subscription: radios[*].frequency"] -->|"expanded at build time"| E1["radios[0].frequency"] + Sub -->|"expanded at build time"| E2["radios[1].frequency"] + + E1 --> CB["Single callback, invoked once per matching variable that changes"] + E2 --> CB +``` + +## 12. Mapping to a RESTful API + +If the schema is meant to drive a generated external API from the outset, it is worth designing the path grammar in section 11 so it maps onto REST conventions directly, rather than bolting a URL scheme on afterwards. + +**Naming convention.** Name every object the way its REST collection would be named, not the way a single C struct instance would be: `leds`, `radios`, `sensors`, plural and lower_snake_case, because `GET /leds` returning a collection reads naturally where `GET /led` does not. A `single`-cardinality object is the one exception -- `device` and `network` stay singular, because there is only ever one and no collection is implied. Getting this right in the schema from the start means the internal path grammar and the generated URL scheme use the same names verbatim, rather than the API layer needing its own translation table between an internal singular name and an external plural one. + +**Path mapping.** The dot/bracket grammar used internally translates to a URL path almost mechanically: `radios[1].antennas[0].gain_db` becomes `/radios/1/antennas/0/gain_db`. A request to the object itself without an index (`/radios`) is naturally the collection; a request with an index (`/radios/1`) is naturally that single item. + +**Access mode drives allowed verbs.** This is where the `access` flag from section 6 stops being just documentation and becomes the thing that determines what the generated API actually accepts: + +| Access mode | GET | PUT/PATCH | Notes | +| --- | --- | --- | --- | +| `read_only` | 200 with value | 405 Method Not Allowed | firmware is the only writer | +| `read_write` | 200 with value | 200/204 on success, 400 on a min/max/step violation | normal config path | +| `write_only` | 403 Forbidden or 204 with no body | 200/204 on success | value is never echoed back, e.g. a credential | + +**Cardinality drives whether instances can be created or removed.** A `fixed_array` object exposes no `POST /leds` or `DELETE /leds/2` -- the count is fixed at build time, so only `GET`/`PUT` on existing indices make sense. A `dynamic_array` object can reasonably support `POST /paired_devices` to add an instance (up to `max_instances`) and `DELETE /paired_devices/3` to remove one, with the `active_count_resource` updated as a side effect rather than being independently settable. + +**Transactions map onto a single PATCH.** Rather than inventing separate `begin`/`commit` endpoints over HTTP, a single `PATCH /radios/1` with a JSON body containing both `frequency` and `mode` maps naturally onto the transaction semantics from section 10: the whole request body is validated and applied as one unit, or rejected as one unit, which is exactly what a REST `PATCH` on a resource is supposed to mean. + +**Subscription needs a channel REST doesn't have natively.** Plain REST is pull-based, so the callback/pub-sub model from section 4 needs a separate mechanism: either a `POST /subscriptions` endpoint that registers a webhook URL against a path pattern (including the wildcards from section 11), or a WebSocket/Server-Sent-Events endpoint for push delivery. Either way, the pattern syntax the external API accepts should be the same one used internally, generated from the same schema, rather than a second dialect a client has to learn. + +**The schema itself is worth serving.** Since the API layer runs on hardware far less constrained than the firmware's own runtime, it costs little to expose the schema metadata (type, min/max/step, help text, access mode) at a discovery endpoint, for example `OPTIONS /radios/1/frequency` or a dedicated `/schema` route. This lets external tooling, including a generated web UI, introspect the API at runtime rather than needing its own build-time code generation step -- build-time generation is the right tradeoff inside the firmware, where every byte and cycle matters, but is not a constraint the external API layer needs to inherit. + +## 13. Example: JSON Definition for a Simple System + +The snippet in section 6 showed a single object (`radios`) in isolation. The example below is a larger, more representative schema for a small IoT device, using the plural, RESTful naming convention from section 12 throughout and covering every object type discussed in this document: a singleton, fixed arrays, a reusable named type, genuine object nesting, and resources whose meaning depends on a sibling field. + +```json +{ + "types": { + "percentage": { + "base": "uint8", + "unit": "%", + "min": 0, + "max": 100, + "step": 1 + }, + "frequency_hz": { + "base": "int32", + "unit": "Hz", + "min": 863000000, + "max": 870000000, + "step": 100000, + "help": "Sub-GHz ISM band frequency" + }, + "radio_mode": { + "base": "enum", + "values": ["idle", "tx", "rx", "sleep"] + }, + "hw_binding": { + "base": "string", + "max_length": 24, + "help": "Symbolic BSP identifier for the underlying peripheral or pin, e.g. TIM3_CH1 or GPIOB_PIN3; resolved to an address by the BSP, never a literal register offset in the schema itself" + } + }, + "objects": { + "device": { + "cardinality": "single", + "help": "Identity and status of this device", + "resources": { + "serial_number": { "type": "string", "max_length": 16, "access": "read_only", "persistent": true, "audit": false, "help": "Factory-assigned serial number" }, + "fw_version": { "type": "string", "max_length": 12, "access": "read_only", "persistent": true, "audit": false, "help": "Running firmware version" }, + "uptime_seconds": { "type": "uint32", "unit": "s", "access": "read_only", "persistent": false, "audit": false, "help": "Seconds since last boot" } + } + }, + "leds": { + "cardinality": "fixed_array", + "max_instances": 3, + "help": "Status LEDs", + "resources": { + "state": { "type": "bool", "access": "read_write", "persistent": false, "audit": false, "help": "On/off state of this LED" }, + "brightness": { "type": "percentage", "access": "read_write", "persistent": true, "audit": false, "default": 100, "help": "Brightness of this LED" }, + "hw_binding": { "type": "hw_binding", "access": "read_only" } + } + }, + "sensors": { + "cardinality": "fixed_array", + "max_instances": 2, + "help": "Temperature sensor channels", + "resources": { + "value_c": { "type": "float", "unit": "C", "precision": 1, "access": "read_only", "persistent": false, "audit": false, "help": "Last measured temperature, degrees Celsius" }, + "sample_interval_s": { "type": "uint16", "unit": "s", "min": 1, "max": 3600, "step": 1, "default": 60, "access": "read_write", "persistent": true, "audit": false, "help": "How often this sensor channel is sampled" }, + "hw_binding": { "type": "hw_binding", "access": "read_only", "help": "e.g. I2C1/TMP102" } + } + }, + "radios": { + "cardinality": "fixed_array", + "max_instances": 2, + "help": "Sub-GHz radio subsystems", + "resources": { + "frequency": { "type": "frequency_hz", "access": "read_write", "persistent": true, "audit": true }, + "mode": { "type": "radio_mode", "access": "read_write", "persistent": true, "audit": true }, + "rssi": { "type": "int16", "unit": "dBm", "access": "read_only", "persistent": false, "audit": false, "help": "Last measured received signal strength" }, + "hw_binding": { "type": "hw_binding", "access": "read_only", "help": "e.g. SPI1/CC1200" } + }, + "objects": { + "uart": { + "cardinality": "single", + "help": "Virtual serial port tunnelled over this radio's link, if supported", + "resources": { + "supported": { "type": "bool", "access": "read_only", "help": "Whether this radio instance can carry a serial bridge at all" }, + "baud_rate": { "type": "uint32", "access": "read_write", "persistent": true, "audit": false, "help": "Effective baud rate of the bridged serial stream; meaningless if supported is false" }, + "flow_control_enabled": { "type": "bool", "access": "read_write", "persistent": true, "audit": false, "help": "Software flow control over the radio link; there is no RTS/CTS wiring to bridge, so this is protocol-level, not hardware" } + } + } + } + }, + "serial_ports": { + "cardinality": "fixed_array", + "max_instances": 2, + "help": "Physical UART serial ports", + "resources": { + "baud_rate": { "type": "uint32", "access": "read_write", "persistent": true, "audit": false, "help": "Baud rate" }, + "flow_control_capable": { "type": "bool", "access": "read_only", "help": "Whether RTS/CTS wiring is present on this port" }, + "flow_control_enabled": { "type": "bool", "access": "read_write", "persistent": true, "audit": false, "help": "Hardware flow control enabled; only takes effect if flow_control_capable is true" }, + "hw_binding": { "type": "hw_binding", "access": "read_only", "help": "e.g. USART2" } + } + }, + "network_interfaces": { + "cardinality": "fixed_array", + "max_instances": 2, + "help": "Network connectivity interfaces", + "resources": { + "interface_type": { "type": "enum", "values": ["wifi", "ethernet"], "access": "read_only", "help": "Underlying interface technology" }, + "connected": { "type": "bool", "access": "read_only", "persistent": false, "audit": true, "help": "Whether this interface currently has an active connection" }, + "ip_address": { "type": "string", "max_length": 15, "access": "read_only", "persistent": false, "audit": false, "help": "Current IPv4 address, if connected" }, + "ssid": { "type": "string", "max_length": 32, "access": "read_write", "persistent": true, "audit": true, "help": "Wi-Fi network name; not meaningful when interface_type is ethernet" }, + "password": { "type": "string", "max_length": 63, "access": "write_only", "persistent": true, "audit": true, "help": "Wi-Fi network password; never readable back once set" } + } + }, + "gpios": { + "cardinality": "fixed_array", + "max_instances": 4, + "help": "General-purpose digital IO pins", + "resources": { + "direction": { "type": "enum", "values": ["input", "output"], "access": "read_write", "persistent": true, "audit": false, "help": "Pin direction" }, + "state": { "type": "bool", "access": "read_write", "persistent": false, "audit": false, "help": "Logic level; for an input pin this reflects the sampled value and external writes should be rejected" }, + "hw_binding": { "type": "hw_binding", "access": "read_only", "help": "e.g. GPIOB_PIN3" } + } + }, + "pwm_outputs": { + "cardinality": "fixed_array", + "max_instances": 2, + "help": "PWM output channels", + "resources": { + "duty_cycle": { "type": "percentage", "access": "read_write", "persistent": true, "audit": false, "help": "Output duty cycle" }, + "frequency_hz": { "type": "uint32", "unit": "Hz", "min": 1, "max": 20000, "step": 1, "access": "read_write", "persistent": true, "audit": false, "help": "PWM output frequency" }, + "hw_binding": { "type": "hw_binding", "access": "read_only", "help": "e.g. TIM3_CH1" } + } + }, + "adc_inputs": { + "cardinality": "fixed_array", + "max_instances": 2, + "help": "Analogue-to-digital input channels", + "resources": { + "raw_counts": { "type": "uint16", "access": "read_only", "help": "Last conversion result, raw ADC counts" }, + "voltage_mv": { "type": "uint16", "unit": "mV", "access": "read_only", "help": "Last conversion result, converted to millivolts" }, + "hw_binding": { "type": "hw_binding", "access": "read_only", "help": "e.g. ADC1_IN5" } + } + } + } +} +``` + +A few things worth noticing in this particular schema: + +- The serial-over-radio bridge is now a nested object, `radios[N].uart`, rather than a separate top-level collection cross-referencing a radio by index -- this is the same nesting grammar from section 11, and it is a better fit here because the bridge genuinely belongs to a specific radio instance rather than standing alongside it +- `radios[N].uart.supported` plays the same role as `flow_control_capable` did before: not every radio instance necessarily has bridging wired up, and since every instance of an array (or a nested object under an array) shares the same resource set, a read-only capability flag is how that variation is represented +- `serial_ports` is now only physical UARTs, so it lost the `transport`/`backing_radio_instance` fields entirely -- removing them, rather than leaving them unused, matters once real firmware is generated from this, since an unused field still costs struct space and generated API surface +- `adc_inputs` replaces the earlier PWM-capture channels with actual analogue input: raw counts alongside a converted millivolt value, which is the pairing most consumers actually want (raw for calibration work, converted for everything else) +- every hardware-facing object (`leds`, `sensors`, `radios`, `serial_ports`, `gpios`, `pwm_outputs`, `adc_inputs`) now carries an `hw_binding` resource, using the shared `hw_binding` named type -- a short symbolic string such as `TIM3_CH1` or `GPIOB_PIN3` that names the peripheral or pin, not a raw register address; resolving that symbolic name to an actual address is the BSP's job, exactly as described in the companion document, so the schema stays portable across boards even though `hw_binding`'s *value* will differ per board + +## 14. Design Principles Worth Holding Onto + +- Define the data model once, in the schema, and generate the C structs, API, UI, and documentation from it; hand-maintaining the same structure in three places is where those three things quietly drift apart +- Make cardinality explicit (`single`, `fixed_array`, `dynamic_array`) rather than inferring it from a bare instance count; a reviewer should be able to tell whether an array can ever grow without checking the generator's defaults +- Reuse named types for anything declared more than once; one edit to a shared type should be able to propagate everywhere it is used, rather than requiring a search across every object +- Keep `audit` opt-in per variable, not a global switch; a log that captures everything is as hard to use as no log at all +- Treat `min`/`max`/`step` as enforced contracts, not documentation -- the store should reject an invalid set outright, rather than trusting every caller to have checked first +- Reach for a transaction when two or more variables must never be observed in a half-updated state relative to each other +- Remember that wildcard subscriptions cost nothing at runtime on an embedded target, provided the schema is fixed at build time and the generator expands them to concrete IDs ahead of time +- Design the internal path grammar and the external REST path scheme together, from the same schema, rather than treating the API as an afterthought bolted onto an already-fixed internal addressing scheme +- Name every object as its REST collection would be named (plural, lower_snake_case), with `single`-cardinality objects as the only deliberate exception +- Where one field's meaning depends on another (a port's flow control depending on its transport, a pin's writability depending on its direction), the schema can document the relationship, but enforcing it is a validator's job, not something a static `access` flag alone can carry +- Where a resource belongs to a specific hardware instance rather than the device as a whole, nest it under that instance rather than cross-referencing it by index from a separate top-level collection +- Keep `hw_binding` symbolic (a name the BSP resolves), never a literal register address in the schema itself -- the schema should not need to change when a board respin moves a peripheral to a different pin diff --git a/content/guides/defect-management-and-debugging.md b/content/guides/defect-management-and-debugging.md new file mode 100644 index 0000000..81c402e --- /dev/null +++ b/content/guides/defect-management-and-debugging.md @@ -0,0 +1,272 @@ +--- +title: Defect management and debugging +--- + +# Defect management and debugging + +This guideline defines the engineering standards for reporting, triaging, diagnosing, and resolving software defects across embedded firmware, edge devices, and cloud infrastructure. It establishes rigorous debugging disciplines, root cause analysis (RCA) frameworks, and blameless post-mortem procedures. + +## Purpose and Scope + +Unstructured defect reports and undisciplined "trial-and-error" debugging waste engineering effort, extend outage durations, and frequently introduce secondary bugs into production code. + +This standard establishes: +- Standardised defect creation criteria (DC-1 to DC-4). +- A unified Severity versus Priority classification matrix. +- A systematic 38-point debugging checklist (DBG-1 to DBG-38). +- Root Cause Analysis (RCA) methodologies and blameless post-mortem templates. + +## 1. Defect Creation Standards + +Every defect logged in the engineering issue tracker (Jira / GitHub Issues) MUST adhere to the four defect creation rules: + +### DC-1: Define the System Context and Environment + +The defect report MUST precisely specify the target execution environment: +- **Target Platform & Architecture**: Target hardware variant, processor architecture (for example, Nordic nRF52840, STM32H7, x86-64). +- **Software Versions**: Exact firmware version string, Git commit SHA, OS/RTOS version (for example, FreeRTOS v10.4.3, Zephyr v3.5.0, Ubuntu 22.04 LTS). +- **Configuration Parameters**: Any non-standard compiler flags, configuration settings, or feature toggles enabled during reproduction. +- **Workflow State**: The operational state of the system immediately preceding the failure. + +### DC-2: Research and Reproduce the Root Cause + +The author MUST perform preliminary investigation prior to submitting the ticket: +- **Reproducibility**: Retest the failure sequence until an exact, deterministic step-by-step reproduction recipe is established. +- **Intermittency**: If the issue is non-deterministic (for example, race condition or timing jitter), state the empirical reproduction frequency (for example, "occurs in approximately 3 out of 100 power cycles"). +- **Boundary Isolation**: Narrow down which subsystem triggers the fault by examining error return codes and system audit logs. + +### DC-3: Attach Concrete Supporting Evidence + +Defect reports MUST include direct diagnostic evidence rather than verbal assertions: +- Serial console logs, syslog captures, or cloud distributed traces (with correlation IDs). +- Logic analyser captures (sigrok/Saleae) or Wireshark PCAPs for communication bus or networking faults. +- Core dump summaries, register state dumps (PC, LR, SP, HardFault status registers), or stack backtraces. +- Visual evidence (screenshots, annotated diagrams, or step recorder recordings). + +### DC-4: Format Objectively Without Speculation or Blame + +All text MUST adhere to professional, objective technical reporting: +- Maintain strict factual neutrality: describe what happened versus what was expected. +- Avoid emotional language, opinions, speculation, or assigning individual blame. +- Use structured headings: **Summary**, **Environment**, **Preconditions**, **Steps to Reproduce**, **Expected Result**, **Actual Result**, and **Attachments**. + +### Standard Defect Report Template + +```markdown +### Summary +LTE Modem fails to reconnect after waking from Deep Sleep when network signal is marginal (-115 dBm). + +### Environment +- **Device**: Smart Gateway Rev C (Board ID: GW-REV-C3) +- **Firmware Version**: v2.4.1 (Git commit: `a3b7c89f`) +- **Modem Firmware**: Quectel BG95-M3 (Revision: BG95M3LAR02A03) +- **RTOS**: FreeRTOS v10.4.3 + +### Preconditions +1. Device provisioned with valid SIM card on Band 20. +2. RF attenuator adjusted to simulate weak cellular signal (-115 dBm RSRP). + +### Steps to Reproduce +1. Command device into deep sleep via CLI: `system sleep --duration 300` +2. Wait 300 seconds for RTC wakeup timer to expire. +3. Observe UART diagnostic console during modem power-rail re-initialisation. + +### Expected Result +Modem driver re-initialises UART communication, registers with the cellular tower within 45 seconds, and transitions to connected state. + +### Actual Result +Modem power pin toggles correctly, but AT command response times out after 10,000 ms. The driver enters an infinite retry loop, depleting the backup battery within 4 hours. + +### Supporting Evidence +- Console log: [console_dump_reconnect_fail.log] +- Oscilloscope capture: [power_rail_glitch_ch1.png] +- Logic analyser trace: [uart_at_timeout.sr] +``` +[[CAPTION:Figure]] Standard defect report example for embedded hardware/firmware integration. + +## 2. Severity and Priority Classification + +Defects MUST be classified across two independent dimensions: **Severity** (technical impact on the system) and **Priority** (urgency of commercial resolution). + +### 2.1 Severity Levels + +| Severity | Label | Definition | Technical Criteria | +| :--- | :--- | :--- | :--- | +| **Sev-1** | Critical / Outage | Total system stoppage or catastrophic data corruption. | Device crash, boot loop, hard fault, security vulnerability exploited in wild, or complete cloud service unavailability with no workaround. | +| **Sev-2** | Major / Degraded | Severe impairment of core functionality. | Primary business capability or sensor telemetry stream non-functional; partial data loss; workaround requires significant manual intervention. | +| **Sev-3** | Moderate | Non-critical functionality broken; workaround exists. | Secondary feature failure (for example, diagnostic LED sequence incorrect, non-essential report failure); system continues operating safely. | +| **Sev-4** | Minor / Cosmetic | Negligible operational impact. | Typographical error in logs, minor web portal UI alignment defect, non-conforming variable name in debug build. | + +[[CAPTION:Table]] Technical severity levels and criteria. + +### 2.2 Priority Levels and SLAs + +| Priority | Label | Target Resolution SLA | Release Mechanism | +| :--- | :--- | :--- | :--- | +| **P1** | Blocker | Work begins immediately; continuous effort until resolved. | Dedicated hotfix release deployed out-of-band. | +| **P2** | High | Resolved within current active sprint. | Included in the next scheduled minor release. | +| **P3** | Medium | Scheduled into backlog for upcoming release cycle. | Standard sprint planning backlog. | +| **P4** | Low | Resolved as opportunistic maintenance when adjacent code is modified. | Future release candidate. | + +[[CAPTION:Table]] Priority levels and resolution service-level agreements. + +## 3. Systematic Debugging Discipline + +Engineers MUST approach defect resolution methodically, following the 38-point debugging checklist adapted from classic software engineering practices: + +### 3.1 General Debugging Discipline + +| ID | Checklist Question | +| :--- | :--- | +| **DBG-1** | Have you set a maximum time limit for quick initial debugging before stepping back to formulate a structured plan? | +| **DBG-2** | Have you checked the issue tracker, knowledge base, and commit history to ensure the issue has not already been diagnosed? | +| **DBG-3** | Are you utilising all available debugging tools (hardware debuggers, JTAG/SWD, compiler warnings, static analysers, sanitizers)? | +| **DBG-4** | Do you operate on the default assumption that the defect resides in your own code rather than the compiler, OS, or hardware? | +| **DBG-5** | Have you strictly eliminated superstitious, trial-and-error changes made without understanding root cause? | +| **DBG-6** | Are you using multiple complementary techniques (differential debugging, binary search, assertions, execution traces)? | +| **DBG-7** | Do you solicit advice and code inspection from peers when progress stalls? | +| **DBG-8** | If diagnosis exceeds the initial estimate, have you communicated the delay to project stakeholders? | +| **DBG-9** | Do you know when to step back, take a break, and allow subconscious problem-solving to operate? | +| **DBG-10** | Do you verify that the proposed fix actually resolves the root problem rather than masking symptoms? | +| **DBG-11** | Do you document the root cause and learning points to prevent recurrence across the team? | + +[[CAPTION:Table]] General debugging discipline checklist items. + +### 3.2 Techniques for Finding Errors + +| ID | Diagnostic Technique | +| :--- | :--- | +| **DBG-12** | Formulate an explicit hypothesis based on observed data before executing the next test run. | +| **DBG-13** | Refine and minimise the test case until it contains the smallest reproducible sequence of steps. | +| **DBG-14** | Attempt to reproduce the error through different pathways or operating conditions. | +| **DBG-15** | Generate additional telemetry data and register inspections to validate or invalidate competing hypotheses. | +| **DBG-16** | Evaluate negative tests: identify what does *not* fail to narrow the search space. | +| **DBG-17** | Brainstorm alternative hypotheses if initial assumptions fail validation. | +| **DBG-18** | Narrow the suspicious code region using binary search (git bisect or commenting out submodules). | +| **DBG-19** | Inspect code modified recently in git commit history. | +| **DBG-20** | Expand the inspection window if the defect involves corrupted memory or delayed side effects. | +| **DBG-21** | Isolate components and test them incrementally in a synthetic test harness. | +| **DBG-22** | Pay particular attention to routines that have historically exhibited high defect densities. | +| **DBG-23** | Use systematic brute-force instrumented logging when interactive debuggers cannot capture timing issues. | +| **DBG-24** | Check for classic failure modes: off-by-one errors, uninitialised pointers, stack overflows, integer truncation. | +| **DBG-25** | Explain the problem aloud line-by-line to a colleague or inanimate object ("rubber duck debugging"). | +| **DBG-26** | Take a deliberate break to reset cognitive perspective when stuck. | + +[[CAPTION:Table]] Techniques for finding errors checklist items. + +### 3.3 Techniques for Fixing Errors + +| ID | Remediation Practice | +| :--- | :--- | +| **DBG-27** | Ensure you thoroughly understand the root problem before touching any line of production source code. | +| **DBG-28** | Check the proposed solution against architectural guidelines and quality standards. | +| **DBG-29** | Write a failing automated regression test that reproduces the defect before applying the fix. | +| **DBG-30** | Maintain composure; avoid rushing out unverified "hotfixes" under production pressure. | +| **DBG-31** | Confirm that your diagnosis explains all observed symptoms, not merely a subset. | +| **DBG-32** | Preserve the original failing branch and log captures for post-incident review. | +| **DBG-33** | Fix the underlying architectural or algorithmic defect, not merely the immediate symptom. | +| **DBG-34** | Change code only with clear, defensible architectural justification. | +| **DBG-35** | Understand the wider system context to ensure the fix does not break downstream invariants. | +| **DBG-36** | Make one atomic change at a time; test and verify between changes. | +| **DBG-37** | Search the entire codebase for identical or similar flawed code patterns. | +| **DBG-38** | Re-run full automated regression test suites to guarantee zero unintended side effects. | + +[[CAPTION:Table]] Techniques for fixing errors checklist items. + +## 4. Root Cause Analysis (RCA) and Blameless Post-Mortems + +For every Sev-1 and Sev-2 incident, the team MUST carry out a Root Cause Analysis (RCA). It MUST publish a blameless postmortem within 5 working days of the incident being resolved. + +### 4.1 The 5-Whys Methodology + +Teams must trace failures back to underlying systemic, procedural, and cultural causes rather than stopping at the immediate technical trigger: + +```mermaid +flowchart TD + A[Symptom: IoT Gateways Disconnected from Cloud] -->|Why?| B[1. Cellular connection timed out] + B -->|Why?| C[2. Modem DNS queries failed] + C -->|Why?| D[3. DNS server IP changed by telco] + D -->|Why?| E[4. Gateway had hardcoded static DNS instead of DHCP] + E -->|Why?| F[Root Cause: No automated configuration test for dynamic carrier parameters] +``` +[[CAPTION:Figure]] 5-Whys root cause analysis tree. + +### 4.2 Ishikawa (Fishbone) Diagram + +Complex distributed faults must be analysed across the five core engineering dimensions: + +```mermaid +flowchart LR + subgraph Environment + E1[Poor Cellular RSSI] + E2[Thermal Throttling] + end + subgraph Code + C1[Unbounded Buffer] + C2[Missing Null Check] + end + subgraph Process + P1[Skipped Peer Review] + P2[Inadequate Soak Testing] + end + subgraph Tooling + T1[Compiler Optimisation Bug] + T2[CI Runner Timeout] + end + Environment --> Fault[System Crash] + Code --> Fault + Process --> Fault + Tooling --> Fault +``` +[[CAPTION:Figure]] Ishikawa fishbone root cause categorisation. + +### 4.3 Blameless Post-Mortem Template + +```markdown +# Blameless Post-Mortem: [INCIDENT-ID] - [Brief Title] + +## Executive Summary +- **Date of Incident**: [YYYY-MM-DD] +- **Duration of Outage**: [X hours, Y minutes] +- **Severity**: [Sev-1 / Sev-2] +- **Impacted Systems**: [List of services, firmware versions, or customer cohorts] +- **Lead Investigator**: [Name] + +## Incident Timeline (All times in UTC) +- **14:02**: Firmware OTA campaign initiates deployment of v2.4.1 to batch 1 (500 units). +- **14:15**: Automated cloud alerts trigger: disconnect rate exceeds 20% threshold. +- **14:22**: On-call engineer initiates incident triage; halts OTA rollout. +- **15:05**: Hardware debugger connects to laboratory unit; HardFault identified in modem driver. +- **16:30**: Patch verified in lab; hotfix v2.4.2 compiled and validated. +- **17:15**: Hotfix dispatched to impacted fleet; telemetry streams recover. + +## Root Cause +Modem re-initialisation routine assumed valid UART response within 100 ms. In low-signal environments, the cellular modem required up to 450 ms to respond, causing a null pointer dereference in the response parser. + +## What Went Well +- Automated disconnect alerts triggered within 13 minutes of rollout. +- Staged deployment policy contained the failure to batch 1 (only 500 of 50,000 units affected). +- Rollback was executed promptly without cascading database failure. + +## What Went Wrong +- Pre-release soak testing was conducted solely under optimal laboratory RF signal conditions (-65 dBm). +- Unit test suite mocked UART responses synchronously with zero latency. + +## Action Items +| ID | Action Item | Owner | Target Date | Tracking Key | +| :--- | :--- | :--- | :--- | :--- | +| **ACT-1** | Add RF attenuation sweep (-60 to -115 dBm) to automated HIL test matrix. | [Name] | 2026-10-15 | HW-402 | +| **ACT-2** | Implement asynchronous timeout protection and watchdog kick in modem driver. | [Name] | 2026-09-30 | FW-891 | +| **ACT-3** | Update the defect review checklist to mandate latency injection in mocks. | [Name] | 2026-10-05 | DOC-112 | +``` +[[CAPTION:Figure]] Blameless post-mortem report template. + +## Related Documents + +- [Design Checklist Guide](../templates/design-review-checklist.md) +- [Code Review Checklist](../templates/code-review-checklist.md) +- [C Coding Standards](../standards/c-coding-standard.md) +- [Code Review Guide](code-review.md) +- [Architecture Review Checklist](../templates/architecture-review-checklist.md) +- [Fundamental Software Architecture Principles](../standards/software-architecture-principles.md) +- [API Design and Error Management Standards](../standards/api-design.md) diff --git a/content/guides/git/01-how-git-works.md b/content/guides/git/01-how-git-works.md new file mode 100644 index 0000000..42cb92d --- /dev/null +++ b/content/guides/git/01-how-git-works.md @@ -0,0 +1,1977 @@ +--- +title: How Git works +--- + +# How Git works + +*Written for engineers who want to understand Git rather than memorise commands.* + +## Mental Model: What Git Is + +A lot of pain with Git comes from thinking about it as: + +- A timeline of commits +- A central server and client +- Files changing over time + +This is wrong (or at least not the useful view). + +The right mental model: + +### Git Is a Content-Addressable Database of Snapshots, Connected Into a Directed Acyclic Graph (DAG) + +Let’s unpack that without getting academic. + +### Git stores snapshots, not diffs + +When you commit: + +- Git stores **what your project looks like now**, not a diff from the previous commit. +- Internally, each commit points at a **tree** (your folder hierarchy). +- Trees point at **blobs** (file contents). +- The entire structure is immutable. Once stored, Git never edits it; it adds new objects. + +This is why: + +- Rewriting history does not break the past; it makes **new commits**. +- Rollback doesn’t delete old work; it points somewhere else. + +### The DAG: Directed Acyclic Graph + +This matters more than people realise. + +Git commits form a DAG: + +- Directed: each commit points to one or more parents +- Acyclic: no cycles; a commit never points “forward” +- Graph: not a straight line; merges create multiple parents, and branching creates diverging nodes + +If you zoom out, a DAG with a branch and a merge looks like this: + +```mermaid +gitGraph + commit id:"dag-C0" + commit id:"dag-C1" + branch feature + checkout feature + commit id:"dag-C2'" + checkout main + commit id:"dag-C2" + merge feature id:"dag-M" + commit id:"dag-C3" +``` +[[CAPTION:Figure]] A branch and a merge in the commit DAG + +People imagine this instead, as a straight line: + +```mermaid +flowchart LR + C1 --> C2 --> C3 +``` +[[CAPTION:Figure]] The commonly (and incorrectly) imagined linear history + +The DAG is not linear. It’s a **web of snapshots**. + +This is useful because branching and merging are cheap: you are **moving pointers**. + +- A **branch** is a label pointing at a commit +- `HEAD` is a pointer to “your current branch” +- Moving branches does not rewrite the DAG; it moves the label + +That is incredibly powerful. + +### The DAG explained step-by-step + +Let’s illustrate two developers working: + +```mermaid +gitGraph + commit id:"tw-C0" + commit id:"tw-C1" + branch devA + checkout devA + commit id:"tw-A1" + commit id:"tw-A2" + checkout main + branch devB + checkout devB + commit id:"tw-B1" + checkout main + merge devA id:"tw-M1" +``` +[[CAPTION:Figure]] Two developers branching and merging into main + +Important details: + +#### Commit IDs are hashes + +The commit hash is **content + metadata**: + +`commit = hash(tree + parents + message + timestamp + author...)` + +Change anything → new hash. + +That’s why rebase creates **new commits**:\ +You change the commit parent → different hash. + +#### No central truth + +With Git, there isn’t one single “timeline”.\ +There are many possible paths. + +- `main` is a pointer **your team agrees** is the official one +- Feature branches are different paths in the DAG + +#### Merge = “join two DAG paths” + +- Git finds the **merge base** +- Calculates diffs from base → both branches +- Tries to apply both into a **new merge commit** + +This is three-way merge, which is far smarter than diffing two files. + +### HEAD, branch pointers, and detached HEAD + +- `HEAD` points to the **current branch** +- That branch pointer points to a **commit** +- That commit points to a tree + +Example: + +```mermaid +flowchart LR + HEAD --> main["refs/heads/main"] --> C5 +``` +[[CAPTION:Figure]] HEAD, a branch pointer, and the commit it references + +When you `checkout ` (not branch): + +```mermaid +flowchart LR + HEAD -.-> C3 +``` +[[CAPTION:Figure]] Detached HEAD pointing directly at a commit + +That’s “detached HEAD”: `HEAD` points straight at the commit, with no branch label between.\ +You’re not on a branch.\ +If you commit from here, the commit is **unreferenced** until you create a branch. + +That’s why people “lose” commits: they make them where no pointer references them. + +Reflog solves that later. + +## Working Model: Local vs Remote + +Git is **distributed**: + +- Your local `.git` is a full database +- Remotes are **other copies** +- `push` and `pull` are **synchronisation events** + +You don’t “check files out from the server”.\ +You **replicate the DAG**, then collaborate via branch pointers. + +## Zones: Working directory, Index, Repository + +Understanding these is essential: + +### Working Directory + +Your actual files. + +### Index (Staging Area) + +Snapshot you want to commit **next**. + +### Repository + +All commits and history in `.git`. + +Cycle: + +```mermaid +flowchart LR + W["Working directory"] -->|"git add"| I["Index"] -->|"git commit"| R["Repository"] -->|"git push"| O["Remote"] +``` +[[CAPTION:Figure]] The working directory, index, and repository cycle + +This explains why: + +- `git commit` does not commit **everything changed**, only what you staged. +- You can stage part of a file. + +## Basic workflow with explanation + +```bash +git status +git add file.c +git commit -m "Fix A" +git push origin main +``` + +Nothing magical.\ +No server-side locking.\ +No checkout/in/out.\ +No central timeline. + +## Branching in Depth + +### Branches Are Pointers + +A branch is: + +```mermaid +flowchart LR + fx["refs/heads/featureX"] --> C7 +``` +[[CAPTION:Figure]] A branch as a pointer to a commit + +When you commit, Git: + +- Makes a new commit +- Moves `featureX` to the new commit + +That’s it. + +### Creating and switching + +```bash +git checkout -b feature/api +``` + +This creates: + +```mermaid +flowchart LR + HEAD --> fa["feature/api"] --> C3 +``` +[[CAPTION:Figure]] HEAD following a newly created feature branch + +Commits now extend the **feature branch**, not main. + +## Merging: Detailed Explanation + +Merging is not file copying. + +### Merge base + +To merge `A` and `B`: + +1. Find their **last common commit** (merge base). +1. Calculate changes from base → A +1. Calculate changes from base → B +1. Apply both to base +1. If conflicts → ask you + +Example: + +```mermaid +gitGraph + commit id:"mb-A" + commit id:"mb-M" + branch feature + checkout feature + commit id:"mb-C2" + commit id:"mb-C3" + checkout main + commit id:"mb-C1" +``` +[[CAPTION:Figure]] Merge base of two diverged branches + +Merge base = `M`. + +This is why you can merge branches even if they diverged months ago. + +### Merge types detailed + +#### Fast-forward merge + +History is linear, with no divergence. Feature branched off `B`, and `main` has not moved since: + +```mermaid +gitGraph + commit id:"fa-A" + commit id:"fa-B" + branch feature + checkout feature + commit id:"fa-C" + commit id:"fa-D" +``` +[[CAPTION:Figure]] Two branches before a fast-forward merge + +Since `main` did not move, Git moves the `main` pointer forward to `D`; no new commit is created: + +```mermaid +gitGraph + commit id:"fb-A" + commit id:"fb-B" + commit id:"fb-C" + commit id:"fb-D" tag:"main" +``` +[[CAPTION:Figure]] State after a fast-forward merge + +#### Merge commit + +Both `main` and `feature` moved since they diverged, so Git creates a new merge commit `M` with two parents: + +```mermaid +gitGraph + commit id:"mg-A" + commit id:"mg-B" + branch feature + checkout feature + commit id:"mg-C2" + checkout main + commit id:"mg-C3" + merge feature id:"mg-M" +``` +[[CAPTION:Figure]] A merge commit joining two diverged branches + +Merge creates new commit `M`. + +### No fast-forward merge (`--no-ff`) + +Teams often want to preserve branch context.\ +So they **force a merge commit** even if fast-forward is possible. + +This keeps feature branches visible in history. + +### Merge conflicts + +Conflicts happen when: + +- Same line changed +- Adjacent lines changed +- File moved in one branch and edited in another +- Binary files can’t be merged + +When Git cannot reconcile both sides automatically, it writes conflict markers directly into the file: + +```text +<<<<<<< HEAD +int retries = 3; +======= +int retries = 5; +>>>>>>> feature/tuning +``` + +- Everything between `<<<<<<< HEAD` and `=======` is **your current branch's version**. +- Everything between `=======` and `>>>>>>> feature/tuning` is the **incoming branch's version**. +- You edit the file to keep the version you want (or blend both), then delete all three marker lines. + +Conflict resolution is not about guessing: it is about **intent**. + +Good practice: + +- Resolve conflicts **one commit at a time** +- Do not keep one side without checking it +- Compile and test after merge + +## Merge Strategies + +`git merge` picks a strategy automatically based on how many branches you are merging, but you can choose one explicitly with `git merge -s `. This is a different question from the fast-forward or merge-commit outcome described above. The strategy is the *algorithm* Git uses to combine the branches, not the shape of the resulting history. + +| Strategy | What it does | When to use | +| --------------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------- | +| `ort` | The default since Git 2.34. A three-way merge that detects renames and handles criss-cross merge bases better than its predecessor. | Almost always; this is what runs when you type plain `git merge`. | +| `recursive` | The previous default three-way merge algorithm. Still available and still correct, just superseded by `ort`. | Rarely chosen deliberately; occasionally useful for compatibility with older tooling. | +| `--ff-only` | Not a merge strategy as such, but a flag that refuses to create a merge commit: it only succeeds if the merge could fast-forward. | Enforcing a linear history on shared branches, or checking whether a merge would be trivial before doing it. | +| `ours` (`-s ours`) | Records a successful merge but keeps your current branch's tree entirely, discarding every change from the other side. | Deliberately marking a branch as merged without taking its content, for example retiring an old branch while preserving its history in the graph. | +| `octopus` | Merges more than two branches in a single commit. Chosen automatically when you run `git merge branchA branchB branchC`. Has no built-in conflict resolution: it fails outright if there is any conflict. | Combining several independent, low-risk topic branches at once, typically by a release manager. | +| `resolve` | An older two-branch three-way merge algorithm, simpler and more conservative than `recursive`/`ort`, without rename detection. | Rarely needed today; kept for compatibility and as a fallback for edge cases `ort` handles poorly. | +| `subtree` | A variant of the recursive/ort algorithm that treats one project's history as if it were always a subdirectory of the other, adjusting paths on the fly. | Merging in another project's history at a specific path; this is the strategy `git subtree` (below) uses internally. | +[[CAPTION:Table]] Git merge strategies and when each applies + +### Strategy Options vs Strategy + +Do not confuse the strategy itself with `-X` **strategy options**, which tune the `ort`/`recursive` algorithm's conflict resolution rather than replacing it: + +```bash +git merge -X ours feature/x # on a per-hunk conflict, prefer our side; still a real three-way merge +git merge -X theirs feature/x # on a per-hunk conflict, prefer their side; still a real three-way merge +git merge -s ours feature/x # ignore the other branch's changes entirely, for every file +``` + +`-X ours` only wins conflicting hunks in your favour and still incorporates the other branch's non-conflicting changes. `-s ours` is far more drastic: nothing from the other branch reaches the working tree at all, only a merge commit recording that it happened. + +## Rebase: Detailed View + +Rebase is misunderstood. + +### What Rebase Does + +Rebase: + +1. Takes your commits +1. Temporarily removes them +1. Moves branch pointer to another base +1. Replays commits one by one: + +```mermaid +flowchart TD + base --> F1p["F1'"] --> F2p["F2'"] +``` +[[CAPTION:Figure]] Commits replayed onto a new base during rebase + +These are **new commits**.\ +Original ones remain in reflog until pruned. + +### Why Rebase + +For feature work: + +- You want a clean story: “main → my work” +- Without messy merge commits + +Rebase lets: + +- Code review be easier +- Bisection simpler +- History readable + +#### When NOT to rebase + +Never rebase a branch that others already use.\ +You change commit IDs → their work breaks. + +Rule: + +- Local = rebase ok +- Shared = prefer merge + +## Log and Reflog: Powerful Inspection + +### `git log`: Reading the DAG + +Examples: + +```bash +git log --oneline --graph --decorate --all +``` + +This shows: + +- Branch pointers +- Tags +- Merge structure + +Also helpful: + +```bash +git log --patch # show diffs +git log --stat # file changes +git log --first-parent # follow merges on main +git log path/to/file.c # history of a file +``` + +`--first-parent` is brilliant for release history. + +### `git reflog`: The Time Machine + +Reflog tracks **pointer movements**: + +- HEAD moved +- Branch moved +- Checkout happened +- Reset happened + +Even after you think you “deleted” commits, they are in reflog. + +```bash +git reflog +``` + +Recovery example: + +```bash +git reset --hard +``` + +This is **how you recover broken history**. + +## Bisect: Finding the Commit That Broke Something + +`git bisect` performs a binary search across the DAG to find the commit that introduced a regression. It is invaluable when you know "it worked at some point" but not where it stopped. + +### How it works + +1. Tell Git a known-bad commit (usually `HEAD`) and a known-good commit (an older tag or commit). +1. Git checks out a commit halfway between the two. +1. You test it and tell Git `good` or `bad`. +1. Git halves the remaining range and repeats. +1. After `log2(n)` steps, Git identifies the exact first-bad commit. + +```mermaid +flowchart LR + good["good commit"] --> mid1["?"] --> mid2["?"] --> bad["bad commit (HEAD)"] +``` +[[CAPTION:Figure]] Binary search narrowing to the first bad commit + +### Manual bisect + +```bash +git bisect start +git bisect bad # HEAD is broken +git bisect good v1.4.0 # this tag was fine +# Git checks out a midpoint commit +# ...build and test... +git bisect good # or: git bisect bad +# repeat until Git reports the first bad commit +git bisect reset # return to where you started +``` + +### Automated bisect + +If the regression can be detected by a script or test command, let Git drive the whole search: + +```bash +git bisect start HEAD v1.4.0 +git bisect run npm test +``` + +Git runs the command at each step and interprets a non-zero exit code as `bad`, zero as `good`, so no manual testing is required. + +## Pull Requests: Real-World Workflow + +PRs are workflow, not Git. But they use Git theory. + +Modern development flow: + +### Creating a PR + +1. Branch: + +```bash +git checkout -b feature/mag-tamper +``` + +1. Commit locally until complete. +1. Pull latest main into branch: + +```bash +git fetch origin +git rebase origin/main # or git merge origin/main +``` + +1. Push branch: + +```bash +git push -u origin feature/mag-tamper +``` + +1. Create PR: + + - Source: `feature/mag-tamper` + - Target: `main` (or `develop`) + +### Good PR hygiene + +A good PR: + +- Is **small** +- Solves one problem +- Has clear title: what + why +- Contains description: + - Motivation + - Changes + - Testing + - Impact + - Steps to verify + +### Review etiquette + +Rules for reviewers: + +- Review code, not people +- Ask questions before demands +- Suggest alternatives +- Provide rationale +- Don’t bike-shed + +Rules for authors: + +- Respond with respect +- Document decisions +- Don’t ignore comments +- Avoid making PRs “3-in-1” refactors, features, formatting + +### Merge options in PR UI + +You usually get three options for combining a pull request. These are different from both the fast-forward or merge-commit outcome and the `ort`/`recursive`/`ours` algorithm choice covered earlier. This is the menu a PR tool (GitHub, GitLab and similar) offers when you click merge. Starting from the same point, `main` at `A`, with `feature` carrying two commits: + +```mermaid +gitGraph + commit id:"ps-A" + branch feature + checkout feature + commit id:"ps-F1" + commit id:"ps-F2" +``` +[[CAPTION:Figure]] Starting point shared by all three PR merge options + +#### Merge Commit (PR Option) + +Preserves branch history: every commit from `feature` stays, joined by a new merge commit with two parents. + +```mermaid +gitGraph + commit id:"pm-A" + branch feature + checkout feature + commit id:"pm-F1" + commit id:"pm-F2" + checkout main + merge feature id:"pm-M" +``` +[[CAPTION:Figure]] Resulting history from a merge commit + +A good choice when context matters. + +#### Squash and merge + +Combines all of `feature`'s commits into a single new commit on `main`. The individual `F1`/`F2` commits and the branch structure are gone from `main`'s history. + +```mermaid +gitGraph + commit id:"pq-A" + commit id:"pq-Squashed F1+F2" +``` +[[CAPTION:Figure]] Resulting history from squash and merge + +Good for messy feature history. + +#### Rebase and merge + +Replays `feature`'s commits one by one onto the tip of `main` as new commits, then fast-forwards, with no merge commit at all. + +```mermaid +gitGraph + commit id:"pd-A" + commit id:"pd-F1'" + commit id:"pd-F2'" +``` +[[CAPTION:Figure]] Resulting history from rebase and merge + +Linearises history. Good for clean timelines. + +#### Choosing between them + +| Option | Resulting history | Individual commits kept? | Good For | +| ---------------- | ------------------------------------------- | ------------------------- | -------------------------------------------- | +| Merge commit | Branch shape preserved, plus a merge commit | Yes, unchanged | Preserving how and when work happened | +| Squash and merge | One new commit on `main` | No, collapsed into one | Messy or WIP feature history you do not want on `main` | +| Rebase and merge | Linear, feature commits replayed onto `main`| Yes, but with new commit IDs | A clean, bisectable, linear timeline | +[[CAPTION:Table]] Comparison of the three pull request merge options + +Team should have a default policy. + +## Cherry-Pick: Selective History + +Use cases: + +- Backport fix to old release branch +- Take isolated commit without full merge + +Risk: + +- Can duplicate commits across branches +- Future merges may conflict badly + +Rule: + +- Cherry-pick **isolated fixes** +- Avoid cherry-picking **whole feature work** + +## Tags: Marking Releases + +A tag is a fixed pointer to a specific commit. Unlike a branch, a tag never moves once created; it exists to mark a point in history, most commonly a release. + +### Lightweight vs Annotated Tags + +- A **lightweight** tag is just a name pointing at a commit, with no extra metadata: `git tag v1.2.3`. +- An **annotated** tag is a full Git object: it stores the tagger, date, and a message, and can be GPG-signed. Use annotated tags for anything you intend to keep, including every release: + +```bash +git tag -a v1.2.3 -m "Release 1.2.3: PAN recovery fix" +git tag -s v1.2.3 -m "Release 1.2.3" # signed, if you use GPG +``` + +### Working with Tags + +```bash +git tag # list tags +git tag -l "v1.2.*" # list tags matching a pattern +git push origin v1.2.3 # push a single tag +git push origin --tags # push all tags +git tag -d v1.2.3 # delete locally +git push origin --delete v1.2.3 # delete on the remote +git describe --tags # nearest tag reachable from HEAD +``` + +### Naming and Placement + +Tag names conventionally follow [Semantic Versioning](https://semver.org): `vMAJOR.MINOR.PATCH`, for example `v2.1.0`. Where you tag depends on the branching strategy. On trunk-based or GitHub Flow projects, tag the commit on `main` right after it is deployed. On GitFlow, release branching or cherry-pick release branching, tag the commit on the `release/x.y` branch once it is finalised, not on `main`. + +```mermaid +gitGraph + commit id:"tg-A" + commit id:"tg-B" + commit id:"tg-C" tag:"v1.2.3" + commit id:"tg-D" +``` +[[CAPTION:Figure]] A tag pinned to a specific commit, unaffected by later commits + +### Tags vs Branches + +Both are refs, both point at a commit, and both are cheap to create, but they mean different things and Git treats them differently: + +| Question | Branch | Tag | +| ---------------------------------- | ------------------------------------------ | ------------------------------------------------- | +| Does it move? | Yes, automatically, with every commit made on it | No; it stays on the commit it was created against | +| What does it represent? | Ongoing work: a line of development | A fixed point: a release, a milestone | +| Can you commit "onto" it? | Yes, that is the point | No; there is nothing to check out and extend the way you would a branch | +| Typical lifetime | Created and deleted routinely | Created once, kept indefinitely | +| Stored under | `refs/heads/` | `refs/tags/` | +[[CAPTION:Table]] How tags and branches differ + +A practical rule of thumb: if you expect to add more commits to it, it is a branch. If it marks a point you want to return to exactly, it is a tag. + +## Reset vs Revert: When and Why + +### Revert + +Safe for shared history: + +```bash +git revert +``` + +Adds a new commit undoing the change. + +### Reset + +Moves branch pointer: + +```bash +git reset --soft HEAD~1 +git reset --mixed HEAD~1 +git reset --hard HEAD~1 +``` + +Only safe when: + +- Never pushed +- Only you have it + +## Debugging Git mistakes + +### “I lost my commits” + +Solution: reflog. See `git reflog`: The Time Machine, above, for how this works. + +### Detached HEAD + +Solution: + +```bash +git branch recovery +``` + +See HEAD, branch pointers, and detached HEAD, above, for why this happens. + +### Bad merge + +Solution: + +```bash +git revert -m 1 +``` + +`-m 1` tells revert which parent of the merge commit to treat as the mainline; see Merge types detailed, above, for what a merge commit's parents are. + +## Worktrees: Multiple Branches at Once + +A worktree is a second working directory backed by the same `.git` database. It lets you have more than one branch checked out at the same time, without stashing, without a second clone, and without duplicating the object history. + +```mermaid +flowchart LR + G[(".git" database)] + W1["Working directory: main"] + W2["Working directory: hotfix/urgent"] + G --- W1 + G --- W2 +``` +[[CAPTION:Figure]] One `.git` database backing two working directories + +### When Worktrees Help + +- A hotfix lands while you have uncommitted work in progress: check out the hotfix branch in a second worktree instead of stashing. +- Running the previous release's test suite while continuing development on `main`. +- Reviewing a colleague's pull request locally without disturbing your own checkout. + +### Worktree Commands + +```bash +git worktree add ../hotfix-dir hotfix/urgent # new worktree on an existing branch +git worktree add -b review/pr-42 ../review-dir origin/pr-42 # new branch + worktree +git worktree list +git worktree remove ../hotfix-dir +git worktree prune # clean up after manually deleting a worktree directory +``` + +### Caveats + +- The same branch cannot be checked out in two worktrees at once; Git refuses with "already checked out". +- All worktrees share the same object database and remotes; deleting the original clone breaks every worktree that depends on it. +- Worktrees are local-only. They are not something you push, pull, or see reflected for other clones of the repository. + +## Submodules: Nesting Repositories + +A submodule embeds one Git repository inside another, at a specific subdirectory, pinned to one exact commit. The parent repository does not store the submodule's files or history, only a pointer. The pointer is its URL and the commit it should be at, recorded in a `.gitmodules` file and a special gitlink entry. + +```mermaid +flowchart LR + P["Parent repo"] -->|"pointer: commit abc123"| S["Submodule repo (separate history)"] +``` +[[CAPTION:Figure]] A parent repository pointing at a pinned commit of a submodule + +### When Submodules Help + +- Vendoring a shared library or firmware component that several independent repositories need, at an explicitly pinned version. +- Splitting a large codebase into separately-versioned repositories that are still assembled together for a build. + +### Submodule Commands + +```bash +git submodule add https://example.com/lib.git libs/lib # add a submodule +git clone --recurse-submodules https://example.com/app.git # clone, including submodules +git submodule update --init --recursive # populate submodules after a plain clone +git submodule status # commit each submodule is pinned to, and whether it's clean +git submodule foreach 'git pull origin main' # run a command in every submodule +``` + +To change what a submodule points at: commit and push inside the submodule first, then commit the updated pointer in the parent repository. Forgetting the second step is the most common submodule mistake; the parent repository silently keeps pointing at the old commit. + +### Caveats and Alternatives + +- Submodules are a frequent source of confusion: a plain `git clone` leaves submodule directories empty until `submodule update --init` runs, and CI pipelines need explicit configuration to check them out. +- Checking out a submodule leaves it in a detached HEAD by design, since it is pinned to a commit rather than a branch; work done there needs its own branch and its own push before the pointer commit in the parent will make sense to anyone else. +- **`git subtree`** is the main alternative; see below. +- A **monorepo** avoids the pointer/pinning problem entirely by keeping everything in one repository, at the cost of a larger single checkout and coarser access control. +- For dependencies that have a package manager (npm, pip, NuGet), version pinning through that package manager is usually simpler than a Git submodule. + +## Subtrees: Embedding History Directly + +Where a submodule stores a *pointer* to another repository, `git subtree` copies that repository's files and history directly into a subdirectory of the parent repository. There is no separate clone step and no `.gitmodules` file: once merged in, a subtree is ordinary, versioned content as far as anyone cloning the parent repository is concerned. Under the hood it uses the `subtree` merge strategy described above. + +```mermaid +flowchart LR + P["Parent repo"] -->|"subtree merge, --prefix=libs/lib"| S["Subtree content, now part of parent's own history"] +``` +[[CAPTION:Figure]] A subtree merges another project's files and history directly into a subdirectory + +### Subtree Commands + +```bash +git subtree add --prefix=libs/lib https://example.com/lib.git main --squash +git subtree pull --prefix=libs/lib https://example.com/lib.git main --squash +git subtree push --prefix=libs/lib https://example.com/lib.git main +``` + +`--squash` collapses the incoming project's history into a single commit rather than importing every commit it ever had; drop it only if you specifically want the full imported history. + +### Subtree vs Submodule + +| Question | Submodule | Subtree | +| ---------------------------------------- | --------------------------------------------- | ---------------------------------------------------- | +| What the parent repo stores | A pointer (URL + commit) | The actual files and (optionally) history | +| Consumer experience after plain clone | Empty directory until `submodule update --init` | Works immediately; nothing extra to run | +| Pushing changes back upstream | Straightforward: commit and push inside the submodule | Awkward: `subtree push` re-derives commits to send | +| Repository size | Small; only a pointer is stored | Grows with the vendored content, unless `--squash` is used | +| Clarity that content is external | Explicit and visible in `.gitmodules` | Implicit; nothing marks the directory as vendored beyond convention | +[[CAPTION:Table]] Choosing between a submodule and a subtree + +As a rule of thumb, use a submodule when you want an explicit, easily updated pointer to a fast-moving dependency. Use a subtree when consumers should be able to clone and build with no extra steps, and you rarely push changes back upstream. + +## Git Hooks: Automating Actions + +A hook is a script Git runs automatically at a specific point in its workflow. Hooks live as executable files under `.git/hooks/`, named after the event they respond to. + +### Client-Side and Server-Side Hooks + +| Hook | Runs | Typical use | +| -------------------- | ---------------------------------------------- | ---------------------------------------------------- | +| `pre-commit` | Before a commit message is even written | Run linters or formatters; block the commit on failure | +| `commit-msg` | After the message is written, before the commit is created | Enforce a commit message format, for example a ticket-number prefix | +| `post-commit` | After the commit is created | Notifications; local logging | +| `pre-push` | Before `git push` sends anything | Run the test suite; block pushes on failure | +| `post-checkout` | After `checkout` or `switch` | Rebuild generated files; warn about stale dependencies | +| `post-merge` | After a merge completes | Reinstall dependencies if the lockfile changed | +| `pre-receive` / `update` / `post-receive` | On the **server**, when a push arrives | Enforce branch-protection policy; trigger CI; notify chat | +[[CAPTION:Table]] Common Git hooks and what they are used for + +### The Catch: Hooks Are Not Versioned by Default + +`.git/hooks/` lives inside `.git`, and `.git` is never cloned or pushed; it is regenerated locally by every clone. That means hooks placed there do not travel with the repository, and every teammate would need to install them by hand. Two ways around this: + +```bash +git config core.hooksPath .githooks # point Git at a versioned directory instead +``` + +Commit a `.githooks/` directory alongside the code, and every clone that runs this one config command uses the team's shared hooks. Alternatively, tools such as Husky, `pre-commit`, or Lefthook manage this automatically, usually by installing themselves via a `post-checkout` or dependency-install step. + +### Example: A Minimal `pre-commit` Hook + +```bash +#!/bin/sh +if git diff --cached | grep -q "TODO"; then + echo "Commit blocked: staged changes contain a TODO" + exit 1 +fi +``` + +Save this as `.githooks/pre-commit`, make it executable (`chmod +x`), and set `core.hooksPath` as above. A non-zero exit code blocks the action; a zero exit code allows it through. + +## Git Branching Strategies + +### Main-Only (Trunk-Only / Single-Branch) + +#### Description + +- There is only one long-lived branch (typically `main`). +- All work, whether feature, bugfix, or experiments, is committed directly to `main`. +- No persistent feature branches. +- Typical branch names: none needed; everything happens on `main`. + +```mermaid +gitGraph + commit id:"mo-M0" tag:"main" + commit id:"mo-M1" + commit id:"mo-M2" + commit id:"mo-M3" +``` +[[CAPTION:Figure]] Main-Only branching: all work committed directly to main + +#### When to Use + +- Small teams or solo projects. +- Prototyping, proof-of-concept, or minimal codebases. +- Projects with frequent releases and minimal complexity. + +#### Pros + +- Minimal setup. +- No branch overhead. +- No merge conflicts due to long-lived branches. + +#### Cons + +- Risky: every change goes straight into shared history, which makes it straightforward to break things. +- Little isolation, making it hard to test big changes without risk. +- Hard to track work per feature or separate stable vs experimental changes. + +#### Verdict + +Only viable for minimal or short-lived projects. For anything non-trivial, isolation (feature branches or similar) is usually worthwhile. + +### Feature-Branching + +#### Description + +- Each new feature, bugfix, or task gets its own branch, typically named something like `feature/xyz`, `bugfix/1234`, etc. +- Developers work in isolation in that branch. +- When done + reviewed, branch is merged into `main`. +- Typical branch names: `feature/`, `bugfix/`. + +```mermaid +gitGraph + commit id:"fe-M0" tag:"main" + branch feature/foo + checkout feature/foo + commit id:"fe-F1" + commit id:"fe-F2" + checkout main + merge feature/foo id:"fe-M1" +``` +[[CAPTION:Figure]] Feature-Branching: isolated branches merged into main + +#### When to Use + +- Teams working on multiple features in parallel. +- Codebases needing isolation per feature for safe development. +- Most modern development workflows. + +#### Pros + +- Good isolation per feature. +- Parallel development without interference. +- Straightforward to discard or delay features if needed. +- Clear mapping between branch and work item. + +#### Cons + +- If branches live long, risk of merge conflicts and integration drift. +- Overhead of branch management, merging, testing. +- Feature branches need regular syncing to prevent "falling too far behind". + +#### Verdict + +The “default safe mode” for most teams: balances isolation, flexibility, and safety. + +### GitFlow + +#### Description + +- Multiple long-lived branch types: `main` (production), `develop` (integration), plus support branches: `feature/*`, `release/*`, `hotfix/*`. [abtasty+2Medium+2](https://www.abtasty.com/blog/git-branching-strategies/?utm_source=chatgpt.com) +- Features branch off `develop`, then merge back into `develop`. +- When you prepare a release, you branch a `release/x.y.z` from `develop`, stabilize, then merge into both `main` and `develop`. +- For urgent hotfixes, branch off `main`, merge fix, then merge back to `develop`. +- Typical branch names: `feature/*` and `release/x.y.z` off `develop`, `hotfix/*` off `main`. + +```mermaid +gitGraph + commit id:"gf-V0" tag:"main" + branch develop + checkout develop + commit id:"gf-D1" + branch feature/foo + checkout feature/foo + commit id:"gf-F1" + checkout develop + merge feature/foo id:"gf-FD" + branch release/1.0.0 + checkout release/1.0.0 + commit id:"gf-R1" + checkout main + merge release/1.0.0 id:"gf-M1" + checkout develop + merge release/1.0.0 id:"gf-D2" + branch hotfix/urgent + checkout hotfix/urgent + commit id:"gf-H1" + checkout main + merge hotfix/urgent id:"gf-H-merge" + checkout develop + merge hotfix/urgent id:"gf-D3" +``` +[[CAPTION:Figure]] GitFlow: main, develop, feature, release, and hotfix branches + +#### When to Use + +- Larger teams, multiple concurrent streams, multiple versions in production or long-lived support branches. +- Organizations requiring formal release cycles, QA and staging phases, hotfix support. + +#### Pros + +- Clear, structured workflow. +- Isolation of work streams, proper release preparation, controlled hotfixes. +- Good for long-term maintenance and version support. + +#### Cons + +- Complex: many branches, many merging points. +- Slow: high overhead, especially for small or fast-moving teams. +- Risk of merge mistakes if developers aren’t disciplined. + +#### Verdict + +Good for complex projects, mature products, or multiple maintained versions. Probably overkill for small or fast-iterating teams. + +### GitHub Flow + +#### Description + +- A simplified model: `main` holds production-ready code. +- Developers create a branch per feature/bug fix (similar to feature-branching), then merge to `main` via pull request when ready. No separate `develop`, no release branches. +- Typical branch names: `feature/`, or any short descriptive name off `main`; no fixed prefix is required. + +```mermaid +gitGraph + commit id:"gh-M0" tag:"main" + branch feature/foo + checkout feature/foo + commit id:"gh-F1" + checkout main + merge feature/foo id:"gh-M1" +``` +[[CAPTION:Figure]] GitHub Flow: feature branches merged directly into main + +#### When to Use + +- Teams practising continuous integration / continuous delivery (CI/CD), frequent small releases. +- Small–medium projects, web apps, SaaS, where quick iteration and fast deployment matter. + +#### Pros + +- Straightforward to understand and use. +- Encourages small, focused changes. +- Works well with CI/CD pipelines and frequent deployment. + +#### Cons + +- `main` must always stay production-ready, which requires robust testing and deployment safeguards. +- Harder to manage long-term release versions or patch older versions. +- Less formal structure: risks slip if discipline is poor. + +#### Verdict + +Excellent for agile, fast-paced teams with good automated testing and deployment. + +### GitLab Flow + +#### Description + +- A middle ground between GitHub Flow and GitFlow: keeps the single `main` integration branch from GitHub Flow, but adds downstream **environment branches** (for example, `staging` and `production`) or **release branches** for versioned software. +- Feature branches come off `main` and merge back via merge request, same as GitHub Flow. +- Deployments happen by promoting `main` into the next environment branch, one merge at a time. +- Typical branch names: `feature/*` off `main`, plus the fixed environment names `staging` and `production`. +- Distinct from Environment Branching (below): `main` stays the single integration point here, and the environment branches only ever receive promotions from it. + +```mermaid +gitGraph + commit id:"gl-M0" + branch staging + commit id:"gl-S0" + checkout main + branch production + commit id:"gl-P0" + checkout main + branch feature/foo + commit id:"gl-F1" + checkout main + merge feature/foo id:"gl-M1" + checkout staging + merge main id:"gl-S1" + checkout production + merge staging id:"gl-P1" +``` +[[CAPTION:Figure]] GitLab Flow: main promoted through staging and production + +#### When to Use + +- Teams that want GitHub Flow's simplicity but need a visible, auditable path from `main` to each deployed environment. +- Products with a staging/QA gate before production, or that need versioned releases without the full weight of GitFlow. + +#### Pros + +- Simpler than GitFlow, more structured than GitHub Flow. +- Environment branches double as a live record of what is deployed where. +- Works for both continuous delivery and scheduled releases. + +#### Cons + +- More branches than GitHub Flow, and still needs discipline to avoid environment drift. +- Cherry-picking or hotfixing "backwards" (for example, production before main) can get confusing. +- Choosing where a fix belongs (main vs environment branch) is a recurring judgement call. + +#### Verdict + +A sensible default when GitHub Flow feels too loose but GitFlow feels too heavy. + +### Trunk-Based Development + +#### Description + +- Developers integrate changes frequently into a single shared branch, often `main` (the “trunk”). +- Use short-lived feature branches or commit directly, but merge back very frequently (daily or even more often). Some teams use feature flags to hide incomplete work. +- Typical branch names: short, disposable branches such as `/`, alive for a day or two at most. + +```mermaid +gitGraph + commit id:"tb-M0" tag:"main" + commit id:"tb-M1" + branch short-lived/feat1 + checkout short-lived/feat1 + commit id:"tb-F1" + checkout main + merge short-lived/feat1 id:"tb-M2" + branch short-lived/feat2 + checkout short-lived/feat2 + commit id:"tb-F2" + checkout main + merge short-lived/feat2 id:"tb-M3" +``` +[[CAPTION:Figure]] Trunk-Based Development: short-lived branches merged frequently + +#### When to Use + +- Teams with good automated testing & CI/CD. +- Rapid release cadence, frequent small changes. +- Projects where feature-toggle / flags are used to hide incomplete features. + +#### Pros + +- Minimises integration pain, merge conflicts. +- Encourages small incremental changes. +- Supports fast delivery / continuous deployment. + +#### Cons + +- Requires discipline, strong testing, feature-flags if features aren’t fully complete. +- Harder to isolate large changes or long-running features. +- Risk that `main` becomes unstable if process breaks. + +#### Verdict + +Effective for teams set up for CI/CD, with strong automation, testing and code discipline. + +### Environment Branching + +#### Description + +- Dedicated long-lived branches represent deployed environments, typically `development` → `staging` → `production`. +- Feature branches are created from, and merged back into, `development`. +- Promoting a change to the next environment is a merge (or PR) from one environment branch into the next. +- Typical branch names: `feature/*` off `development`, plus the fixed environment names `development`, `staging`, `production`. +- Distinct from GitLab Flow (above): there is no separate `main` here; `development` is itself one of the three environment branches feature work lands on. + +```mermaid +gitGraph + commit id:"en-M0" + branch development + checkout development + commit id:"en-D0" + branch staging + commit id:"en-S0" + checkout development + branch production + commit id:"en-P0" + checkout development + branch feature/foo + commit id:"en-F1" + checkout development + merge feature/foo id:"en-D1" + checkout staging + merge development id:"en-S1" + checkout production + merge staging id:"en-P1" +``` +[[CAPTION:Figure]] Environment Branching: development promoted through staging and production + +#### When to Use + +- Organisations with a formal deployment pipeline and distinct environment owners or sign-off gates. +- Regulated or infrastructure-heavy contexts where "what is in staging vs production" needs to be answerable from the branch list alone. + +#### Pros + +- Clear, visual mapping between branches and the deployment pipeline. +- Straightforward to see, and roll back, what is live in each environment. +- Supports formal change-approval processes between environments. + +#### Cons + +- Environment branches can drift apart if promotions are skipped or done out of order. +- Extra merge overhead compared to a single trunk. +- Does not suit frequent, small releases well; better for staged rollouts. + +#### Verdict + +Worth it when the environments themselves need governance, not only the code. + +### Release-Branching / Maintenance Branching + +#### Description + +- `main` (or trunk) is used for ongoing development. +- For each planned version or release, a dedicated `release/x.y` (or similar) branch is created. +- Bug fixes can go into release branches. Release branches may receive hotfixes. +- Once released, release branch may be merged into main (or remain for maintenance), depending on policy. Also allowing multiple active release versions. +- Typical branch names: `release/x.y`, with `hotfix/x.y.z` for urgent fixes against a specific release. +- Distinct from GitFlow's `release/*` branches: GitFlow cuts a release branch to stabilise one upcoming release and then retires it, while here release branches are often kept indefinitely, one per version still under support. + +```mermaid +gitGraph + commit id:"rb-M0" tag:"main" + commit id:"rb-M1" + branch release/1.0 + checkout release/1.0 + commit id:"rb-R1" + checkout main + commit id:"rb-M2" + branch release/2.0 + checkout release/2.0 + commit id:"rb-R2" +``` +[[CAPTION:Figure]] Release-Branching: dedicated branches per release version + +#### When to Use + +- Products that need long-term maintenance of multiple versions (for example, enterprise or LTS support). +- Use cases where you cannot always keep everyone on the latest version; older versions must be maintained. + +#### Pros + +- Clean version separation. +- Ability to patch old versions without interfering with ongoing development. +- Clear release history and stability. + +#### Cons + +- Branch proliferation. +- Cherry-picking fixes across branches is error-prone. +- More management overhead. + +#### Verdict + +Useful for mature products needing long-term support and multiple active versions, but adds overhead and complexity. + +### Cherry-pick Release Branching (Selective Release) + +This is the pattern for a team that runs **trunk-based development for merging** (every completed feature lands on `main`) but wants a **curated** release. Rather than branching `release/x.y` from `main` with everything up to that point, the release branch starts from a stable point. Only specific, already-merged features are cherry-picked onto it. + +#### Description + +- `main` is the single source of truth: all completed features merge here as they finish (trunk-based style). +- When a release is due, a `release/x.y` branch is cut. +- Instead of merging `main` wholesale, individual feature commits (or merge commits) are **cherry-picked** from `main` onto the release branch, so a feature can be "done" and sitting in `main` without necessarily shipping in the next release. +- Anything not cherry-picked waits for a later release. +- Typical branch names: ordinary `feature/*` branches merged into `main`, plus `release/x.y` cut for each curated release. + +```mermaid +gitGraph + commit id:"cp-M0" tag:"main" + branch feature/A + checkout feature/A + commit id:"cp-A1" + checkout main + merge feature/A id:"cp-MA" + branch feature/B + checkout feature/B + commit id:"cp-B1" + checkout main + merge feature/B id:"cp-MB" + branch feature/C + checkout feature/C + commit id:"cp-C1" + checkout main + merge feature/C id:"cp-MC" + branch release/1.0 + checkout release/1.0 + cherry-pick id:"cp-MA" parent:"cp-A1" + cherry-pick id:"cp-MC" parent:"cp-C1" +``` +[[CAPTION:Figure]] Cherry-pick Release Branching: selected features cherry-picked into a release branch + +Here, features A and C ship in `release/1.0`; feature B is complete on `main` but held back on purpose. + +#### When to Use + +- No canonical name exists for this in the way "GitFlow" or "GitHub Flow" do, but it is a documented, real-world pattern: teams sometimes call it "cherry-pick to release", a "curated release branch", or "selective release". It shows up wherever teams want the integration benefits of trunk-based merging without every merged feature being release-committed. +- Regulatory or contractual gating, where a specific feature must not ship until a separate approval lands. +- Staggered rollout across customers or regions using the same codebase. +- Teams without mature feature-flag infrastructure who still want to decouple "merged" from "released". + +#### Pros + +- Decouples integration (merging early, avoiding long-lived branches) from release timing. +- No feature-flag machinery required to hold work back. +- `main` always reflects the full, current state of development. + +#### Cons + +- Manual and error-prone: someone has to track which commits go into which release. +- Cherry-picks can conflict, especially if the held-back feature and a shipped feature touch the same code. +- History diverges between `main` and `release/x.y` in a way that is harder to reason about than a straight merge. +- Feature flags are usually the safer alternative for the same goal, if the team can invest in them. + +#### Verdict + +A pragmatic escape hatch rather than a strategy to adopt by default. Reach for feature flags first, and use selective cherry-picking when flags are not feasible. + +### Forking Workflow + +#### Description + +- Fundamentally different from the others: instead of branches within one shared repository, each contributor has their **own server-side fork** of the repository. +- Contributors branch and commit within their own fork, then open a pull request from their fork back to the main (upstream) repository. +- The upstream repository owner reviews and merges; no contributor needs write access to the main repository. +- Typical branch names: whatever a contributor likes in their own fork; only upstream `main` (and any release branches) needs a fixed name. + +```mermaid +flowchart LR + subgraph Upstream["Upstream repository"] + M[main] + end + subgraph ForkA["Contributor A's fork"] + FA[feature/foo] + end + subgraph ForkB["Contributor B's fork"] + FB[feature/bar] + end + M -->|fork| FA + M -->|fork| FB + FA -->|pull request| M + FB -->|pull request| M +``` +[[CAPTION:Figure]] Forking Workflow: contributors submit pull requests from their own forks + +#### When to Use + +- Open-source projects with many external, untrusted, or occasional contributors. +- Any project where write access to the canonical repository should stay controlled. + +#### Pros + +- Strongest possible isolation: no contributor can push directly to upstream. +- Forces every change through review via pull request. +- Scales to large, distributed, loosely affiliated contributor bases. + +#### Cons + +- Heavier setup and mental overhead (keeping a fork in sync with upstream). +- Not needed, and mostly overkill, for a single trusted internal team. +- Requires more Git fluency (remotes, syncing forks) from contributors. + +#### Verdict + +The right default for open-source projects; unnecessary ceremony for a closed internal team that already uses feature branches and PRs. + +### Branch Naming Conventions + +Most of the strategies above lean on a `/` naming convention, such as `feature/checkout-flow` or `release/2.1`. This works because Git branches are just refs, stored under `.git/refs/heads/`, and a `/` in the name creates a real subdirectory in that path. Two consequences follow directly from that: + +- A namespace prefix (`feature/`, `release/`, `hotfix/`) groups related branches together in tooling that understands the convention (branch lists, CI triggers, protection rules), without needing any special Git feature. +- You cannot have a branch called `release` and a branch called `release/1.0` at the same time. Git stores the first as the file `refs/heads/release` and the second as `refs/heads/release/1.0`, and a file cannot also be a directory. Git refuses the second branch with an error such as `cannot lock ref ... 'refs/heads/release' exists; cannot create 'refs/heads/release/1.0'`. Pick a convention (bare `release` for a single active release, or `release/x.y` for several) and stay on one side of it. + +A common set of prefixes, independent of which strategy you use: + +| Prefix | Typical use | +| ------------- | -------------------------------------------------- | +| `feature/*` | New functionality, not yet in `main` | +| `bugfix/*` | A fix for an issue found before release | +| `hotfix/*` | An urgent fix against code already in production | +| `release/*` | Stabilising a specific version ahead of release | +| `chore/*` | Tooling, dependency bumps, or other non-user-facing work | +| `docs/*` | Documentation-only changes | +[[CAPTION:Table]] Common branch-name prefixes and what they signal + +### Comparing the strategies + +| Strategy | Branch overhead | Release control | Good Fit | +| -------------------------------- | ---------------- | --------------------------- | -------------------------------------------- | +| Main-Only | None | None (every commit is live) | Solo projects, prototypes | +| Feature-Branching | Low | Loose (merge when ready) | Most teams, most projects | +| GitFlow | High | Strict, scheduled releases | Large teams, multiple maintained versions | +| GitHub Flow | Low | None (deploy on merge) | CI/CD teams shipping continuously | +| GitLab Flow | Medium | Staged, via environment branches | Staging/production gate before release | +| Trunk-Based Development | Minimal | None (feature flags instead) | Mature CI/CD, high commit frequency | +| Environment Branching | Medium | Staged, per environment | Formal, governed deployment pipelines | +| Release-Branching / Maintenance | Medium-high | Per-version, long-lived | Products supporting several live versions | +| Cherry-pick Release Branching | Medium | Selective, per feature | Curated releases without feature-flag tooling | +| Forking Workflow | High (per contributor) | Via PR review | Open-source, untrusted or external contributors | +[[CAPTION:Table]] Comparison of the branching strategies covered in this guide + +## Useful links + +(only links, no references in the content) + +- [https://www.youtube.com/watch?v=WBg9mlpzEYU](https://www.youtube.com/watch?v=WBg9mlpzEYU) +- [https://www.youtube.com/watch?v=Mfz8NQncwiQ](https://www.youtube.com/watch?v=Mfz8NQncwiQ) +- [https://www.youtube.com/watch?v=hae9zg0-sZY](https://www.youtube.com/watch?v=hae9zg0-sZY) +- [https://www.youtube.com/watch?v=dgOpnebZkRo](https://www.youtube.com/watch?v=dgOpnebZkRo) + +## Appendix: Command Reference + +### Basic Concepts and Terms + +| Term | Meaning | +| -------------------- | --------------------------------------------------------- | +| Repository | The full project history (in `.git`) | +| Working Directory | The actual files you’re editing | +| Index (Staging Area) | The next commit snapshot | +| Commit | A snapshot of staged changes | +| Branch | A pointer to a commit | +| HEAD | Where you are in the history (usually the current branch) | +| DAG | Directed Acyclic Graph that models history | +| Remote | Another copy of the repo (for example, `origin`) | +| Worktree | A second checkout of the repo in another directory | +| Bisect | Binary search across commits to find a regression | +[[CAPTION:Table]] Basic Git concepts and terms + +### Global Setup + +These are one-time settings. + +#### Identity + +```bash +git config --global user.name "Your Name" +git config --global user.email "you@example.com" +``` + +#### Defaults and Safety + +```bash +git config --global init.defaultBranch main +git config --global pull.rebase false # or true if your team prefers +git config --global core.autocrlf input # depends on OS +``` + +#### Credential Helper + +```bash +git config --global credential.helper store +``` + +### Repository Creation and Cloning + +#### Start a New Repo + +```bash +git init +``` + +Creates `.git/` and starts tracking. + +#### Clone Existing + +```bash +git clone +cd repo +``` + +#### Add Remote Later + +```bash +git remote add origin +``` + +### Status, Diff, Add, Commit + +#### Status + +```bash +git status +``` + +Shows: + +- modified files +- staged files +- untracked files +- branch info + +#### Diff + +```bash +git diff +git diff --staged +git diff HEAD~1 HEAD +``` + +#### Stage changes + +```bash +git add file.c +git add . # everything (careful) +git add -p # interactive hunks +``` + +#### Commit + +```bash +git commit -m "Short summary" +git commit +``` + +Without `-m` opens editor for detailed message. + +#### Ignoring and cleaning + +```bash +echo "*.log" >> .gitignore +git clean -n # preview what would be removed +git clean -fd # remove untracked files and directories +``` + +`.gitignore` only stops **untracked** files being tracked; it does not affect files Git already knows about. `git clean` removes untracked files from the working directory; always run `-n` first to preview. + +### Branch Management and Switching + +#### List branches + +```bash +git branch +git branch -a # includes remote branches +``` + +#### Create branch + +```bash +git branch feature/x +``` + +#### Create and switch + +```bash +git checkout -b feature/x +``` + +#### Switch + +```bash +git checkout feature/x +``` + +or the newer, narrower form: + +```bash +git switch feature/x +``` + +`checkout` is overloaded: it switches branches and restores files, which is why `switch` (branches) and `restore` (files) were split out. Prefer `switch` and `restore` for clarity in day-to-day use; `checkout` still works everywhere and is what most older docs and scripts use. + +#### Rename branch + +```bash +git branch -m old-name new-name # rename current or named branch +git push origin -u new-name # push the new name +git push origin --delete old-name # remove the old name upstream +``` + +#### Track a remote branch + +```bash +git branch -vv # show tracking status for all local branches +git branch --set-upstream-to=origin/feature/x # set tracking on the current branch +git checkout --track origin/feature/x # create + track in one step +``` + +#### Find merged and stale branches + +```bash +git branch --merged main # branches fully merged into main, safe to delete +git branch --no-merged main # branches with unmerged work +git fetch --prune # remove local references to deleted remote branches +``` + +#### Delete branch + +```bash +git branch -d feature/x # safe delete (refuse if unmerged) +git branch -D feature/x # force delete +git push origin --delete feature/x # delete on the remote +``` + +#### Worktrees (multiple branches checked out at once) + +```bash +git worktree add ../hotfix-dir hotfix/urgent +git worktree list +git worktree remove ../hotfix-dir +``` + +A worktree checks out another branch into a separate directory, backed by the same `.git` database. Useful for working on a hotfix without stashing or disturbing your current branch's state. + +### Fetch, Pull, Push + +#### Fetch + +```bash +git fetch origin +``` + +Updates remote-tracking (does **not** merge). + +#### Pull + +```bash +git pull +git pull origin main +``` + +Fetch + merge (or rebase if configured). + +#### Push + +```bash +git push origin main +git push -u origin feature/x +``` + +`-u` sets upstream so later `git push` works without args. + +### Merging + +#### Fast-forward Merge (Quick Reference) + +```bash +git checkout main +git merge feature/x +``` + +If no divergence, branch pointer moves forward. + +#### No Fast-forward Merge + +```bash +git merge --no-ff feature/x +``` + +Forces a merge commit even if FF possible. + +#### Conflict resolution + +```bash +git status +git diff +# manually edit conflict markers +git add . +git commit +``` + +### Rebase + +#### Rebase onto latest main + +```bash +git checkout feature/x +git fetch origin +git rebase origin/main +``` + +#### Interactive rebase (tidy history) + +```bash +git rebase -i HEAD~5 +``` + +Options: + +- pick +- squash +- reword +- drop +- fixup (like squash, but discards the fixup commit's message) + +#### Fixup and autosquash + +```bash +git commit --fixup # creates a "fixup! ..." commit +git rebase -i --autosquash HEAD~5 +``` + +`--fixup` marks a commit as belonging to an earlier one. `--autosquash` then reorders the interactive rebase list so those fixups land next to, and squash into, the commit they target, without you moving lines by hand. + +#### Abort or continue + +```bash +git rebase --abort +git rebase --continue +``` + +### Undoing Mistakes + +#### Unstage + +```bash +git restore --staged file.c +``` + +#### Discard local changes + +```bash +git restore file.c +``` + +#### Reset (local history rewrite) + +```bash +git reset --soft HEAD~1 +git reset --mixed HEAD~1 +git reset --hard HEAD~1 +``` + +> **Warning:** Never use reset on pushed commits. + +#### Revert (safe for shared history) + +```bash +git revert +``` + +Creates a new commit that undoes a past commit. + +### Cherry-Pick + +Apply a specific commit from elsewhere: + +```bash +git checkout main +git cherry-pick +``` + +Used for **backporting** fixes; use sparingly. + +### Stash + +Save work without committing: + +```bash +git stash push -m "WIP" +git stash push --include-untracked # also stash new, untracked files +git stash list +git stash show stash@{0} +git stash apply stash@{0} # restore, keep it in the stash list +git stash pop # restore, then delete from the stash list +git stash drop stash@{0} # delete without restoring +``` + +`apply` is safer when you might want the same stash again elsewhere; `pop` is the common case of "restore and forget". + +### Log and Reflog + +#### Log + +```bash +git log +git log --oneline +git log --oneline --graph --decorate --all +git log --patch +git log --stat +``` + +Useful patterns: + +- `--first-parent` for release history +- `-- path/to/file` for file history + +#### Reflog + +```bash +git reflog +``` + +Shows **HEAD movements**.\ +Use to recover lost work: + +```bash +git reset --hard +``` + +### Bisect + +```bash +git bisect start +git bisect bad +git bisect good v1.4.0 +# test, then: git bisect good | git bisect bad +git bisect reset + +git bisect start HEAD v1.4.0 +git bisect run npm test # automated +``` + +### Tags and Releases + +#### Create tags + +```bash +git tag v1.0.0 +git tag -a v1.0.0 -m "Description" +``` + +#### Push tags + +```bash +git push origin v1.0.0 +git push origin --tags +``` + +### Submodules + +```bash +git submodule add https://example.com/lib.git libs/lib +git clone --recurse-submodules https://example.com/app.git +git submodule update --init --recursive +git submodule status +git submodule foreach 'git pull origin main' +``` + +### Subtrees + +```bash +git subtree add --prefix=libs/lib https://example.com/lib.git main --squash +git subtree pull --prefix=libs/lib https://example.com/lib.git main --squash +git subtree push --prefix=libs/lib https://example.com/lib.git main +``` + +### Merge Strategies (Quick Reference) + +```bash +git merge -s ort feature/x # default; usually omitted +git merge --ff-only feature/x # refuse unless a fast-forward is possible +git merge -s ours feature/x # keep our tree entirely, discard theirs +git merge -X ours feature/x # normal merge, prefer our side on conflicting hunks +git merge branchA branchB branchC # octopus, for more than two branches at once +``` + +### Hooks + +```bash +git config core.hooksPath .githooks # use a versioned hooks directory +chmod +x .githooks/pre-commit # hooks must be executable +``` + +### Useful Command Flows + +This is where the cheat-sheet becomes a **practical reference**. + +#### Feature Development Flow + +```bash +git checkout main +git pull +git checkout -b feature/abc +# work… +git add . +git commit -m "Implement X" +git fetch origin +git rebase origin/main +git push -u origin feature/abc +# create PR +``` + +#### Basic Merge Flow + +```bash +git checkout main +git pull +git merge feature/abc +git push +``` + +#### PR Fixup Flow (after comments) + +```bash +git checkout feature/abc +# apply fixes +git add . +git commit -m "Address review feedback" +git push +``` + +Or tidy history before merge: + +```bash +git rebase -i origin/main +git push --force-with-lease +``` + +Use `--force-with-lease` for safety (protects peers’ work). + +#### Recover Lost Work + +```bash +git reflog +git checkout +git branch recovery +``` + +#### Clean Up Local Branches + +```bash +git branch --merged +git branch -d feature/old +``` diff --git a/content/guides/git/02-branching-strategies.md b/content/guides/git/02-branching-strategies.md new file mode 100644 index 0000000..b1477e7 --- /dev/null +++ b/content/guides/git/02-branching-strategies.md @@ -0,0 +1,1282 @@ +--- +title: Branching strategies +--- + +# Branching strategies + +Effective management of code changes is critical for any development team. A robust Git branching strategy is essential for ensuring streamlined collaboration and the delivery of high-quality software. + +In this guide, we will examine the most prevalent branching strategies, explore modern approaches, discuss common anti-patterns, and provide criteria for selecting the optimal strategy for your team. + +## Why Your Team Needs a Branching Strategy + +A well-defined branching strategy is fundamental for establishing an efficient and problem-free development environment. Key benefits include: + +- **Enhanced Collaboration**: Allows multiple developers to work on isolated features without interfering with each other's progress. + +- **Risk Mitigation**: Isolates new features or bug fixes, thereby protecting the main codebase from instability. The production branch remains clean and deployable. + +- **Organised Workflow**: Prevents complex merge scenarios by providing a clear structure for integrating changes. + +- **Quality Control**: Integrates with pull requests, code reviews, and CI/CD pipelines to ensure every change is tested and reviewed prior to deployment. + + +## Popular Git Branching Strategies Explained + +Let's break down the most widely used strategies, complete with their pros, cons, and visual diagrams. + +### 1\. GitFlow + +Originally introduced by Vincent Driessen in 2010, GitFlow is a highly structured strategy designed for projects with formal release cycles and long-term maintenance needs. + +- **Popularity**: Still popular, especially in traditional enterprise and projects with strict release schedules. + +- **Age**: Introduced by Vincent Driessen in 2010. + +- **Use Cases/Who Used It**: Often used by teams with formal release cycles, long-term support requirements, and complex, versioned software (e.g., software vendors, some enterprise applications, embedded systems with distinct versions). + + +**How GitFlow Works:** GitFlow defines five core branches, each with a specific purpose: + +- `main`: Contains production-ready code. Every commit here is a stable release. + +- `develop`: The integration branch where new features are merged before going live. + +- `feature/*`: For building new functionality. Branches off from `develop` and merges back into it. + +- `release/*`: Used to prepare a new version for production. Created from `develop` and eventually merged into both `main` and `develop`. + +- `hotfix/*`: For urgent fixes on production. Created from `main`, then merged back into both `main` and `develop`. + +```mermaid +gitGraph +commit msg: "Initial" +branch develop +commit msg: "Dev Init" +checkout main +merge develop +branch feature/X +commit msg: "Feat X 1" +commit msg: "Feat X 2" +checkout develop +merge feature/X +branch staging +checkout staging +commit msg: "Staging Test" +checkout develop +merge staging +checkout main +merge staging +commit msg: "Prod Deploy" +``` + + +**When to Use GitFlow:** + +- Your team follows scheduled release cycles. + +- You need to maintain multiple versions (e.g., patch v1.x while working on v2.x). + +- You have formal QA or staging environments. + +- Your project has complex release management or compliance requirements. + + +**Pros:** + +- Clear separation of concerns. + +- Ideal for parallel development across different product versions. + +- Structured workflow enforces discipline and quality control. + +- Predictable release cycles. + + +**Cons:** + +- Adds complexity, not ideal for small teams or fast-moving startups. + +- Not suited for continuous deployment. + +- Can result in long-lived branches, increasing merge conflicts. + +- Slows down delivery due to multiple approval stages. + + +### 2\. GitHub Flow + +GitHub Flow is the minimalist cousin of GitFlow, designed for teams that ship fast and often, especially those embracing continuous deployment. + +- **Popularity**: Very popular, especially in web development and SaaS. + +- **Age**: Gained prominence with GitHub's rise, early 2010s. + +- **Use Cases/Who Used It**: Widely adopted by many web startups, SaaS companies, and open-source projects that practice continuous delivery. + + +**How GitHub Flow Works:** The workflow is refreshingly straightforward: + +1. Create a feature branch from `main`. + +2. Push commits to the feature branch. + +3. Open a pull request for code review and automated tests. + +4. Once approved, merge back to `main`. + +5. Deploy immediately (optional but encouraged). Everything in `main` should always be production-ready. + +```mermaid + gitGraph + commit msg: "v1.0" + branch feature/login + commit msg: "Login 1" + commit msg: "Login 2" + checkout main + merge feature/login + commit msg: "v1.1" + branch feature/dashboard + commit msg: "Dash 1" + checkout main + merge feature/dashboard + commit msg: "v1.2" +``` + + +**When to Use GitHub Flow:** + +- Your team practices continuous deployment. + +- You have strong automated testing in place. + +- You ship frequent, small releases. + +- You're working in a small to mid-sized team. + + +**Pros:** + +- Simple and intuitive to adopt. + +- Perfect for rapid iteration and fast feedback loops. + +- Plays well with CI/CD pipelines. + +- Encourages frequent integration, reducing big-bang merges. + + +**Cons:** + +- Not designed for multiple production versions or LTS support. + +- In large teams, it can result in frequent merge conflicts. + +- No formal release staging or QA branching. + +- Requires strong test automation. + + +### 3\. GitLab Flow + +GitLab Flow is a hybrid strategy that builds on GitFlow and GitHub Flow, integrating environments and deployment workflows. It's particularly useful in projects with clear DevOps pipelines. + +- **Popularity**: Popular among teams using GitLab. + +- **Age**: Introduced by GitLab in 2014. + +- **Use Cases/Who Used It**: Teams heavily integrated with GitLab's CI/CD, often in cloud-native development, aiming for a balance between simplicity and structured deployments. + + +**How GitLab Flow Works:** There are two common patterns: + +- **Production Branch Model**: Developers create feature branches from `main` or `develop`, merge back, and then tag releases and deploy from `main` to environments. + +- **Environment-Based Model**: Each environment (e.g., `pre-prod`, `staging`, `production`) has a dedicated branch. Merges promote changes from lower to higher environments. + +```mermaid +gitGraph + commit msg: "Initial" + + branch develop + commit msg: "Dev Init" + + checkout main + merge develop + + branch feature/X + commit msg: "Feat X 1" + commit msg: "Feat X 2" + + checkout develop + merge feature/X + + branch staging + checkout staging + commit msg: "Staging Test" + + checkout develop + merge staging + + checkout main + merge staging + commit msg: "Prod Deploy" + +``` + + +**When to Use GitLab Flow:** + +- You want to map branches to environments or deployments. + +- You use GitLab CI/CD pipelines and want tight integration. + +- Your team needs flexibility but also structure between releases. + + +**Pros:** + +- Tightly integrates with CI/CD and DevOps tooling. + +- Supports both continuous delivery and versioned releases. + +- Encourages better traceability. + + +**Cons:** + +- Can be confusing for beginners due to multiple models. + +- Needs strict discipline to avoid inconsistencies across environment branches. + + +### 4\. Environment Branching + +In this strategy, each deployment environment (like `dev`, `qa`, `staging`, `prod`) has its own branch. Code progresses through the pipeline by merging from one environment branch to the next. + +- **Popularity**: Declining, often considered an anti-pattern in modern contexts. + +- **Age**: Older practice, predates widespread modern CI/CD. + +- **Use Cases/Who Used It**: Primarily found in legacy systems or environments with very manual deployment processes, where strict control over environment progression is needed, or in highly regulated environments with specific audit requirements. + +```mermaid +gitGraph + commit msg: "Initial" + + %% Development + branch dev + commit msg: "Dev A" + commit msg: "Dev B" + + %% QA – branch from dev, diverge first, then merge dev in + branch qa + checkout qa + commit msg: "QA Test" %% diverge qa + merge dev %% bring latest dev into qa + + %% Staging – branch from qa, diverge first, then merge qa in + branch staging + checkout staging + commit msg: "Staging Test" %% diverge staging + merge qa %% bring qa into staging + + %% Prod – branch from staging, diverge first, then merge staging in + branch prod + checkout prod + commit msg: "Prod Release" %% diverge prod + merge staging %% promote to prod + +``` + + +**When to Use Environment Branching:** + +- You're working with legacy systems or manual promotion processes. + +- Your CI/CD tooling is limited or non-existent. + +- You want explicit control over what gets pushed to each environment. + + +**Pros:** + +- Very explicit control over deployments. + +- Simple to understand in legacy or manual release workflows. + + +**Cons:** + +- High risk of divergence between environments. + +- Code may behave differently if not carefully synchronised. + +- Difficult to scale with modern CI/CD practices. + +- Often considered an anti-pattern in modern DevOps setups. + + +### 5\. Trunk-Based Development + +Trunk-Based Development is the go-to for high-performing teams that deploy multiple times a day. It's simple on the surface but demands real discipline. The idea? Everyone works off a single, main branch (`main` or `trunk`), with no long-lived feature branches. + +- **Popularity**: Growing rapidly, especially in high-performing tech companies. + +- **Age**: Conceptually old (pre-Git), gained modern prominence in the early 2010s with the rise of CI/CD. + +- **Use Cases/Who Used It**: Adopted by companies like Google, Amazon, Microsoft, and other tech giants focused on high-frequency deployments and large-scale continuous integration. Ideal for SaaS products. + + +**How Trunk-Based Development Works:** + +- There's one main branch (`main` or `trunk`). All development happens here. + +- Developers commit directly to `main`, often multiple times per day. + +- Changes are small, incremental, and backed by automated tests. + +- Incomplete features are hidden behind feature flags, allowing code to ship without being user-visible. + +```mermaid + gitGraph + commit msg: "C1" + commit msg: "C2" + commit msg: "C3" + commit msg: "C4 (Feature A Flagged)" + commit msg: "C5" + commit msg: "C6 (Feature B Flagged)" + commit msg: "C7" +``` + + +**When to Use Trunk-Based Development:** + +- Your team follows continuous integration religiously. + +- You have a strong automated testing setup (unit + integration + end-to-end). + +- You're building SaaS products or anything that updates frequently. + +- Your team is experienced with feature flagging and CI/CD tools. + + +**Pros:** + +- No more painful merges, everyone works on the same code path. + +- Encourages true continuous integration and early bug discovery. + +- Delivers the fastest feedback loops from dev to prod. + +- Simplifies the workflow. + + +**Cons:** + +- Requires robust testing to avoid breaking production. + +- Not ideal for large, monolithic features (unless behind flags). + +- You'll need to manage feature flags carefully to avoid technical debt. + +- Demands a high level of discipline across the team. + + +### 6\. Release Branching + +If your product has long-term users, multiple versions in the wild, and strict release schedules, Release Branching is a powerful choice. It helps isolate each major release for bug fixing and updates without slowing down future development. + +- **Popularity**: Moderate, used for specific release needs. + +- **Age**: Common practice since version control systems became widespread. + +- **Use Cases/Who Used It**: Software vendors, embedded systems, or any project requiring long-term support for specific versions and scheduled releases. Often seen in products with on-premise deployments. + + +**How Release Branching Works:** + +- When a version is ready for stabilisation, a `release` branch is created from the mainline (`main` or `develop`). + +- All bug fixes and release-specific tweaks go into this branch. + +- Meanwhile, the mainline stays open for ongoing feature development. + +- Once finalised, the release branch is tagged and deployed. Hotfixes can be applied directly to this release branch if needed. + +```mermaid + gitGraph + commit msg: "Initial Main" + commit msg: "Feature 1" + commit msg: "Feature 2" + branch release/v1.0 + commit msg: "Release Prep 1.0" + commit msg: "Bugfix 1.0" + checkout main + commit msg: "Feature 3" + commit msg: "Feature 4" + checkout release/v1.0 + commit msg: "Final Test 1.0" + checkout main + merge release/v1.0 + commit msg: "Post-Release Merge" +``` + + +**When to Use Release Branching:** + +- You support multiple product versions simultaneously (e.g., v1.x, v2.x). + +- Your releases are tied to deadlines (quarterly, client-based, etc.). + +- You need to provide long-term support or hotfixes for older versions. + +- You have a formal stabilisation and QA process before every release. + + +**Pros:** + +- Clean isolation between feature development and stabilisation. + +- Supports parallel development and maintenance efforts. + +- Great for version tracking and rollback. + +- Ideal for products requiring long-term support (LTS). + + +**Cons:** + +- Can get complex with many active branches. + +- Requires disciplined merge practices to sync fixes back into `main`. + +- Easy to fall into branch proliferation. + + +### 7\. Feature Branching + +Feature Branching is perhaps the most widely used and easiest-to-understand strategy. It's the default mental model for most developers: create a branch for each new piece of work, build in isolation, and merge back when done. + +- **Popularity**: Very popular, often the default starting point for new projects and teams. + +- **Age**: Core concept in modern distributed version control. + +- **Use Cases/Who Used It**: Used by almost all teams, often as the base layer for other flows (e.g., GitHub Flow is Feature Branching with specific rules). Applicable across all project types and sizes. + + +**How Feature Branching Works:** + +- Start by branching off from `main` (or `develop`) to work on a specific feature or bug fix. + +- Commit changes to this branch until the work is complete. + +- Once reviewed and tested, the branch is merged back, typically via a pull request. + +- Branches are usually named descriptively (e.g., `feature/login-form`, `bugfix/payment-error`). + + +**When to Use Feature Branching:** + +- Your team is new to Git workflows. + +- You want to isolate features or experiments without risk to mainline stability. + +- You value clear ownership of features or fixes. + +- Your project has medium complexity, without too many parallel changes. + + +**Pros:** + +- Provides clear isolation between features. + +- Super easy to adopt and understand. + +- Encourages better code reviews through pull request workflows. + +- Offers a flexible foundation for more advanced strategies. + + +**Cons:** + +- Can result in long-lived branches if not merged frequently. + +- Integration challenges when multiple feature branches diverge for too long. + +- Potential for merge conflicts, especially on fast-moving teams. + + +### 8\. Forking Workflow + +The Forking Workflow is built for scale, safety, and distributed collaboration, making it the default model for open-source contributions on platforms like GitHub. + +- **Popularity**: Dominant for open-source projects. + +- **Age**: Gained prominence with GitHub's widespread adoption. + +- **Use Cases/Who Used It**: The standard for public open-source projects (e.g., Linux kernel, React, Kubernetes) and often for internal projects with strict security or contribution models where contributors are not directly trusted with write access to the main repository. + + +**How Forking Workflow Works:** + +- A contributor forks the main repository, creating their copy on the server. + +- They clone their fork locally, create a feature or fix branch, and commit changes. + +- Once ready, they open a pull request back to the original repository (the "upstream"). + +- Maintainers review, request changes, and decide whether to merge. + + +**When to Use Forking Workflow:** + +- You're running an open-source project with external contributors. + +- You want strict control over what enters the main codebase. + +- Your team is distributed or made up of untrusted/external developers. + +- You expect a high volume of pull requests from the community. + + +**Pros:** + +- No write access required for contributors, keeps your main branch safe. + +- Full control stays with project maintainers. + +- Scales beautifully with large numbers of contributors. + +- The de facto standard for open-source collaboration. + + +**Cons:** + +- Slightly more complex setup for new contributors. + +- Overkill for small, trusted internal teams. + +- Requires understanding of `origin` vs. `upstream` remotes. + + +### 9\. One Flow + +One Flow is a simplified alternative to GitFlow, aiming to reduce its complexity while retaining some of its benefits for release management. It typically uses only `main` and feature branches, with releases managed by tagging `main`. + +- **Popularity**: Niche but growing as a simpler alternative to GitFlow. + +- **Age**: Emerged as a refinement of GitFlow. + +- **Use Cases/Who Used It**: Teams seeking a more lightweight release management approach than GitFlow, often smaller to mid-sized teams moving towards continuous delivery. + + +**How One Flow Works:** + +- All development happens on short-lived feature branches, branched off `main`. + +- Features are merged into `main` via pull requests. + +- `main` is always deployable. + +- Releases are created by tagging `main` at a stable point. Hotfixes are typically done on a new branch from `main` and then merged back. + +```mermaid + gitGraph + commit msg: "Initial" + branch feature/alpha + checkout feature/alpha + commit msg: "Alpha 1" + commit msg: "Alpha 2" + checkout main + merge feature/alpha tag: "v1.0" + branch feature/beta + checkout feature/beta + commit msg: "Beta 1" + checkout main + merge feature/beta tag: "v1.1" +``` + + +**When to Use One Flow:** + +- You want a simpler model than GitFlow but still need clear release tagging. + +- You practice continuous integration and want `main` to always be stable. + +- Your team prefers fewer long-lived branches. + + +**Pros:** + +- Simpler than GitFlow. + +- Maintains a clean release history with tags. + +- Good for teams moving towards continuous delivery. + + +**Cons:** + +- Less explicit separation for hotfixes compared to GitFlow. + +- Requires discipline to keep `main` always deployable. + + +### 10\. Ship / Show / Ask + +This is less of a strict branching model and more of a philosophy for merging feature branches, often used in conjunction with Trunk-Based Development or GitHub Flow. It governs the pull request (PR) process. + +- **Popularity**: Growing, especially within trunk-based development contexts. + +- **Age**: More recent, philosophical approach. + +- **Use Cases/Who Used It**: Teams practicing Trunk-Based Development, aiming to optimise their code review process based on the risk and impact of changes. + + +**How Ship / Show / Ask Works:** + +- **Ship**: Small, low-risk changes are committed directly to `main` (or merged via PR with minimal review), relying heavily on automated tests and feature flags. This is common in trunk-based development. + +- **Show**: Changes are pushed to a feature branch, a PR is opened, and the code is "shown" to teammates for feedback, but formal approval isn't strictly required before merging. It's about visibility. + +- **Ask**: For larger, riskier, or more complex changes, a formal PR is opened, requiring explicit review and approval from one or more teammates before merging. + + +```mermaid +gitGraph + commit msg: "Initial" + + branch feature/small-change + commit msg: "Small Change" + checkout main + merge feature/small-change tag: "Ship: direct merge or quick PR" + + branch feature/show-feedback + commit msg: "Show Change" + checkout main + merge feature/show-feedback tag: "Show: merge after feedback" + + branch feature/ask-approval + commit msg: "Big Change" + checkout main + merge feature/ask-approval tag: "Ask: merge after review & approval" +``` + +**When to Use Ship / Show / Ask:** + +- You want to optimise your PR workflow based on change risk and complexity. + +- You're implementing Trunk-Based Development and need guidelines for direct commits vs. PRs. + +- You trust your team's judgement and automated tests. + + +**Pros:** + +- Speeds up delivery for low-risk changes. + +- Encourages continuous integration and smaller commits. + +- Tailors the review process to the change's impact. + + +**Cons:** + +- Requires strong team trust and a mature testing culture. + +- Can be challenging to define "small" vs. "large" changes consistently. + + +### 11\. Mainline Development with Release Trains + +This approach grew out of trunk-based development. It suits very large organisations, or products where many independent teams contribute to one codebase and releases happen on a predictable, fixed cadence. + +- **Popularity**: Niche, for very large-scale enterprises. + +- **Age**: Evolved from trunk-based development for large organizations. + +- **Use Cases/Who Used It**: Large organizations like some divisions of Microsoft, Google, or other companies with hundreds or thousands of developers on a single codebase and fixed release cadences (e.g., large-scale operating systems, complex enterprise software suites). + + +**How Mainline Development with Release Trains Works:** + +- **Mainline (Trunk)**: All teams commit frequently to a single `main` branch. This branch is always kept in a releasable state, relying heavily on feature flags. + +- **Release Trains**: At a predefined time (e.g., every two weeks, monthly), a "release train" branch is cut from `main`. This branch becomes the candidate for the next release. + +- **Stabilisation**: Only bug fixes for that specific release are allowed on the release train branch. No new features. + +- **Deployment**: The release train branch goes through final testing and is deployed. + +- **Merge Back**: Critical bug fixes from the release train are merged back into `main`. + + +```mermaid +gitGraph + commit msg: "C1" + commit msg: "C2" + commit msg: "C3" + branch release-train/2025.07 + commit msg: "RT Bugfix 1" + commit msg: "RT Bugfix 2" + checkout main + commit msg: "C4" + commit msg: "C5" + checkout release-train/2025.07 + commit msg: "RT Final Test" + checkout main + merge release-train/2025.07 + branch release-train/2025.08 + commit msg: "RT 2 Bugfix 1" + checkout main + commit msg: "C6" +``` + +**When to Use Mainline Development with Release Trains:** + +- Very large organisations with many teams contributing to a single product. + +- Products with fixed, time-based release cycles. + +- When managing complex dependencies across features. + +- You have a mature CI/CD pipeline and extensive automated testing. + + +**Pros:** + +- Provides predictable release cadences. + +- Enforces continuous integration. + +- Scales well for large development efforts. + +- Clear separation of feature development and release stabilisation. + + +**Cons:** + +- Requires extremely high discipline and automation. + +- Heavy reliance on feature flags can introduce complexity. + +- Not suitable for small teams or projects with infrequent releases. + + +## Automated Backporting / Cherry-Picking + +Even with the best branching strategy, sometimes you need to bring specific commits from one branch to another without merging the entire branch. This is where backporting and cherry-picking come in. + +- **Cherry-Picking**: This Git command allows you to pick a specific commit from one branch and apply it to another. It's useful for: + + - Applying a hotfix from a `hotfix` branch to `main` and `develop` in GitFlow. + + - Bringing a small, isolated bug fix from a release branch back to `main`. + + - Applying a feature from a `feature` branch to another `feature` branch for testing purposes. + + + **How it works**: + + 1. `git checkout ` + + 2. `git cherry-pick ` + +- **Backporting**: This is the process of applying a patch or feature from a newer version of software to an older version. While cherry-picking is the _mechanism_, backporting is the _intent_. It's critical for maintaining multiple release lines. + + **Automating Backports**: For complex projects with many releases, manual cherry-picking can become tedious and error-prone. Automation can help: + + - **CI/CD Pipeline Integration**: Configure your CI/CD system to automatically propose cherry-picks or merge requests for bug fixes from release branches back to `main` or `develop`. Tools like GitLab and GitHub Actions can be scripted for this. + + - **Git Hooks**: Custom Git hooks can be set up to remind developers or even attempt automated cherry-picks on certain merge events. + + - **Dedicated Tools**: Some larger organisations build custom tooling or use commercial solutions that manage backports across multiple release lines. + + + **Key Considerations for Backporting**: + + - **Conflict Resolution**: Automated backports still often require manual conflict resolution, especially if the codebases have diverged significantly. + + - **Testing**: Always thoroughly test backported changes, as the context of the code might be different in the older branch. + + - **Consistency**: Ensure a clear policy on _what_ gets backported (e.g., only bug fixes, no new features) to maintain branch integrity. + + +## Advanced Merge Strategies: Rebase vs. Merge --squash + +Beyond the basic `git merge`, Git offers other powerful ways to integrate changes, each with implications for your repository's history. + +- **`git rebase`**: + + - **Purpose**: Rewrites commit history by moving or combining a sequence of commits to a new base commit. It essentially replays your changes on top of the target branch, creating a linear history. + + - **How it works**: If you're on `feature-branch` and run `git rebase main`, Git takes your commits from `feature-branch` that aren't on `main`, puts them aside, fast-forwards `feature-branch` to `main`, and then reapplies your commits. + + - **Pros**: Creates a clean, linear history that is easier to read and follow. Avoids "merge commits" that can clutter the graph. + + - **Cons**: Rewrites history, which can be problematic if the rebased commits have already been pushed to a shared remote repository. Requires more care and understanding, especially for collaborative branches. + + - **When to use**: Ideal for cleaning up your local feature branch before merging into `main` (e.g., `git pull --rebase` to keep your local branch up-to-date, or `git rebase -i` to squash/reorder commits before a final merge). **Never rebase commits that have been pushed to a shared branch.** + +- **`git merge --squash`**: + + - **Purpose**: Combines all commits from a feature branch into a _single_ new commit on the target branch, without recording the original branch's history. + + - **How it works**: When merging `feature-branch` into `main` with `--squash`, Git applies all changes from `feature-branch` as if they were one large commit. It doesn't create a merge commit, but rather stages the changes for a new commit on `main`. + + - **Pros**: Keeps the `main` branch history clean and concise, showing features as single, atomic changes. Useful for integrating small, self-contained features or bug fixes. + + - **Cons**: Loses the detailed commit history of the feature branch. Can make debugging harder if you need to pinpoint specific changes within the squashed feature. + + - **When to use**: Often used for merging feature branches into a `main` branch in a Trunk-Based Development or GitHub Flow setup where a clean, high-level history is preferred. + + +## How to Select Your Strategy + +Choosing the right branching strategy isn't a one-size-fits-all decision. It depends on how your team works, what you're building, and how often you ship. Here are some key factors to guide your choice: + +- **Team Size & Experience**: + + - **Small teams (2-5 developers)**: GitHub Flow or Trunk-Based Development often work best due to fewer branches and faster iteration. + + - **Larger teams**: GitFlow or Release Branching provides the structure needed for coordination, especially across multiple squads. + + - **New teams**: Start with Feature Branching or GitHub Flow – simple and easy to grasp. + + - **Mature teams**: Can evolve toward Trunk-Based or GitFlow as tooling and CI/CD processes mature. + +- **Release Frequency**: + + - **Deploying daily or multiple times per day?** Go with GitHub Flow, Trunk-Based, Ship/Show/Ask, or Mainline Development with Release Trains. They're designed for speed. + + - **Shipping on a schedule (e.g., quarterly releases)?** GitFlow, One Flow, or Release Branching gives you the stability and process control you need. + +- **Product Complexity & Maintenance Needs**: + + - **Simple apps or MVPs**: Keep it lightweight with GitHub Flow or Feature Branching. + + - **Complex, enterprise-grade systems with long-term support**: Benefit from GitFlow's or Release Branching's separation of concerns and version management capabilities. + +- **Compliance Requirements**: + + - If you're in a regulated industry (finance, healthcare, defence), you'll likely need the formal approval gates and documentation trail offered by GitFlow or Release Branching. + +- **Tooling & CI/CD Maturity**: + + - Teams with robust CI/CD pipelines and extensive automated testing can more easily adopt Trunk-Based Development or Mainline Development. + + - Teams with limited CI/CD might lean towards strategies with more manual gates like Environment Branching (though it's generally an anti-pattern for modern setups). + + +## Git Branching Anti-Patterns to Avoid + +While Git offers immense flexibility, some practices can lead to chaos and frustration. Here are common anti-patterns to steer clear of: + +- **Branch per Developer**: This happens when each developer has a single, long-lived branch where all their work resides. + + - **Why it's bad**: Leads to massive, infrequent merges, making conflict resolution a nightmare. It defeats the purpose of continuous integration and makes code reviews difficult. + + - **Instead**: Encourage small, frequent commits to feature branches (or directly to `main` in trunk-based development) and regular merging/rebase from the mainline. + +- **Branch per Component/Module**: Creating a separate branch for each logical component or module of your application. + + - **Why it's bad**: If components are interdependent, this leads to complex synchronisation issues, broken builds, and a fragmented view of the overall system. It's a sign of poor architectural decomposition or over-branching. + + - **Instead**: Focus on feature-based branching, even if a feature touches multiple components. For truly independent components, consider a monorepo strategy with careful dependency management, or separate repositories. + +- **Long-Lived Feature Branches (without rebase/merge discipline)**: While feature branches are good, letting them live for weeks or months without integrating changes from the mainline. + + - **Why it's bad**: High risk of merge conflicts, stale code, and difficulty integrating into the main codebase. + + - **Instead**: Encourage frequent merging/rebasing from `main` (or `develop`) into the feature branch, and aim for short-lived branches (days, not weeks). + +- **No Branching Strategy at All**: The "wild west" approach where developers create branches haphazardly or commit directly to `main` without any rules. + + - **Why it's bad**: Pure chaos, frequent broken builds, difficult to track changes, and no quality gates. + + - **Instead**: Even a simple strategy like GitHub Flow is better than no strategy. Define and document your team's approach. + +- **Environment Branching (in modern contexts)**: As discussed earlier, using separate branches for `dev`, `QA`, `prod` etc., and promoting code by merging between them. + + - **Why it's bad**: Leads to divergence, makes it hard to trace changes, and doesn't scale with modern CI/CD. + + - **Instead**: Use CI/CD pipelines to deploy from a single, stable `main` branch to different environments, relying on tags or specific commits for releases. + + +## Monorepo-Specific Considerations + +While many branching strategies apply to both single repositories and monorepos, the unique characteristics of monorepos introduce specific considerations: + +- **Increased Reliance on Trunk-Based Development**: Monorepos often thrive on trunk-based development. With many teams contributing, long-lived feature branches can quickly lead to merge conflicts across numerous interdependent packages. Frequent, small commits to `main` (with feature flags) are crucial. + +- **Atomic Commits Across Packages**: Changes that span multiple packages within the monorepo should ideally be committed atomically to `main`. This ensures that the entire codebase remains consistent and buildable at all times. + +- **Build System Importance**: A robust build system (e.g., Bazel, Nx, Lerna) is paramount. It must be able to: + + - Efficiently build only affected packages. + + - Manage dependencies between packages. + + - Run tests only for changed code and its dependents. This allows fast CI/CD cycles even with a large codebase. + +- **Feature Flagging**: Essential for deploying incomplete features to `main` without impacting production users, especially when features span multiple packages and teams. + +- **Code Ownership and Review**: Clear guidelines on code ownership and streamlined review processes are critical to prevent bottlenecks in a high-volume monorepo. + + +## Managing External Dependencies with Git Submodules (e.g., RTOS, IP Stack) + +For embedded applications, it's common to pull in external components like an RTOS (Real-Time Operating System) or an IP stack from separate, often third-party, repositories. Git submodules provide a way to manage these external dependencies within your main project repository. + +A Git submodule allows you to embed one Git repository inside another Git repository as a subdirectory. It records the specific commit (SHA-1) of the external repository that your main project uses. + +**How Git Submodules Work:** + +1. **Adding a Submodule**: `git submodule add ` This clones the external repository into the specified path and adds an entry to your `.gitmodules` file and a special "gitlink" entry to your index. + +2. **Cloning a Project with Submodules**: `git clone --recurse-submodules ` Or, after cloning the main repository: `git submodule init` (to initialise local configuration files) `git submodule update` (to clone the submodules into their respective directories) + +3. **Updating a Submodule**: To update a submodule to a newer version (a different commit) of its external repository: `cd ` `git pull origin ` (or `git checkout `) `cd ../..` (back to main repo root) `git add ` (to record the new submodule commit) `git commit -m "Updated submodule X to new version"` + + +**Managing Submodules in an Embedded Application (RTOS/IP Stack Example):** + +- **Isolation**: The RTOS or IP stack lives in its own repository, allowing independent development, versioning, and bug fixing by its maintainers (who might be a third party or a separate internal team). + +- **Reproducibility**: Your main embedded application repository pins the exact version (commit SHA) of the RTOS/IP stack it was built and tested against. This ensures that anyone cloning your project gets the exact same versions of all dependencies, crucial for embedded development. + +- **Controlled Updates**: You decide _when_ to update the RTOS or IP stack. You pull changes into the submodule, test them thoroughly within your application's context, and only then commit the submodule's new SHA to your main repository. This prevents unexpected breaking changes from external updates. + +- **Branching Strategy Impact**: Your main application's branching strategy (`main`, `feature` branches, etc.) now also dictates how updates to submodules are integrated. For example, a new RTOS version might be integrated on a `feature` branch, thoroughly tested, and then merged into `main` like any other feature. + + +**Pros of Git Submodules for External Dependencies:** + +- **Strong Version Pinning**: Guarantees exact dependency versions, critical for reproducible builds in embedded systems. + +- **Clear Separation**: Keeps external components separate from your application code. + +- **Controlled Integration**: You explicitly choose when to pull in updates. + + +**Cons of Git Submodules:** + +- **Complexity**: Can be tricky to manage, especially for new users. Operations like cloning, updating, and branching can require specific submodule commands. + +- **Detached HEAD**: Working directly within a submodule often puts it in a detached HEAD state, which can be confusing. + +- **Branching Challenges**: If the submodule repository has its own complex branching, managing which submodule branch corresponds to which main project branch can be difficult. + +- **No Dependency Resolution**: Git submodules simply point to a commit; they don't resolve transitive dependencies or manage build order like package managers. + + +For embedded applications, submodules offer precise version control, but teams sometimes avoid their complexity. The alternatives are a package manager where the ecosystem has one (such as PlatformIO), or simply copying the code in when it is very stable and rarely updated. However, for actively developed external components, submodules remain a common and effective solution. + +## Metrics for Success + +To truly understand if your chosen branching strategy is effective, it's crucial to track relevant metrics. These metrics provide objective data to inform continuous improvement and architectural decisions. + +- **Merge Conflict Rate**: + + - **Definition**: The frequency or percentage of merges that result in conflicts requiring manual resolution. + + - **Why it matters**: High conflict rates indicate long-lived branches, large changes being integrated infrequently, or insufficient communication/collaboration. It directly impacts developer productivity and morale. + + - **Goal**: Aim for a low conflict rate, ideally close to zero, especially on `main` or `develop` branches. + +- **Lead Time for Changes (Cycle Time)**: + + - **Definition**: The total time from when a code change is first committed until it is successfully deployed to production. + + - **Why it matters**: A key DevOps metric. Shorter lead times indicate a more efficient and agile development process, allowing faster feedback and quicker delivery of value. Branching strategies that promote continuous integration (like trunk-based development) typically lead to shorter lead times. + + - **Goal**: Continuously reduce lead time. + +- **Deployment Frequency**: + + - **Definition**: How often your team deploys code to production. + + - **Why it matters**: High deployment frequency is a hallmark of high-performing teams. It indicates that your branching strategy and CI/CD pipeline enable rapid, low-risk releases. + + - **Goal**: Increase deployment frequency, ideally to multiple times per day for mature teams. + +- **Change Failure Rate / Rollback Frequency**: + + - **Definition**: The percentage of deployments that result in a failure (e.g., outage, degradation) requiring a rollback or hotfix. + + - **Why it matters**: A high failure rate suggests issues with quality gates (testing, reviews) within your branching strategy or CI/CD. + + - **Goal**: Maintain a very low change failure rate. + +- **Branch Lifetime**: + + - **Definition**: The average duration that a feature or topic branch exists before being merged or closed. + + - **Why it matters**: Longer branch lifetimes correlate with higher merge conflict rates and increased integration risk. + + - **Goal**: Keep branch lifetimes short, especially for feature branches (e.g., less than a few days). + +- **Pull Request (PR) Metrics**: + + - **PR Size**: Number of lines changed or files affected per PR. Smaller PRs are generally easier to review and merge. + + - **PR Review Time**: Time from PR creation to approval. Long review times can indicate bottlenecks or large PRs. + + - **PR Merge Rate**: Percentage of PRs that are eventually merged. + + +By regularly monitoring these metrics, teams can gain objective insights into the health of their branching strategy and identify areas for improvement. + +## The Power of Communication in Git Workflows + +No matter how perfect your branching strategy is on paper, its success hinges on clear and consistent communication within your team. + +- **Before You Start**: Discuss the scope of your work with teammates, especially if you anticipate touching shared areas of the codebase. A quick chat can prevent hours of merge conflict resolution. + +- **During Development**: + + - **Frequent Updates**: Keep your team informed about your progress on a feature branch. + + - **Early Warnings**: If you encounter significant refactoring or complex changes that might impact others, communicate early. + + - **Pair Programming/Mob Programming**: These practices inherently improve communication and reduce branching issues by having multiple eyes on the same code. + +- **During Code Review (Pull Requests)**: + + - **Clear Descriptions**: Write concise and informative pull request descriptions. + + - **Constructive Feedback**: Provide clear, actionable feedback during reviews. + + - **Open Dialogue**: Use PR comments for discussion, not just approvals. + +- **Conflict Resolution**: + + - **Don't Suffer Alone**: If you hit a major merge conflict, ask for help! Often, the person who introduced the conflicting change can help resolve it quickly. + + - **Communicate Intent**: When resolving conflicts, understand the intent behind both sets of changes to ensure the correct resolution. + +- **Documentation**: Your branching strategy and workflow rules should be clearly documented and easily accessible (e.g., in your project's `README` or a dedicated "development guidelines" document). Review and update it regularly. + + +## Common Migration Paths Between Strategies + +Teams often evolve their branching strategies as their size, project complexity, and release cadence change. Understanding these common migration paths can help anticipate future needs and plan for transitions effectively. + +### 1\. From Feature Branching / GitHub Flow to Trunk-Based Development + +- **Reasons for Migration:** + + - **Increased Deployment Frequency:** As teams mature and aim for continuous delivery (deploying multiple times a day), the overhead of merging many feature branches becomes a bottleneck. trunk-based development, with its focus on small, frequent commits to `main`, is ideal for this. + + - **Reduced Merge Conflicts:** With more developers and faster iteration, long-lived feature branches lead to frequent and complex merge conflicts. trunk-based development minimizes this by keeping the `main` branch constantly integrated. + + - **Improved Feedback Loops:** trunk-based development enables faster feedback from CI/CD pipelines, as changes are integrated into the main line more quickly. + +- **Typical Triggers:** Growth in team size, adoption of DevOps principles, desire for faster time-to-market. + + +### 2\. From GitFlow to GitHub Flow / One Flow / Trunk-Based Development + +- **Reasons for Migration:** + + - **Reduced Complexity:** GitFlow's numerous long-lived branches (develop, release, hotfix) can be complex to manage for teams that don't need its full structure. + + - **Faster Delivery:** GitFlow's formal release cycles can slow down delivery for teams aiming for continuous deployment. Simpler flows like GitHub Flow or trunk-based development enable much faster releases. + + - **Less Manual Overhead:** The multiple merge steps and branch management in GitFlow can be manual and error-prone. Modern flows often rely more heavily on automation. + +- **Typical Triggers:** Shift towards continuous delivery, simplification of release processes, smaller team size, or a desire to reduce operational overhead. + + +### 3\. From Environment Branching (Anti-Pattern) to GitHub Flow / Trunk-Based Development + +- **Reasons for Migration:** + + - **Eliminating Divergence:** Environment branching often leads to different codebases in different environments, making debugging and consistency a nightmare. + + - **Enabling CI/CD:** Modern CI/CD pipelines are designed to deploy a single artefact from a single, stable branch (`main`) to various environments, rather than merging between environment-specific branches. + + - **Improved Traceability:** It's harder to trace specific changes through multiple environment branches. A single source of truth (`main`) with clear deployments is more transparent. + +- **Typical Triggers:** Adoption of modern DevOps practices, struggles with environment consistency, desire for automated deployments. + + +### 4\. From Feature Branching (Basic) to GitHub Flow (with PRs and CI) + +- **Reasons for Migration:** + + - **Formalizing Code Review:** Basic feature branching might not enforce code reviews. GitHub Flow explicitly integrates pull requests as a mandatory step. + + - **Automated Quality Gates:** Adding CI/CD to run tests on every pull request ensures higher code quality before merging. + +- **Typical Triggers:** Desire to improve code quality, reduce bugs, and increase team collaboration through structured reviews. + + +### Key Considerations During Migration + +- **Pilot Programme:** Start with a small team or project to test the new strategy before rolling it out widely. + +- **Training:** Provide comprehensive training to the entire team on the new workflow, including Git commands and best practices. + +- **Tooling Alignment:** Ensure your CI/CD tools, code hosting platform, and other development tools support the new strategy. + +- **Clear Communication:** Explain _why_ the change is happening, the expected benefits, and address any concerns from the team. + +- **Feature Flags:** These are often crucial when moving to faster, more integrated strategies like trunk-based development, as they allow incomplete features to be merged without impacting production. + + +## Comparison Table + +Here's a quick comparison of the main strategies: + +Strategy + +Complexity + +Release Cadence + +Best For + +Key Feature + +**GitFlow** + +High + +Scheduled / Formal + +Large teams, multiple versions, strict releases + +Dedicated branches for features, releases, hotfixes + +**GitHub Flow** + +Low + +Continuous / Rapid + +Small-mid teams, frequent deployments + +Single `main` branch, short-lived feature branches + +**GitLab Flow** + +Medium + +Flexible / CI/CD + +Teams using GitLab, environment mapping + +Integrates with CI/CD, environment branches (optional) + +**Trunk-Based Development** + +Low + +Continuous / Daily + +High-performing teams, continuous delivery + +Single `main` (trunk), feature flags + +**Release Branching** + +Medium + +Scheduled / LTS + +Multiple product versions, formal QA + +Dedicated branches for specific releases + +**Feature Branching** + +Low + +Flexible + +New teams, isolated development + +Branch per feature, merge via PR + +**Forking Workflow** + +Medium + +Flexible + +Open-source projects, external contributors + +Forking for contributions, strict control + +**One Flow** + +Medium + +Continuous / Tagged + +Simpler GitFlow alternative, clear releases + +`main` + feature branches, tagged releases + +**Ship / Show / Ask** + +Low + +Continuous / Rapid + +trunk-based development teams, risk-based PR workflow + +Flexible PR process based on change risk + +**Mainline Dev. (Release Trains)** + +High + +Fixed Cadence + +Very large organisations, complex products + +Mainline + periodic release train branches + +## Evolution of Strategies + +Your team's branching strategy isn't set in stone. As your team grows, matures, and your product evolves, your workflow might need to adapt. Here's a common path of evolution: + +1. **Starting Simple (Feature Branching / GitHub Flow)**: + + - New or small teams often begin here. They are easy to understand and implement, allowing teams to get comfortable with Git and pull requests. + + - `Feature Branching` provides clear isolation. + + - `GitHub Flow` pushes towards continuous integration and deployment. + +2. **Adding Structure (GitFlow / One Flow / Release Branching)**: + + - As a team grows, releases become more formal, or multiple product versions need support, more structure might be required. + + - `GitFlow` is a natural progression for teams needing strict release cycles and hotfix management. + + - `One Flow` offers a lighter alternative to GitFlow, maintaining release clarity without as much overhead. + + - `Release Branching` becomes crucial if you need to support older versions while developing new features. + +3. **Embracing Speed & Automation (Trunk-Based Development / Mainline Development)**: + + - For teams aiming for high-frequency deployments and true continuous delivery, the focus shifts to minimal branching and maximum automation. + + - `Trunk-Based Development` is the pinnacle of this, requiring significant investment in automated testing and feature flagging. + + - `Mainline Development with Release Trains` is the enterprise-scale version of trunk-based development, managing many teams and fixed release cadences. + + +This evolution isn't linear for every team. Sometimes, a smaller team might jump straight to trunk-based development if they have a strong DevOps culture and robust automation from day one. The key is to **start simple, iterate, and adapt** as your needs change. Regular retrospectives on your workflow can help identify when it's time to evolve. + +## Final Thoughts + +Choosing a Git branching strategy is ultimately about finding the right balance between structure and flexibility, and tailoring it to how your team works. Consider your team size, experience level, release cadence, and product complexity when making the call. + +Start simple. Pick the strategy that meets your current needs without overcomplicating things. Establish clear guidelines, evolve as your team matures, and most importantly, be consistent. The best branching strategy is the one your team understands and follows every day. + +At the end of the day, your branching strategy isn't just a technical detail; it's a core part of how your team collaborates, reviews, and ships software. Choose wisely, implement intentionally, and you'll turn version control from a source of chaos into a well-oiled machine. Happy coding! 💻 diff --git a/content/guides/git/03-choosing-a-version-control-system.md b/content/guides/git/03-choosing-a-version-control-system.md new file mode 100644 index 0000000..dfdb8a3 --- /dev/null +++ b/content/guides/git/03-choosing-a-version-control-system.md @@ -0,0 +1,57 @@ +--- +title: Choosing a version control system +--- + +# Choosing a version control system + +A version control system (VCS), like Git, is the backbone of our **Docs as Code** philosophy. When managing complex, integrated products, it's essential to consider how our VCS strategy impacts the entire team, particularly in a multi-disciplinary environment. The goal is to accommodate different paces of work while maintaining a single source of truth and clear traceability. + +This guide outlines key factors to consider when designing a version control strategy, rather than dictating a single, rigid workflow. + +## The Challenge of Multi-Disciplinary Repositories + +In a multi-disciplinary product (software, firmware, hardware), not all teams operate at the same pace. Software and firmware can iterate rapidly, while hardware has long lead times and high iteration costs. Your branching strategy must accommodate this reality while maintaining a single source of truth and clear traceability. + +## Key Considerations for Your Strategy + +Instead of adopting a prescriptive workflow, it's better to design a strategy based on a few core principles. These principles will help you create a system that works for your team's unique context and constraints. + +### 1. The Single Source of Truth + +This is about defining a definitive reference point for the latest, validated state of the product. It’s the branch that represents what’s released to customers or ready to be built. + +* **Core Principle:** Every strategy should aim to create a single, authoritative branch for the latest, validated version of the entire product. This is often a `main` or `trunk` branch that is highly protected. + +* **What to Consider:** What rules need to be in place (e.g., automated checks, required reviews) to ensure only validated work is integrated into this branch? How will you handle hotfixes and urgent changes? + +### 2. Decoupling Workflows + +Your strategy should allow different teams to work at their own pace without blocking each other. This is about managing the inherent tension between agile, fast-paced work and structured, long-cycle work. + +* **Fast-Paced Work:** For software, a lightweight branching model (like short-lived feature branches) can support rapid iteration and continuous integration. This keeps the main branch clean and makes it easy for teams to collaborate on small, incremental changes. + +* **Structured Work:** For hardware, which might have longer design and manufacturing cycles, a more structured approach might be necessary. This could involve creating longer-lived branches that align with physical hardware revisions or formal releases. These branches serve as stable targets for software and firmware teams. + +This hybrid approach allows teams to work in parallel. Imagine a hardware team working on `hardware-revA` while a software team iterates on new features on a `develop` branch. When a new hardware prototype is ready, the software team can easily integrate their latest code with the stable hardware branch for testing. + +### 3. Traceability and Historical Context + +Your VCS strategy should make it easy to understand the "why" behind every change. This is essential for debugging issues, onboarding new team members, and satisfying compliance requirements. + +* **Meaningful Commits:** Require commits to be a single, logical unit of work with clear, descriptive messages. A good practice is to adopt a convention like `[Category]: [Description]` (e.g., `docs: Add ADR template` or `feat: Implement new API endpoint`). This provides context at a glance. + +* **Pull Request Discipline:** Use pull requests to document the discussion, trade-offs, and approvals for every change. This provides a rich history that goes beyond just the code itself and serves as a living record of decisions. + +* **Release Tagging:** Use tags to mark important milestones, such as a formal product release. This allows you to easily find the exact state of the product (code, docs, and all designs) at any point in time. + +## Best Practices for All Artifacts + +Regardless of your chosen strategy, these best practices apply to every type of artefact in your repository: + +* **Atomic Commits:** Each commit should be a logical, indivisible unit of work. For documentation, this means a single change per commit (e.g., "docs: Add ADR template"). This makes it easy to revert changes and see the history of a document. + +* **Use LFS for Large Files:** If your repository includes large files like hardware design files (CAD, PCB layouts), use a large file storage solution. This keeps the repository size manageable and prevents cloning times from becoming a bottleneck for the entire team. + +* **Continuous Integration (CI):** Automate checks on every push and PR. For code, this means running linters and tests. For documentation, you can use a tool like `markdownlint` to check for formatting errors. For hardware, you could even have a CI job that validates design files against a set of rules. + +* **Branch Protection:** Enforce rules to prevent direct commits to your main branches. This ensures all changes go through a proper review and validation process before being integrated. This simple rule is one of the most effective ways to maintain a clean and reliable codebase. diff --git a/content/guides/git/index.md b/content/guides/git/index.md new file mode 100644 index 0000000..87c2b68 --- /dev/null +++ b/content/guides/git/index.md @@ -0,0 +1,17 @@ +--- +title: Git +--- + +# Git + +A practical guide to Git for developers: how it works underneath, the branching strategies teams use, and how to choose a version control system. + +The rules for commits, branches and versions are in [EG-014](../../guidelines/version-control-and-versioning.md), and the [Git primer](../../primers/git-best-practices.md) is a shorter introduction. + +| Chapter | Covers | +| --- | --- | +| [How Git works](01-how-git-works.md) | Snapshots and the commit graph, the index, branches, merging, rebasing, reflog, bisect, tags, reset and revert, fixing mistakes | +| [Branching strategies](02-branching-strategies.md) | Trunk-based development, GitHub flow, GitFlow and release branches, back-porting, merge strategies, anti-patterns, monorepos | +| [Choosing a version control system](03-choosing-a-version-control-system.md) | What to weigh when picking or changing a version control system | + +[[CAPTION:Table]] The chapters of the Git guide. diff --git a/content/guides/high-level-design.md b/content/guides/high-level-design.md new file mode 100644 index 0000000..7756a72 --- /dev/null +++ b/content/guides/high-level-design.md @@ -0,0 +1,177 @@ +--- +title: Writing a high-level design +--- + +# Writing a high-level design + +This guideline defines the engineering standard, narrative structure, and quantitative capacity estimation framework for authoring High-Level Design (HLD) documents. It bridges the gap between Product Requirements Documents (PRDs) or Business Architecture and low-level Software Design Specifications ([Writing a software design specification](software-design-specification.md)). + +## 1. Purpose and Architecture Context + +A High-Level Design document establishes a shared architectural vision for a proposed system or major capability. It enables technical leaders, software engineers, security specialists, and operational teams to evaluate structural soundness, identify technical risks, and align on integration patterns before detailed implementation begins. + +```mermaid +flowchart LR + A["Business Requirements / PRD"] --> B["High-Level Design (HLD)"] + B --> C["Software Design Specification (SDS)"] + B --> D["Architecture Decision Records (ADR)"] + C --> E["Implementation and Testing"] +``` + +[[CAPTION:Figure]] High-Level Design within the systems engineering lifecycle. + +### 1.1 When an HLD is Required + +An HLD must be drafted, reviewed, and approved via the Architecture Review Board ([Running an architecture review board](architecture-review-board.md)) under any of the following conditions: + +- A new software service, container, embedded device platform, or cloud subsystem is being introduced. +- A cross-cutting change impacts two or more bounded contexts, teams, or external API contracts. +- Significant changes are proposed to data persistence topologies, messaging protocols, or network boundaries. +- The system introduces non-trivial operational cost, strict latency budgets, or regulatory compliance obligations (such as ETSI EN 303 645, UK PSTI, or GDPR). + +### 1.2 Narrative Structure: Architecture as a Story + +An effective HLD functions as an engaging technical narrative. Rather than dumping uncontextualised class diagrams or database schemas, the author should guide the reader through: + +- **The Context**: What problem are we solving, why now, and what are the explicit system boundaries? +- **The Blueprint**: High-level component decomposition and data flows that an executive or non-engineer can grasp. +- **The Calculations**: Grounded capacity sizing (traffic, storage, memory, bandwidth) that validates technical feasibility. +- **The Trade-offs**: Why chosen approaches triumphed over viable alternatives. +- **The Operability**: How the system will be deployed, observed, secured, and maintained in production. + +## 2. Capacity Estimation and System Sizing Framework + +Quantitative sizing transforms architectural assertions into measurable engineering requirements. The following mathematical framework must be used in the HLD to size compute, storage, caching, and network bandwidth. + +### 2.1 Traffic and Request Rate Calculations + +To determine Queries Per Second (QPS), calculate both average and peak loads. Peak load typically introduces a peak multiplier ($M_p$) between $2.0$ and $4.0$ depending on user behaviour patterns and batch telemetry schedules. + +$$\mathrm{QPS}_{\mathrm{avg}} = \frac{\mathrm{Daily\ Active\ Users\ (DAU)} \times \mathrm{Requests\ per\ User\ per\ Day}}{86{,}400 \mathrm{seconds}}$$ + +$$\mathrm{QPS}_{\mathrm{peak}} = \mathrm{QPS}_{\mathrm{avg}} \times M_p$$ + +For example, an IoT telemetry ingestion platform serving 50,000 connected smart gateways, each transmitting a telemetry packet once every 30 seconds: + +- Packets per gateway per day: $86{,}400 / 30 = 2{,}880$ +- Total daily events: $50{,}000 \times 2{,}880 = 144{,}000{,}000$ events/day +- Average ingestion QPS: $144{,}000{,}000 / 86{,}400 = 1{,}667 \mathrm{QPS}$ +- Peak ingestion QPS ($M_p = 2.5$ during post-outage reconnection storm): $1{,}667 \times 2.5 = 4{,}167 \mathrm{QPS}$ + +### 2.2 Storage Sizing and Growth Projections + +Storage calculations must account for raw payload size, database indexing overhead (typically $25\%\mathrm{to}35\%$), and cluster replication factors (for example, $3\times$ replication in distributed datastores such as Kafka, Cassandra, or PostgreSQL clusters). + +$$\mathrm{Daily\ Raw\ Data} = \mathrm{Daily\ Ingested\ Records} \times \mathrm{Average\ Payload\ Size\ (Bytes)}$$ + +$$\mathrm{Daily\ Total\ Storage} = \mathrm{Daily\ Raw\ Data} \times (1 + \mathrm{Index\ Overhead}) \times \mathrm{Replication\ Factor}$$ + +$$\mathrm{Annual\ Storage} = \mathrm{Daily\ Total\ Storage} \times 365$$ + +| Metric Parameter | Ingestion Telemetry Example | User Profile Service Example | +| :--- | :--- | :--- | +| Daily Records | $144{,}000{,}000$ packets | $5{,}000{,}000$ profile updates | +| Average Size per Record | $600\mathrm{bytes}$ (CBOR / JSON) | $4\mathrm{KB}$ | +| Daily Raw Volume | $86.4\mathrm{GB}$ | $20.0\mathrm{GB}$ | +| Indexing Overhead | $30\%$ ($1.30\times$) | $25\%$ ($1.25\times$) | +| Cluster Replication Factor | $3\times$ | $3\times$ | +| Effective Daily Storage | $336.96\mathrm{GB/day}$ | $75.00\mathrm{GB/day}$ | +| 1-Year Uncompressed Storage | $123.0\mathrm{TB/year}$ | $27.4\mathrm{TB/year}$ | +| 3-Year Projected Footprint | $369.0\mathrm{TB}$ | $82.2\mathrm{TB}$ | + +[[CAPTION:Table]] Capacity sizing reference table for multi-year storage planning. + +### 2.3 Network Bandwidth Sizing + +Network capacity must specify ingress and egress throughput, including transport overhead (TLS handshakes, HTTP/2 frame headers, or MQTT wrapper headers): + +$$\mathrm{Bandwidth}_{\mathrm{Ingress}} = \mathrm{QPS}_{\mathrm{peak}} \times \mathrm{Average\ Request\ Payload\ Size}$$ + +$$\mathrm{Bandwidth}_{\mathrm{Egress}} = \mathrm{QPS}_{\mathrm{peak}} \times \mathrm{Average\ Response\ Payload\ Size}$$ + +Using the 4,167 peak QPS telemetry scenario with a 600-byte payload: + +- Raw Ingress: $4{,}167 \times 600\mathrm{bytes/sec} = 2.50\mathrm{MB/s} = 20.0\mathrm{Mbps}$ +- Acknowledgment Egress ($100\mathrm{bytes}$ response): $4{,}167 \times 100\mathrm{bytes/sec} = 0.42\mathrm{MB/s} = 3.36\mathrm{Mbps}$ + +### 2.4 In-Memory Caching Sizing (Pareto 80/20 Rule) + +When designing low-latency caching layers (Redis, Memcached), architects should apply the Pareto principle: approximately $20\%$ of hot entities account for $80\%$ of read traffic. + +$$\mathrm{Cache\ Working\ Set} = \mathrm{Daily\ Unique\ Active\ Records} \times 0.20 \times \mathrm{Cached\ Entity\ Size} \times (1 + \mathrm{Metadata\ Overhead})$$ + +If $10{,}000{,}000$ entities are read per day with an average entity size of $2\mathrm{KB}$ and $15\%$ memory allocator overhead: + +- $\mathrm{Working\ Set} = 10{,}000{,}000 \times 0.20 \times 2\mathrm{KB} \times 1.15 = 4.6\mathrm{GB}$ RAM working set required. + +## 3. High-Level Design Document Template + +The following template outlines the required sections for all formal HLD submissions. Authors must clone and complete this structure. + +### 3.1 Template Outline + +```markdown +# High-Level Design: [System or Service Name] + +| Document Attribute | Value | +| :--- | :--- | +| Document Identifier | HLD-[Project]-[Number] | +| Author(s) | [Name, Role] | +| Reviewer(s) / Approvers | [ARB Lead, Principal Engineer, SecOps Lead] | +| Target Release / Milestone | [Milestone / Sprint / Date] | +| Status | [Draft / Under Review / Approved / Superseded] | + +[[CAPTION:Table]] HLD Document metadata. + +## 1. Executive Summary and Context +- **Problem Statement**: Brief 2-3 paragraph explanation of the business need, customer pain point, or technical limitation. +- **Goals**: Explicit, measurable outcomes (for example, "Process 5,000 sensor readings/sec with p99 latency < 200 ms"). +- **Non-Goals**: Explicitly out-of-scope capabilities to prevent scope creep. +- **Assumptions and Dependencies**: External services, hardware availability, or upstream team deliverables. + +## 2. High-Level Architecture Overview +- **System Context Diagram**: C4 Level 1 diagram illustrating users, external systems, and boundaries. +- **Key Architectural Decisions**: Core technology choices (languages, frameworks, datastores) linking to ADRs ([Writing ADRs and RFCs](decision-records.md)). +- **Subsystem Decomposition**: High-level responsibilities of containers or microservices. + +## 3. Detailed Architecture and Component Interactions +- **Container / Component Diagram**: C4 Level 2 / Level 3 diagram showcasing communication protocols. +- **Data Flow and Sequence**: Key runtime sequence diagrams illustrating happy-path and edge-case execution. +- **Data Model and Persistence**: Logical schema, datastore selection rationale, indexing strategy, and partition keys. +- **API and Interface Contracts**: Protobuf, OpenAPI, or AsyncAPI references complying with [API design and error handling](../standards/api-design.md). + +## 4. Capacity and Sizing Analysis +- **Traffic Model**: Daily Active Users / Devices, QPS average and peak calculations. +- **Storage and Growth**: Raw vs indexed storage, replication, and 3-year projection. +- **Network Bandwidth**: Ingress and egress throughput requirements. +- **Memory and Caching**: Cache hit-rate targets, eviction policies, and cluster memory sizing. + +## 5. Architectural Quality Attributes (Non-Functional Requirements) +- **Availability and Fault Tolerance**: Target SLA (99.9% / 99.99%), failover mechanics, circuit breakers. +- **Latency and Throughput**: p50, p95, and p99 latency targets under peak load. +- **Scalability**: Horizontal vs vertical scaling rules, autoscaling triggers (CPU, queue depth). +- **Security and Compliance**: STRIDE threat modelling, authentication/authorisation (OAuth2/mTLS), encryption at rest/in transit, compliance standards (ETSI EN 303 645, GDPR). +- **Data Integrity and Consistency**: ACID vs BASE, event ordering, idempotency mechanisms. + +## 6. Operability, Observability, and Deployment +- **Deployment Topology**: Cloud regions, multi-zone clustering, container orchestration (Kubernetes), or embedded firmware partition map. +- **Observability Strategy**: Metrics (RED/USE methods), distributed tracing (OpenTelemetry), structured log schemas. +- **Disaster Recovery and Backups**: Recovery Point Objective (RPO) and Recovery Time Objective (RTO). +- **Rollback and Migration Plan**: Blue/Green or Canary deployment strategy, schema migration sequencing. + +## 7. Trade-offs and Alternatives Considered +- **Alternative 1**: Description, evaluation, and rationale for rejection. +- **Alternative 2**: Description, evaluation, and rationale for rejection. +``` + +[[CAPTION:Figure]] High-Level Design document template structure and mandatory sections. + +## 4. Architectural Review and Gate Criteria + +All HLD documents must undergo formal evaluation prior to implementation sign-off. Reviewers assess documents against the McConnell-style Architecture Review Checklist defined in [Architecture review checklist](../templates/architecture-review-checklist.md): + +- **ARC-1 to ARC-6 (General Quality)**: Are goals clear, boundaries well-defined, and assumptions documented? +- **ARC-7 to ARC-14 (Design Considerations)**: Are loose coupling, high cohesion, and standard patterns utilised? +- **ARC-15 to ARC-23 (Operational & Quality Requirements)**: Are capacity estimates mathematically verified, failure modes mitigated, and security controls explicit? + +Upon successful review, the HLD is registered in the Architecture Registry and referenced by subsequent Software Design Specifications ([Writing a software design specification](software-design-specification.md)). diff --git a/content/guides/hil-and-automated-testing/01-the-iot-testing-pyramid.md b/content/guides/hil-and-automated-testing/01-the-iot-testing-pyramid.md new file mode 100644 index 0000000..2f02ba3 --- /dev/null +++ b/content/guides/hil-and-automated-testing/01-the-iot-testing-pyramid.md @@ -0,0 +1,69 @@ +--- +title: The IoT Testing Pyramid +--- + +# The IoT Testing Pyramid + +In traditional web development, testing takes place within standardized, homogeneous virtual machines or container environments. In embedded systems and IoT engineering, software execution is inextricably linked to analog electronics, peripheral bus timing, radio link instability, battery degradation, and environmental extremes. + +Attempting to run all tests on physical hardware leads to severe development bottlenecks: hardware benches are expensive, scarce, and fragile. Conversely, testing solely in software simulators results in defect escapes when code encounters real silicon errata. + +This chapter introduces the **Four-Layer Embedded Testing Pyramid**, analyzes the economics of defect discovery, and establishes test execution allocations across the delivery lifecycle. + +## 1. The Four-Layer Embedded Testing Pyramid + +To maximise testing velocity while maintaining absolute hardware reliability, engineering teams must organise testing across four distinct layers: + +```mermaid +flowchart TD + subgraph Pyramid["The Embedded Testing Pyramid"] + L1["Layer 1: Host-Native Unit Tests
(Ceedling / Catch2 / GTest)
Execution: Seconds | Cost: Minimal | Scope: 70%"] + L2["Layer 2: Emulated System Integration
(QEMU / Renode)
Execution: Minutes | Cost: Low | Scope: 20%"] + L3["Layer 3: Hardware-in-the-Loop (HIL) Racks
(Real Silicon + SCPI Instruments)
Execution: 15-60 Mins | Cost: Moderate | Scope: 8%"] + L4["Layer 4: Pre-Production Field Pilot
(Canary Deployments in Real World)
Execution: Days/Weeks | Cost: High | Scope: 2%"] + end + L1 --> L2 + L2 --> L3 + L3 --> L4 +``` + +[[CAPTION:Figure]] The four layers of the connected systems testing pyramid. + +### 1.1 Layer Characteristics & Responsibilities + +| Test Layer | Execution Target | Execution Time | Infrastructure | Key Focus Areas | +| :--- | :--- | :--- | :--- | :--- | +| **Layer 1: Host Unit** | Developer x86/ARM host | < 30 seconds | Cloud CI runners | Business logic, state machines, math algorithms, CBOR/Protobuf serialization | +| **Layer 2: Virtual Emulation**| QEMU / Renode emulator | 1 to 5 minutes | Cloud CI runners | RTOS scheduling, multi-task IPC, peripheral driver state machines, network stack | +| **Layer 3: Physical HIL** | Real PCBA & Silicon | 15 to 60 minutes | Self-hosted hardware racks | Analog sensors, sleep currents, flash wear, power brownouts, conducted RF | +| **Layer 4: Field Pilot** | Deployed canary fleet | 1 to 4 weeks | Real cellular/satellite links | Real-world coverage dead zones, roaming handovers, weather cycling | + +[[CAPTION:Table]] Characteristics of the four embedded testing layers. + +## 2. Shift-Left Economics of Embedded Defect Discovery + +The financial and operational cost of remediating a firmware defect increases by an order of magnitude with every development stage it escapes: + +```mermaid +xychart-beta + title "Relative Cost to Remediate Firmware Defects by Discovery Phase" + x-axis ["Host Unit Test", "Emulation", "Physical HIL", "Factory Test", "Field Deployment"] + y-axis "Relative Cost ($)" 0 --> 10000 + bar [10, 50, 250, 1500, 10000] +``` + +[[CAPTION:Figure]] Exponential escalation of bug remediation costs across the hardware lifecycle. + +1. **Host Unit Level ($10)**: A developer identifies an integer overflow or buffer overrun within seconds via unit tests before pushing code. +1. **Physical HIL Level ($250)**: An automated overnight test rack catches an I2C bus lockup or excessive sleep current prior to release candidate tagging. +1. **Factory Production Level ($1,500)**: A defective bootloader slips past bench testing and halts contract manufacturing assembly lines, idling factory workers. +1. **Field Deployment ($10,000+)**: A sleep state deadlock drains batteries across 50,000 deployed units, requiring emergency on-site technician truck rolls or complete product recalls. + +## 3. Recommended Test Allocation Strategy + +Engineering squads must enforce the **70 / 20 / 8 / 2 Rule**: + +- **70% of Tests in Layer 1 (Host Unit)**: Every algorithm, data structure, mathematical filter, and parser must execute on developer laptops and cloud CI runners without requiring physical microcontrollers. +- **20% of Tests in Layer 2 (Emulation)**: Multi-threaded FreeRTOS task interactions and network protocol handshakes must run inside headless QEMU or Renode instances on every pull request. +- **8% of Tests in Layer 3 (Physical HIL)**: Smoke test suites run on physical hardware test benches for pull requests; comprehensive electrical stress, power profiling, and fault injection suites run nightly. +- **2% of Tests in Layer 4 (Field Pilot)**: New firmware release candidates bake on internal dogfood devices and pre-production customer pilot units before full fleet distribution. diff --git a/content/guides/hil-and-automated-testing/02-host-based-firmware-testing.md b/content/guides/hil-and-automated-testing/02-host-based-firmware-testing.md new file mode 100644 index 0000000..2f1d023 --- /dev/null +++ b/content/guides/hil-and-automated-testing/02-host-based-firmware-testing.md @@ -0,0 +1,128 @@ +--- +title: Host-Based Firmware Testing +--- + +# Host-Based Firmware Testing + +Testing firmware directly on physical target microcontrollers slows development: flashing via SWD/JTAG takes seconds, setting hardware breakpoints disrupts real-time timers, and automated test runners require specialized physical racks. By decoupling application logic from physical silicon registers, engineering teams can execute thousands of unit tests natively on developer workstations and standard cloud CI runners in seconds. + +This chapter details the architecture of Hardware Abstraction Layers (HAL), test mocking using Ceedling and CMock, and memory safety sanitization using AddressSanitizer. + +## 1. Hardware Abstraction Layer (HAL) Decoupling + +To enable host-based compilation and testing, firmware must be structured with strict interface boundaries: + +```mermaid +flowchart TD + App["Application Logic & State Machines
(100% Host Testable)"] + HAL["Hardware Abstraction Layer (HAL Interfaces)
(gpio_read, i2c_transfer, flash_write)"] + + subgraph TargetBuild["Target Build (ARM GCC)"] + SiliconDriver["Silicon Register Drivers
(STM32 HAL / Nordic nrfx)"] + Hardware["Physical Microcontroller HW"] + end + + subgraph HostBuild["Host Build (Host GCC / Clang)"] + MockDriver["Generated Mocks (CMock)
(gpio_read_ExpectAndReturn)"] + UnitTests["Unit Test Runner (Unity / Catch2)"] + end + + App --> HAL + HAL --> SiliconDriver + SiliconDriver --> Hardware + HAL -.-> MockDriver + UnitTests --> MockDriver +``` + +[[CAPTION:Figure]] Architectural separation between testable application logic and target silicon drivers. + +### 1.1 Golden Rules for Host Testability + +1. **Never Access Silicon Registers Directly in Business Logic**: Writing `GPIOA->BSRR = (1 << 5);` directly inside a state machine couples application logic to specific silicon. Always route I/O operations through an abstracted HAL function: `hal_gpio_write(PIN_STATUS_LED, GPIO_HIGH);`. +1. **Encapsulate Real-Time Clocks**: Never call hardware SysTick registers directly to calculate timeouts. Inject a clock interface or pass elapsed milliseconds into state functions, allowing tests to advance simulated time deterministically. + +## 2. Test Mocking with Ceedling & CMock + +**Ceedling** combines the **Unity** test harness with **CMock**, an automated mock generator for C. When provided a C header file, CMock automatically generates mock implementations of every declared function: + +```c +// sensor_manager.h +#pragma once +#include +#include + +bool hal_i2c_read(uint8_t slave_addr, uint8_t reg, uint8_t *data, uint16_t len); +void hal_gpio_write(uint8_t pin_id, bool level); +``` + +When building host tests, CMock generates expectations: + +```c +// test_sensor_manager.c +#include "unity.h" +#include "mock_sensor_manager.h" +#include "telemetry_engine.h" + +void setUp(void) {} +void tearDown(void) {} + +void test_sensor_read_success_should_update_state(void) { + uint8_t simulated_payload[2] = {0x0A, 0x50}; // 26.40 C in fixed point + + // Set expectations on mock interfaces + hal_i2c_read_ExpectAndReturn(0x48, 0x01, NULL, 2, true); + hal_i2c_read_IgnoreArg_data(); + hal_i2c_read_ReturnArrayThruPtr_data(simulated_payload, 2); + + // Expect LED toggled high + hal_gpio_write_Expect(5, true); + + // Execute application function + telemetry_status_t status = telemetry_poll_sensors(); + + TEST_ASSERT_EQUAL(TELEMETRY_OK, status); + TEST_ASSERT_EQUAL_FLOAT(26.40f, telemetry_get_last_temp()); +} + +void test_sensor_read_failure_should_trigger_retry(void) { + // Simulate I2C hardware bus error + hal_i2c_read_ExpectAndReturn(0x48, 0x01, NULL, 2, false); + hal_i2c_read_IgnoreArg_data(); + + telemetry_status_t status = telemetry_poll_sensors(); + + TEST_ASSERT_EQUAL(TELEMETRY_BUS_ERROR, status); +} +``` + +## 3. Detecting Memory Bugs with AddressSanitizer (ASan) + +C firmware running on microcontrollers lacks virtual memory management (MMU); reading past the end of an array silently corrupts adjacent variables without triggering an immediate segmentation fault. + +Compiling host unit tests with **AddressSanitizer (ASan)** and **UndefinedBehaviorSanitizer (UBSan)** instantly flags memory bugs: + +```bash +gcc -fsanitize=address,undefined -g -O1 \ + -Isrc/ -Itests/ src/telemetry_engine.c tests/test_telemetry.c \ + -o run_tests && ./run_tests +``` + +```text +================================================================= +==12048==ERROR: AddressSanitizer: global-buffer-overflow on address 0x0001048 +READ of size 4 at 0x0001048 thread T0 + #0 0x1000018f0 in parse_cbor_buffer telemetry_engine.c:84 + #1 0x1000021c0 in test_parse_truncated tests/test_telemetry.c:42 +================================================================= +``` + +ASan detects buffer overflows, off-by-one errors, stack out-of-bounds reads, and memory leaks before the compiled binary is ever flashed onto physical hardware. + +## 4. Automated Code Coverage Gates in CI + +Every pull request must execute host tests and generate code coverage reports using `gcov` and `lcov`: + +1. **Coverage Standard**: Teams must enforce minimum coverage thresholds: + - Line Coverage: $\ge 85\%$ + - Branch Coverage: $\ge 80\%$ +1. **Zero Compiler Warnings**: The host test build must compile with `-Wall -Wextra -Werror -pedantic` enabled, treating all warnings as blocking errors. diff --git a/content/guides/hil-and-automated-testing/03-qemu-virtual-hardware-simulation.md b/content/guides/hil-and-automated-testing/03-qemu-virtual-hardware-simulation.md new file mode 100644 index 0000000..3c4082b --- /dev/null +++ b/content/guides/hil-and-automated-testing/03-qemu-virtual-hardware-simulation.md @@ -0,0 +1,108 @@ +--- +title: "QEMU & Virtual Hardware Simulation" +--- + +# QEMU & Virtual Hardware Simulation + +Host-based unit tests check pure algorithms and state machines. But they cannot run the real ARM Cortex-M binary, test real FreeRTOS context switches, or check how the code handles peripheral interrupts. Conversely, relying exclusively on physical hardware test benches for system integration creates severe bottlenecks in automated continuous integration (CI) pipelines. + +**Virtual Hardware Simulation** bridges this divide by running target-compiled binary images (`.elf` / `.bin`) inside headless silicon emulators such as **QEMU** and **Renode**. + +```mermaid +flowchart LR + Binary["Target ARM Binary
(firmware.elf)"] --> Sim{"Virtual Simulator"} + Sim -->|Single Core / Fast| QEMU["QEMU (ARM Cortex-M)
- Semihosting
- Virtual UART
- GDB Debug Stub"] + Sim -->|Full Board / Multi-Node| Renode["Renode (Antmicro)
- Peripheral Buses (I2C/SPI)
- External Flash Models
- Robot Framework Integration"] + QEMU --> CI["Automated Cloud CI Runner"] + Renode --> CI +``` + +[[CAPTION:Figure]] Positioning QEMU and Renode for automated virtual hardware simulation. + +## 1. Comparing QEMU and Renode + +| Capability | QEMU (Quick Emulator) | Renode (Antmicro) | +| :--- | :--- | :--- | +| **Primary Strength** | Raw instruction execution speed, GDB debugging | Deterministic multi-peripheral & multi-node networks | +| **Peripheral Modelling** | Limited to standard core peripherals (UART, SysTick) | Highly extensible (I2C sensors, SPI flash, radio transceivers) | +| **Multi-Node Mesh** | Complex network tap configuration | Native support for multi-device BLE and 802.15.4 simulation | +| **Test Automation** | CLI exit codes, semihosting assertions | Native Python API and Robot Framework keywords | +| **Startup Overhead** | Extremely fast (< 100 ms) | Moderate (~2 to 5 seconds) | + +[[CAPTION:Table]] Comparison of QEMU and Renode for embedded system simulation. + +## 2. Fast System Emulation with QEMU + +QEMU emulates target ARM Cortex-M instructions directly, allowing developers to verify RTOS task switching, scheduler behaviour, and stack usage without physical hardware: + +```bash +# Running ARM Cortex-M4 Firmware in Headless QEMU +qemu-system-arm -M lm3s6965evb \ + -cpu cortex-m4 \ + -nographic \ + -semihosting \ + -kernel build/firmware.elf +``` + +### 2.1 Semihosting for Automated CI Verification + +Semihosting enables code running inside the virtual ARM CPU to issue system calls directly to the host operating system: +- Print debug strings directly to the host console (`printf`). +- Read mock sensor configuration files from the host disk. +- Terminate the QEMU process cleanly with a return code (`exit(0)` for pass, `exit(1)` for fail), enabling seamless pass/fail status reporting in CI pipelines. + +## 3. Full-Board & Multi-Peripheral Simulation with Renode + +**Renode** provides cycle-accurate emulation of complete system-on-chip (SoC) architectures, including external SPI NOR flash chips, I2C temperature sensors, and wireless transceivers. + +```mermaid +flowchart TD + subgraph RenodeVM["Renode Virtual Machine"] + MCU["STM32 / Nordic Cortex-M Core"] + I2C["Virtual I2C Bus"] + Sensor["Emulated BME280 Sensor Model"] + SPI["Virtual SPI Bus"] + Flash["Emulated W25Q32 NOR Flash (4 MB)"] + end + MCU --> I2C + I2C --> Sensor + MCU --> SPI + SPI --> Flash +``` + +[[CAPTION:Figure]] Renode board model simulating core MCU and external peripheral components. + +### 3.1 Scripting System Tests with Robot Framework + +Renode integrates with **Robot Framework**, allowing test engineers to write human-readable end-to-end integration test scenarios: + +```robot +*** Settings *** +Suite Setup Setup +Suite Teardown Teardown +Test Teardown Test Teardown +Resource ${RENODEKEYWORDS} + +*** Test Cases *** +Verify Sensor Acquisition And LittleFS Flash Storage + [Documentation] Verifies that firmware reads the virtual I2C sensor, + ... logs reading to LittleFS, and reports over UART. + Execute Command mach create + Execute Command machine LoadPlatformDescription @platforms/boards/sensor-node.repl + Execute Command sysbus.i2c.bme280 Temperature 24.5 + Execute Command sysbus.i2c.bme280 Humidity 60 + Execute Command sysbus LoadELF @build/firmware.elf + + Create Terminal Tester sysbus.uart0 + Execute Command start + + Wait For Line On Uart [BOOT] Initializing LittleFS filesystem... OK + Wait For Line On Uart [SENSOR] BME280 Read: Temp=24.5C, Hum=60% + Wait For Line On Uart [STORAGE] Telemetry record committed to flash sector 4 +``` + +## 4. CI/CD Integration Best Practices + +1. **Test Concurrency in Cloud Runners**: Virtual hardware simulations are headless and container-friendly. A single GitHub Actions workflow can execute 20 Renode instances in parallel across different test suites within minutes. +1. **Deterministic Fault Injection**: Use Renode to inject peripheral timeouts (for example, withholding I2C acknowledgments) to verify that the firmware's bus recovery state machine executes properly. +1. **Stack Overflow & Memory Watermarking**: Use simulator memory watchpoints to verify that no RTOS task stack comes within 256 bytes of overflowing under worst-case interrupt nesting. diff --git a/content/guides/hil-and-automated-testing/04-hil-bench-design-and-automation.md b/content/guides/hil-and-automated-testing/04-hil-bench-design-and-automation.md new file mode 100644 index 0000000..e5f8d9b --- /dev/null +++ b/content/guides/hil-and-automated-testing/04-hil-bench-design-and-automation.md @@ -0,0 +1,145 @@ +--- +title: "HIL Bench Design & Automation" +--- + +# HIL Bench Design & Automation + +While virtual simulators validate logic and scheduling, they cannot detect subtle analog race conditions, crystal oscillator tolerances, radio transmitter harmonics, physical battery droop, or PCB layout flaws. Verifying embedded products before mass manufacturing requires an automated **Hardware-in-the-Loop (HIL)** test bench executing test suites on production-grade silicon. + +This chapter details physical HIL bench construction, automated instrumentation control using SCPI and PyVISA, dynamic power profiling, and pytest test harness orchestration. + +## 1. Physical HIL Test Rack Architecture + +A standardized automated HIL test rack encloses laboratory instrumentation, debug probes, and custom mechanical fixtures within an electrically isolated chassis: + +```mermaid +flowchart TD + subgraph Host["CI Test Orchestrator (Linux Host)"] + Pytest["Pytest Test Harness"] + PyVISA["PyVISA Instrument Driver"] + JLink["SEGGER J-Link CLI / Flasher"] + end + + subgraph Rack["HIL Hardware Instrumentation"] + PSU["SCPI Programmable Power Supply
(Keithley 2281S Battery Simulator)"] + DAQ["Data Acquisition (DAQ) System
(Relays, ADC, DAC, PWM Generator)"] + Atten["Programmable RF Attenuator
(Mini-Circuits 0-90 dB)"] + end + + subgraph Fixture["Pogo-Pin Bed-of-Nails Fixture"] + DUT["Device Under Test (Production PCBA)"] + Box["RF Shielded Enclosure (-80 dB Isolation)"] + end + + Pytest --> PyVISA + Pytest --> JLink + PyVISA --> PSU + PyVISA --> DAQ + PyVISA --> Atten + JLink -->|SWD Flash & RTT| DUT + PSU -->|Variable VCC Rail| DUT + DAQ -->|Analog & Digital Stimuli| DUT + Atten -->|Conducted RF Feed| DUT + DUT --> Box +``` + +[[CAPTION:Figure]] Comprehensive instrumentation topology of an automated HIL test bench. + +### 1.1 Key Hardware Components + +1. **Bed-of-Nails Test Fixture**: Uses spring-loaded gold-plated pogo pins to contact gold test pads on the unpopulated bottom layer of the target PCB. This provides non-intrusive electrical access to power rails, SWD debug lines, UART ports, and analog sensor pins without soldering wires. +1. **Battery Simulator & DC Power Analyzer**: Instruments such as the Keithley 2281S or Keysight N6705 simulate dynamic chemical battery behaviour, replicating internal cell resistance ($R_{internal}$) and voltage discharge curves while logging current from 10 nA to 2 A. +1. **RF Shield Box & Conducted Attenuation**: The DUT is enclosed in a shielded aluminum box providing $> 80\,\mathrm{dB}$ of radio isolation. Cellular antennas connect via coaxial cables through programmable step attenuators, allowing automated software to simulate distance and signal degradation. +1. **High-Speed Debug Probe**: A SEGGER J-Link flasher automates firmware flashing, resets the target MCU, and streams high-bandwidth debug logs via Real-Time Transfer (RTT) without stalling real-time execution. + +## 2. Test Automation Protocols: SCPI and PyVISA + +Automated HIL test frameworks control lab equipment using the **Standard Commands for Programmable Instruments (SCPI)** syntax transmitted over USB, Ethernet, or GPIB: + +```python +# conftest.py: Pytest Fixtures for Automated HIL Bench +import pytest +import pyvisa +import pylink +import time + +@pytest.fixture(scope="session") +def power_supply(): + """Initializes and returns the SCPI programmable power supply.""" + rm = pyvisa.ResourceManager() + psu = rm.open_resource("TCPIP0::192.168.1.105::inst0::INSTR") + psu.timeout = 5000 + psu.write("*RST") + psu.write("VOLT 3.600") + psu.write("CURR 1.000") + psu.write("OUTP ON") + yield psu + psu.write("OUTP OFF") + psu.close() + +@pytest.fixture(scope="session") +def flasher(): + """Initializes SEGGER J-Link for automated flashing and debugging.""" + jlink = pylink.JLink() + jlink.open() + jlink.set_tif(pylink.enums.JLinkInterfaces.SWD) + jlink.connect("STM32L476RG", speed_khz=4000) + yield jlink + jlink.close() +``` + +## 3. Writing Pytest HIL Verification Tests + +With instrumentation encapsulated into reusable pytest fixtures, test engineers write expressive, deterministic integration tests: + +```python +# test_hil_power_and_sensors.py +import pytest +import time + +def test_firmware_flash_and_boot(flasher, power_supply): + """Flashes target binary and verifies clean system boot.""" + # Flash firmware image + flasher.flash_file("build/firmware.hex", 0x08000000) + flasher.reset(halt=False) + time.sleep(0.5) + + # Verify current draw during boot is within expected operating limits + current_ma = float(power_supply.query("MEAS:CURR?")) * 1000.0 + assert 5.0 <= current_ma <= 25.0, f"Abnormal boot current: {current_ma} mA" + +def test_quiescent_deep_sleep_current(power_supply): + """Verifies device achieves sub-5 uA deep sleep current.""" + # Allow device 10 seconds to complete initialization and enter PSM + time.sleep(10.0) + + # Measure average current over 5 samples + samples = [] + for _ in range(5): + reading = float(power_supply.query("MEAS:CURR?")) * 1000000.0 # Convert to uA + samples.append(reading) + time.sleep(0.2) + + avg_sleep_ua = sum(samples) / len(samples) + assert avg_sleep_ua <= 5.0, f"Deep sleep current {avg_sleep_ua:.2f} uA exceeds 5.0 uA limit!" +``` + +## 4. Continuous Integration Test Bench Scheduling + +Because physical test racks cannot be replicated instantaneously like cloud containers, test execution must be scheduled efficiently: + +```mermaid +flowchart TD + Git["Pull Request Committed"] --> Unit["1. Cloud CI: Host Unit Tests (Ceedling / Catch2)"] + Unit --> Sim["2. Cloud CI: Virtual Emulation (QEMU / Renode)"] + Sim --> Gate{"Passes Host & Simulation?"} + Gate -->|Yes| Queue["3. Dispatch to Self-Hosted HIL Runner Queue"] + Gate -->|No| Fail["Reject Pull Request (Block HIL Racks)"] + Queue --> Smoke["4. HIL Smoke Test (5 Mins: Flash, Boot, Sleep Check)"] + Smoke --> Pass["PR Approved for Merge"] +``` + +[[CAPTION:Figure]] Staged gating pipeline protecting physical HIL test racks from invalid builds. + +- **PR Smoke Suite (5 minutes)**: Verifies successful flash, clean boot, sensor read, and sub-5 $\mu\mathrm{A}$ sleep current. +- **Nightly Endurance Suite (4 to 8 hours)**: Executes 1,000 flash erase cycles, tests brownout voltage sweeps from 3.6 V down to 2.2 V, and runs continuous cellular socket reconnect stress tests. diff --git a/content/guides/hil-and-automated-testing/05-network-simulation-and-chaos-testing.md b/content/guides/hil-and-automated-testing/05-network-simulation-and-chaos-testing.md new file mode 100644 index 0000000..fbe348d --- /dev/null +++ b/content/guides/hil-and-automated-testing/05-network-simulation-and-chaos-testing.md @@ -0,0 +1,112 @@ +--- +title: "Network Simulation & Chaos Testing" +--- + +# Network Simulation & Chaos Testing + +Connected devices in the field face hostile physical and network conditions. Cellular towers drop connections without TCP FIN packets, and deep basements weaken radio signals to the noise floor. Ageing batteries produce large voltage dips during radio bursts. Testing firmware only under clean laboratory conditions guarantees catastrophic failures when deployed across thousands of customer sites. + +**Chaos Engineering for IoT** intentionally subjects devices to electrical transients, bus lockups, RF signal degradation, and transport-layer packet corruption on automated test benches. + +```mermaid +flowchart TD + subgraph FaultEngine["Hardware Chaos & Impairment Engine"] + NetChaos["1. Transport Impairment
(Latency, Jitter, Packet Drop via netem)"] + RFChaos["2. RF Signal Degradation
(Dynamic Attenuation Sweeps: 0-90 dB)"] + PowerChaos["3. Electrical Brownouts
(Pulsed Microsecond VCC Dips)"] + BusChaos["4. Physical Bus Faults
(Relay-Induced I2C/SPI Line Shorts)"] + end + FaultEngine --> DUT["Device Under Test (DUT Silicon)"] + DUT --> Assert{"Verification: Zero Crash / Zero Data Loss"} +``` + +[[CAPTION:Figure]] Multi-domain chaos testing architecture for connected hardware resilience. + +## 1. Network Impairment Simulation with Linux `netem` + +When devices communicate with cloud endpoints through a local test gateway or Wi-Fi access point, the host can inject simulated wide-area network degradation using the Linux **Network Emulator (`netem`)**: + +```bash +# Inject 250ms latency with 50ms jitter and 15% random packet loss on test bridge +sudo tc qdisc add dev eth1 root netem \ + delay 250ms 50ms distribution normal \ + loss 15% \ + corrupt 1% +``` + +### 1.1 Test Scenarios & Pass Criteria + +| Injected Network Fault | Simulated Real-World Event | Pass / Fail Acceptance Criteria | +| :--- | :--- | :--- | +| **50% Packet Loss** | Rain fade / marginal RF cell edge | Device backs off retransmissions cleanly; buffers unsent records in LittleFS. | +| **Silent Socket Severance**| Cellular tower drops NAT table without RST/FIN | TCP keepalive timeout fires; socket is cleanly closed and reopened without memory leak. | +| **3-Second Packet Latency**| Satellite NTN backhaul delay | Protocol timers do not trigger prematurely; transactions complete without duplicate processing. | + +[[CAPTION:Table]] Network impairment simulation test matrix. + +## 2. RF Fading & Coverage Enhancement (CE) Sweeps + +Using automated RF programmable step attenuators (such as Mini-Circuits USB attenuators), the HIL bench simulates a device moving through varying signal conditions or being installed deep underground: + +```python +def test_rf_attenuation_sweep_and_reconnection(attenuator, dut_console): + """Sweeps RF attenuation from 10 dB to 85 dB to verify link adaptation.""" + # Start with ideal signal + attenuator.set_attenuation(10.0) # 10 dB loss (RSRP ~ -75 dBm) + time.sleep(2.0) + assert dut_console.get_current_rsrp() > -85 + + # Ramp attenuation down to cell-edge limit + for loss_db in [30, 50, 70, 85]: + attenuator.set_attenuation(loss_db) + time.sleep(5.0) + rsrp = dut_console.get_current_rsrp() + print(f"Loss: {loss_db} dB -> Measured RSRP: {rsrp} dBm") + + # At 85 dB loss, device should adapt to CE Level 2 (repetition coding) + ce_level = dut_console.get_coverage_enhancement_level() + assert ce_level in [1, 2], f"Expected CE Level 1 or 2 under heavy attenuation, got {ce_level}" + + # Return signal and confirm telemetry delivers cleanly + attenuator.set_attenuation(10.0) + time.sleep(5.0) + assert dut_console.wait_for_telemetry_ack(timeout=30) +``` + +## 3. Electrical Brownout Fault Injection + +The most dangerous field condition occurs when a depleted battery encounters a high-power radio burst, causing supply voltage to collapse momentarily: + +```mermaid +sequenceDiagram + participant Bench as Automated HIL Bench + participant PSU as Programmable Power Supply + participant DUT as Embedded Device (NOR Flash) + + Note over Bench,DUT: Firmware begins writing 256-byte sector to LittleFS + DUT->>DUT: Assert SPI Flash Write Command + Bench->>PSU: Inject Microsecond Voltage Dip (3.3V -> 1.8V for 25 ms) + Note over DUT: Microcontroller Brownout Detector (BOR) triggers! + DUT->>DUT: Hardware Core Reset Held Low + Bench->>PSU: Restore Voltage to 3.3V + Note over DUT: System Reboots from Reset Vector + DUT->>DUT: Mount LittleFS Filesystem + Bench->>DUT: Verify LittleFS Mount Status & Checksum + Note over Bench: Filesystem mounted cleanly! Zero sector corruption. +``` + +[[CAPTION:Figure]] Automated brownout injection during active flash writes validating LittleFS copy-on-write resilience. + +### 3.1 Brownout Verification Rules + +1. **Zero Filesystem Corruption**: The filesystem must mount without errors upon reboot. Any torn writes must be rolled back to the last atomic commit. +1. **Zero Watchdog Lockup**: The microcontroller must not lock into an undefined high-current draw state when voltage hovers near the minimum logic threshold. +1. **Crash Reason Recorded in Flash**: Upon reboot, the boot routine must read the MCU reset flags (`RCC_CSR` or equivalent) and log that the reset was caused by a Brownout Reset (BOR) event rather than a clean power-on reset. + +## 4. Hardware Chaos Scorecard & Gating + +Before a firmware binary is signed for production fleet release, it must execute the complete automated chaos matrix: + +1. **100 Consecutive Power Brownout Injections**: Device must achieve 100% recovery with zero unrecoverable flash corruption. +1. **100 I2C Bus Lockup Events**: Hardware recovery routine must release the bus within 9 clock cycles without requiring a system reboot. +1. **24-Hour Continuous Network Chaos Run**: Device must maintain 100% telemetry accounting (all records buffered offline and delivered upon network restoration). diff --git a/content/guides/hil-and-automated-testing/index.md b/content/guides/hil-and-automated-testing/index.md new file mode 100644 index 0000000..6f8caed --- /dev/null +++ b/content/guides/hil-and-automated-testing/index.md @@ -0,0 +1,35 @@ +--- +title: Hardware-in-the-loop and automated testing +--- + +# Hardware-in-the-loop and automated testing + +## Executive Overview + +Developing firmware for physical connected products presents a unique verification challenge: software interacts continuously with physical analog circuits, silicon errata, fluctuating power supplies, and lossy wireless channels. Testing firmware solely on developer benches or relying on manual button-pressing before release leads to severe defect escapes, intermittent field crashes, and expensive recalls. + +This guide sets out how to build an automated, multi-tier testing strategy for connected devices. It covers host-native unit testing, virtual system emulation, physical Hardware-in-the-Loop (HIL) test racks and network chaos engineering. + +```mermaid +flowchart TD + subgraph TestingGuide["Automated Testing & HIL Guide"] + CH1["Ch 1: The IoT Testing Pyramid"] + CH2["Ch 2: Host-Based Firmware Testing"] + CH3["Ch 3: QEMU & Virtual Hardware Simulation"] + CH4["Ch 4: HIL Bench Design & Automation"] + CH5["Ch 5: Network Simulation & Chaos Testing"] + end + TestingGuide --> RobustHardware["Zero-Escape Hardware Deployments"] +``` + +[[CAPTION:Figure]] The five pillars of automated embedded testing and HIL validation. + +## Target Audience & Guide Structure + +This guide is designed for embedded firmware engineers, test automation engineers, hardware designers, and quality assurance architects: + +- **[Chapter 1: The IoT Testing Pyramid](01-the-iot-testing-pyramid.md)**: Four-layer verification model, defect escape economics, and test execution velocity. +- **[Chapter 2: Host-Based Firmware Testing](02-host-based-firmware-testing.md)**: Hardware Abstraction Layer (HAL) mocking, Ceedling, Unity, CMock, and AddressSanitizer execution in cloud CI. +- **[Chapter 3: QEMU & Virtual Hardware Simulation](03-qemu-virtual-hardware-simulation.md)**: Simulating multi-task RTOS applications, virtual peripheral buses, and network interfaces using QEMU and Renode. +- **[Chapter 4: HIL Bench Design & Automation](04-hil-bench-design-and-automation.md)**: Pogo-pin fixtures, SCPI instrument automation, PyVISA controllers, dynamic current profiling, and pytest orchestration. +- **[Chapter 5: Network Simulation & Chaos Testing](05-network-simulation-and-chaos-testing.md)**: Injecting brownouts, RF fading, cellular packet loss, bus lockups, and chaos engineering scenarios. diff --git a/content/guides/index.md b/content/guides/index.md index 398189f..46bdf15 100644 --- a/content/guides/index.md +++ b/content/guides/index.md @@ -5,6 +5,48 @@ author: Mark Bacon # Guides -Long-form guides to IoT endpoint and gateway architecture, fleet operations, cyber security, testing and scaling. +Long-form guides that go into depth on one subject. The [Guidelines](../guidelines/index.md) set the rules, the [Standards](../standards/index.md) set the technical requirements, and the [Primers](../primers/index.md) are short introductions. The guides explain how to do the work well. -This section is moving in from technical-documentation's guides. +## Architecture and design + +[[CAPTION:Table]] Guides to architecture and design. + +| Guide | Covers | +| --- | --- | +| [C4 modelling for IoT and connected systems](c4-model/index.md) | The C4 levels applied to devices and cloud, diagrams as code, and a Structurizr DSL reference | +| [Scaling the practice of architecture](scaling-architecture/index.md) | The advice process and the alignment mechanisms that let autonomous teams make architecture decisions | +| [Writing ADRs and RFCs](decision-records.md) | When to write each, and how to write them well | +| [Writing a high-level design](high-level-design.md) | Capacity estimation and the structure of a high-level design document | +| [Writing a software design specification](software-design-specification.md) | A software design specification kept as docs as code | +| [Running an architecture review board](architecture-review-board.md) | What goes to the board, who sits on it, and how it decides | + +## Connected devices + +[[CAPTION:Table]] Guides to connected devices. + +| Guide | Covers | +| --- | --- | +| [IoT endpoint and edge gateway architecture](iot-endpoint-and-gateway-architecture/index.md) | Device tiers, connectivity, power, edge storage and edge intelligence | +| [IoT fleet operations, telemetry and digital twins](iot-fleet-operations/index.md) | Onboarding, telemetry ingestion, digital twins, OTA campaigns and fleet observability | +| [Cyber security and threat modelling for connected systems](cybersecurity/index.md) | Threat modelling, hardware security, transport security, secure updates and compliance | +| [Hardware-in-the-loop and automated testing](hil-and-automated-testing/index.md) | The testing pyramid, host-based tests, virtual hardware, HIL benches and network chaos testing | + +## Engineering practice + +[[CAPTION:Table]] Guides to engineering practice. + +| Guide | Covers | +| --- | --- | +| [Git](git/index.md) | How Git works, branching strategies and choosing a version control system | +| [Reviewing code](code-review.md) | How to read a change, give feedback and keep reviews useful | +| [Agile requirements and user stories](agile-requirements-and-user-stories.md) | User stories, acceptance criteria and how they trace to requirements | +| [Defect management and debugging](defect-management-and-debugging.md) | Raising and classifying defects, systematic debugging and root cause analysis | +| [Ways of working](ways-of-working/index.md) | Working agreements, onboarding, the development process, CI/CD, documentation and metrics | + +## Deep dives + +[[CAPTION:Table]] The deep dives. + +| Guide | Covers | +| --- | --- | +| [Deep dives](deep-dives/index.md) | Long explanations of CBOR, CoAP, LwM2M, NB-IoT, certificates and keys, RTOS internals, SysVars, MISRA C, static analysis, Git internals, and docs, diagrams and architecture as code | diff --git a/content/guides/iot-endpoint-and-gateway-architecture/01-device-tiers-and-hardware-architecture.md b/content/guides/iot-endpoint-and-gateway-architecture/01-device-tiers-and-hardware-architecture.md new file mode 100644 index 0000000..5153c3c --- /dev/null +++ b/content/guides/iot-endpoint-and-gateway-architecture/01-device-tiers-and-hardware-architecture.md @@ -0,0 +1,101 @@ +--- +title: "Device Tiers & Hardware Architecture" +--- + +# Device Tiers & Hardware Architecture + +Selecting the appropriate silicon architecture, memory hierarchy, and peripheral interconnects forms the foundation of any connected device engineering programme. Inadequate compute capacity prevents future security updates, while over-specifying hardware inflates bill-of-materials (BOM) cost and accelerates battery drain. + +This chapter establishes a systematic hardware categorization framework, contrasts modern silicon execution engines, details memory partitioning strategies, and provides architectural rules for robust peripheral bus integration. + +## 1. Connected Device Classification Framework + +Hardware architectures are categorized across four distinct performance and resource tiers, extending the IETF RFC 7228 classification to encompass modern edge gateways: + +| Tier | Typical Hardware | Volatile Memory (RAM) | Non-Volatile (Flash) | Operating Environment | Primary Functionality | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **Class 0 (Sub-Constrained)** | 8-bit / 16-bit MCU, Cortex-M0+ | < 10 KB | < 100 KB | Bare-metal super-loop | Simple beacon, water leak detector, push button | +| **Class 1 (Constrained Node)** | ARM Cortex-M0+/M3, RISC-V | ~10 KB to 32 KB | ~100 KB to 256 KB | Ultra-light RTOS (FreeRTOS) | Battery environmental sensor, BLE tracker | +| **Class 2 (Connected Endpoint)**| ARM Cortex-M4F/M33, ESP32 | ~64 KB to 512 KB | ~512 KB to 4 MB | Full RTOS (Zephyr / FreeRTOS) | Cellular LPWAN sensor, asset tracker, smart valve | +| **Tier 3 (Edge Gateway)** | ARM Cortex-A53 / A7, NXP `i.MX` | 512 MB to 4 GB | 4 GB to 32 GB eMMC | Embedded Linux / RT-Linux | Protocol translation, local CV/ML, container host | + +[[CAPTION:Table]] Taxonomy of connected device hardware tiers and capabilities. + +```mermaid +flowchart TD + subgraph Tiers["Connected Hardware Tiers"] + C0["Class 0: Bare-Metal Sensor Nodes (< 10 KB RAM)"] + C1["Class 1: Constrained Microcontrollers (~32 KB RAM)"] + C2["Class 2: Security-Hardened Endpoints (256-512 KB RAM)"] + T3["Tier 3: Multi-Core Edge Gateways (1-4 GB RAM)"] + end + C0 -.->|Sub-GHz / BLE| C2 + C1 -->|BLE / Zigbee / Thread| T3 + C2 -->|Cellular LPWAN / LoRaWAN| Cloud["Cloud Ingestion Platform"] + T3 -->|Ethernet / 4G LTE Cat-4| Cloud +``` + +[[CAPTION:Figure]] Interconnection topology across device tiers from ultra-constrained nodes to edge gateways. + +## 2. Silicon Architecture Evaluation + +1. **ARM Cortex-M Series (M0+, M4F, M33)**: The de facto standard for low-power connected microcontrollers. The ARM Cortex-M33 incorporates **ARM TrustZone-M**, establishing hardware-enforced isolation between Secure and Non-Secure processing environments on a single core. This allows cryptographic keys and root-of-trust operations to execute in an isolated enclave shielded from application bugs. +1. **RISC-V Architectures (RV32IMAC)**: Gaining widespread traction for vendor neutrality, low royalty costs, and custom hardware accelerators. Microcontrollers such as the ESP32-C3/C6 provide robust silicon options for wireless nodes combining Wi-Fi 6, BLE 5, and IEEE 802.15.4 (Thread/Zigbee). +1. **ARM Cortex-A Multi-Core Application Processors**: Essential for Tier 3 Edge Gateways requiring full Linux operating systems, networking stacks, memory management units (MMU), and hardware cryptographic engines (ARM CryptoCell / CAAM). + +## 3. Memory Subsystem & Flash Partitioning + +Robust connected endpoints require explicit non-volatile memory layout partitioning to support atomic, fail-safe Firmware Over-The-Air (FOTA) updates and wear-leveled application storage. + +```mermaid +flowchart TD + subgraph FlashLayout["Primary SPI NOR Flash Memory Map (4 MB)"] + BL["0x0000_0000: Secure Bootloader (64 KB, Write-Protected)"] + NVS1["0x0001_0000: Non-Volatile Config / Keys (64 KB)"] + SlotA["0x0002_0000: Active Firmware Bank A (1792 KB)"] + SlotB["0x001E_0000: Candidate Firmware Bank B (1792 KB)"] + LFS["0x003A_0000: LittleFS Spool Storage (384 KB)"] + end +``` + +[[CAPTION:Figure]] Dual-bank flash partitioning layout for atomic A/B firmware updates. + +### 3.1 Dual-Bank A/B Partitioning Rules + +1. **Symmetric Bank Sizing**: Active Bank A and Candidate Bank B must possess identical capacity, each sufficient to store the maximum allowable compressed firmware binary. +1. **Zero Overwrite During Ingestion**: Incoming FOTA chunks must be streamed and verified directly into the inactive candidate bank (Bank B) without touching active execution memory (Bank A). +1. **Atomic Swap Pointer**: Switching from Bank A to Bank B must be executed through an atomic flag update in write-protected Non-Volatile Storage (NVS). If the candidate binary crashes during its trial run, the hardware watchdog forces a reset, and the bootloader automatically reverts to Bank A. + +## 4. Hardware Bus Topologies & Signal Integrity + +Microcontrollers coordinate local sensors, actuators, and communication transceivers through serial peripheral buses. Field reliability demands defensive electrical and firmware design: + +```mermaid +flowchart LR + MCU["Microcontroller (Master)"] + MCU -->|I2C: 100/400 kHz
Pull-Ups Required| I2C_Dev["Temp/Humidity Sensor"] + MCU -->|SPI: Up to 50 MHz
Dedicated CS Lines| Flash["External NOR Flash"] + MCU -->|UART: DMA Ring Buffer
RTS/CTS Flow Control| Radio["Cellular LPWAN Modem"] + MCU -->|Isolated RS-485 / CAN| Bus["Industrial Equipment"] +``` + +[[CAPTION:Figure]] Peripheral bus interfaces linking microcontroller to transceivers and transducers. + +### 4.1 I2C Bus Reliability and Bus-Lockup Clearance + +I2C open-drain lines (SDA and SCL) are susceptible to bus lockups if a slave device experiences a power dip while holding the SDA line low. If SDA remains stuck low, the master cannot generate a START or STOP condition: + +> [!IMPORTANT] +> Firmware must implement a hardware I2C recovery routine. Prior to initializing the hardware I2C peripheral, the MCU GPIO driver must inspect the SDA pin. If SDA is low, the master must manually toggle the SCL pin up to 9 consecutive clock cycles until the slave releases the SDA line, followed by asserting a valid STOP condition. + +### 4.2 UART Interconnects & DMA Ring Buffers + +Communication between the application microcontroller and external cellular or satellite modems must use UART with **Hardware Flow Control (RTS/CTS)**: +- High-speed cellular modems can burst several kilobytes of data when receiving network packets. Without hardware flow control, receiver buffer overruns cause silent packet loss. +- Firmware must use **Direct Memory Access (DMA)** configured as a circular ring buffer to offload UART reception from the CPU, preventing dropped characters during high interrupt loads. + +### 4.3 Industrial Fieldbuses: RS-485 and CAN + +For Tier 3 gateways and industrial nodes interfacing with third-party factory machinery: +- **RS-485 (Modbus RTU)**: Requires galvanic isolation (optocouplers or digital isolators) and differential termination resistors ($120\,\Omega$) to reject high-voltage ground loops and industrial electrical transients. +- **CAN-FD (Controller Area Network Flexible Data-Rate)**: Provides deterministic arbitration, hardware cyclic redundancy checks (CRC), and automatic retransmission for critical machinery control. diff --git a/content/guides/iot-endpoint-and-gateway-architecture/02-connectivity-and-protocol-tradeoffs.md b/content/guides/iot-endpoint-and-gateway-architecture/02-connectivity-and-protocol-tradeoffs.md new file mode 100644 index 0000000..9196e0a --- /dev/null +++ b/content/guides/iot-endpoint-and-gateway-architecture/02-connectivity-and-protocol-tradeoffs.md @@ -0,0 +1,110 @@ +--- +title: "Connectivity & Protocol Trade-Offs" +--- + +# Connectivity & Protocol Trade-Offs + +Connected endpoints and edge gateways rely on wireless and wired communication protocols to exchange telemetry, receive configuration updates, and report critical alarms. Selecting the optimal combination of physical layer (PHY), medium access control (MAC), transport, and application layer protocols dictates network scalability, battery longevity, and operational data expenditure. + +This chapter analyzes modern wireless and wired connectivity options, contrasts IoT application layer protocols, and provides a structured architectural selection matrix. + +## 1. Physical & Link Layer Connectivity Landscape + +IoT connectivity spans short-range local mesh networks to global Non-Terrestrial Network (NTN) satellite links: + +```mermaid +flowchart TD + subgraph Range["Wireless Coverage Hierarchy"] + PAN["Personal Area / Mesh (1 m - 100 m)
BLE 5.x Coded PHY, Thread, Wi-Fi 6 TWT"] + LPWAN["Terrestrial LPWAN (1 km - 15 km)
NB-IoT, LTE-M, LoRaWAN Class A/C"] + NTN["Non-Terrestrial / Satellite (> 500 km)
3GPP Rel 17 Satellite NTN, Iridium SBD"] + end + PAN --> Gateway["Edge Gateway"] + Gateway --> LPWAN + LPWAN --> Cloud["Enterprise Cloud"] + NTN --> Cloud +``` + +[[CAPTION:Figure]] Hierarchical wireless communication topology from mesh to satellite. + +### 1.1 Short-Range & Local Mesh Networks + +1. **Bluetooth Low Energy (BLE 5.x)**: BLE 5 introduces **Coded PHY** ($S=8$ or $S=2$), extending radio range up to 400 meters in line-of-sight while trading throughput down to 125 kbps. Features such as **Periodic Advertising** enable ultra-low-power local telemetry broadcasting without establishing point-to-point connections. +1. **Thread (IEEE 802.15.4 & 6LoWPAN)**: An open IPv6-native wireless mesh standard. Unlike Zigbee, Thread devices are addressable via standard IPv6 addresses, avoiding application-layer translation gateways. Thread incorporates self-healing mesh routing with no single point of failure. +1. **Wi-Fi 6 (IEEE 802.11ax)**: Introduces **Target Wake Time (TWT)**, allowing client devices to negotiate specific sleep schedules with the access point. A Wi-Fi 6 sensor can remain asleep for hours or days, waking only at pre-negotiated microsecond slots, slashing standby power consumption. + +### 1.2 Long-Range Low-Power Wide Area Networks (LPWAN) + +| Technology | Spectrum | Max Range | Data Rate | Roaming | Typical Applications | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **NB-IoT (Cat-NB1/NB2)** | Licensed Cellular (3GPP) | Up to 15 km | ~26 kbps to 127 kbps | Regional / Limited | Stationary sensors, basement meters, pipeline monitors | +| **LTE-M (Cat-M1/M2)** | Licensed Cellular (3GPP) | Up to 10 km | ~375 kbps to 1 Mbps | Global / Extensive | Fleet trackers, mobile assets, large FOTA nodes | +| **LoRaWAN (Class A/C)** | Unlicensed ISM (868/915 MHz) | Up to 15 km | 250 bps to 50 kbps | Private Gateways | Remote agriculture, campus sensors, smart campuses | +| **Satellite NTN** | Licensed L/S/Ku Bands (3GPP Rel 17)| Global Orbit | 100 bps to 10 kbps | Global Maritime/Wild | Ocean buoys, transcontinental shipping, mining | + +[[CAPTION:Table]] Technical comparison of wide-area wireless connectivity options. + +## 2. Application Layer Transport Protocols + +At the application layer, the three predominant protocols for connected devices are **CoAP**, **MQTT**, and **gRPC / HTTP**: + +```mermaid +flowchart LR + subgraph Constrained["Resource-Constrained Endpoints"] + CoAP["CoAP / UDP + DTLS
- 4-Byte Fixed Header
- Asynchronous Observe
- Binary Native"] + end + subgraph Standard["Standard IoT Endpoints"] + MQTT["MQTT / TCP + TLS
- Publish/Subscribe
- Broker Architecture
- Persistent Sessions"] + end + subgraph Gateway["Edge Gateways & Cloud"] + gRPC["gRPC / HTTP/2 + TLS
- High Throughput
- Protobuf Streaming
- Strong Typing"] + end +``` + +[[CAPTION:Figure]] Protocol alignment matching compute constraints and network topologies. + +### 2.1 Constrained Application Protocol (CoAP - RFC 7252) + +CoAP is a specialized RESTful web transfer protocol designed for constrained nodes and lossy networks: +- **UDP Transport**: Operates over UDP with optional Datagram Transport Layer Security (DTLS), avoiding TCP three-way connection handshakes and keepalive packet overhead. +- **Microscopic Header Overhead**: The base CoAP header is fixed at only **4 bytes**, compared to 20 to 60 bytes for TCP/HTTP headers. +- **Observe Pattern (RFC 7641)**: Allows clients to observe resources on an endpoint; the server asynchronously streams state updates whenever the resource state mutates, eliminating continuous client polling. +- **Block-Wise Transfers (RFC 7959)**: Enables chunked transmission of large payloads (such as firmware images) across small network MTUs without relying on IP fragmentation. + +### 2.2 Message Queuing Telemetry Transport (MQTT v3.1.1 / v5.0) + +MQTT is an asynchronous publish/subscribe messaging protocol operating over TCP: +- **Decoupled Architecture**: Publishers and subscribers do not interact directly; all communication routes through an intermediate MQTT broker. +- **Quality of Service (QoS)**: + - *QoS 0 (At most once)*: Fire-and-forget transmission with zero delivery confirmation. + - *QoS 1 (At least once)*: Guarantees delivery via PUBACK acknowledgment; messages may be duplicated. + - *QoS 2 (Exactly once)*: Guarantees single-delivery via a four-step handshake; introduces significant latency and round-trips. +- **MQTT v5 Enhancements**: Introduces **User Properties** (arbitrary key-value metadata), **Topic Aliasing** (reducing string topic names to 2-byte integer IDs on the wire), and **Reason Codes** for granular diagnostic reporting. + +### 2.3 gRPC and HTTP/2 for Tier 3 Gateways + +For Linux-based edge gateways streaming aggregated telemetry from dozens of local machines to cloud event hubs: +- **Multiplexed Streams**: HTTP/2 multiplexes multiple bidirectional RPC streams over a single persistent TCP connection. +- **Strict Interface Definition**: gRPC compiles strongly typed Protocol Buffer contracts, preventing schema mismatches between edge gateways and cloud microservices. + +## 3. Protocol Selection Decision Matrix + +Engineering teams must apply the following decision framework: + +```mermaid +flowchart TD + Start["Select IoT Protocol"] --> Q1{"Is device Class 0 or 1 on battery with LPWAN?"} + Q1 -->|Yes| CoAP["Adopt CoAP over UDP/DTLS with CBOR"] + Q1 -->|No| Q2{"Is device Class 2 with continuous IP connectivity?"} + Q2 -->|Yes| MQTT["Adopt MQTT v5 over TLS with Protobuf or CBOR"] + Q2 -->|No| Q3{"Is device a Linux Gateway aggregating local traffic?"} + Q3 -->|Yes| gRPC["Adopt gRPC / HTTP/2 or Kafka Edge Producer"] + Q3 -->|No| MQTT +``` + +[[CAPTION:Figure]] Decision tree for connected device protocol selection. + +### 3.1 Rules of Thumb for Constrained Architectures + +1. **Never Run TLS Handshakes on Hourly Bursts**: An unoptimized TLS 1.2 or 1.3 handshake transfers 4 KB to 8 KB of certificate chains and cryptographic parameters. If an LPWAN device wakes once per hour, conducts a full TLS handshake, transmits a 50-byte telemetry payload, and sleeps, **over 95% of its battery energy is wasted on cryptographic handshakes**. Use **Session Resumption with Pre-Shared Keys (TLS-PSK)**, **0-RTT connection tickets**, or CoAP/OSCORE (RFC 8613) to eliminate repetitive handshake overhead. +1. **Cap Keepalive Intervals Responsibly**: On cellular networks, carrier firewalls tear down idle TCP NAT state tables after 60 to 120 seconds. If using MQTT over cellular, set keepalive ping intervals to 50 seconds to prevent silent socket severance, or transition to CoAP/UDP where NAT bindings can be refreshed with minimal packet cost. diff --git a/content/guides/iot-endpoint-and-gateway-architecture/03-power-architecture-and-duty-cycling.md b/content/guides/iot-endpoint-and-gateway-architecture/03-power-architecture-and-duty-cycling.md new file mode 100644 index 0000000..2fb5031 --- /dev/null +++ b/content/guides/iot-endpoint-and-gateway-architecture/03-power-architecture-and-duty-cycling.md @@ -0,0 +1,128 @@ +--- +title: "Power Architecture & Duty-Cycling" +--- + +# Power Architecture & Duty-Cycling + +Designing power architectures for connected devices requires balancing peak pulse requirements, sleep quiescent leakage, supply voltage ranges, and thermal dissipation. Whether an endpoint relies on a decade-long primary lithium cell, ambient solar energy harvesting, or Power-over-Ethernet (PoE), power architecture dictates system reliability and field operating life. + +This chapter details power supply topologies, contrasts DC-DC conversion architectures, provides guidelines for firmware duty-cycling, and outlines hardware brownout mitigation mechanisms. + +## 1. Power Supply Topologies + +Connected systems utilise four primary power intake topologies: + +```mermaid +flowchart TD + subgraph Sources["Primary Power Sources"] + Batt["Primary Chemical Battery
(Li-SOCl2 / Li-MnO2)"] + Harvest["Energy Harvesting Hybrid
(Solar PV + LiFePO4 / Supercap)"] + PoE["Power over Ethernet (PoE)
(IEEE 802.3af / 802.3at)"] + Mains["Mains Power AC/DC
(85-265 VAC Isolated SMPS)"] + end + Sources --> PowerMgmt["Power Management & Regulation"] + PowerMgmt --> Rails["System Rails (3.3V, 1.8V, 1.2V)"] +``` + +[[CAPTION:Figure]] Common power intake architectures across endpoint and gateway designs. + +| Topology | Input Range | Primary Advantages | Critical Design Challenges | Best Use Cases | +| :--- | :--- | :--- | :--- | :--- | +| **Primary Lithium** | 2.0 V to 3.6 V | Decade-long life without external wiring | Severe peak pulse voltage drop, passivation | Remote telemetry, pipeline monitors | +| **Solar Harvesting** | 0.5 V to 6.0 V (PV) | Indefinite operational lifetime | Multi-day cloudy periods, winter freeze degradation | Outdoor weather stations, agricultural IoT | +| **PoE (802.3af/at)** | 37 V to 57 V DC | Single cable for Gigabit data and 15W-30W power | Requires isolated flyback transformer, magnetics | Building automation, security cameras, gateways | +| **Mains AC/DC** | 85 V to 264 V AC | Unlimited continuous electrical energy | High voltage isolation (4 kV surge), safety certs | Industrial gateways, street light controllers | + +[[CAPTION:Table]] Comparison of power intake architectures. + +## 2. Voltage Regulation: LDO vs DC-DC Switching Converters + +Microcontrollers and RF radios require stable voltage rails (typically 3.3 V, 1.8 V, or 1.2 V core). Converting battery or bus voltage down to these rails requires evaluating the trade-offs between Low-Dropout Regulators (LDOs) and Switched-Mode DC-DC Buck Converters: + +```mermaid +flowchart LR + subgraph LDO["Low-Dropout Regulator (LDO)"] + LDO_In["Input Voltage (Vin)"] --> LDO_Pass["Pass Transistor
(Dissipates Heat)"] + LDO_Pass --> LDO_Out["Output (Vout)"] + end + subgraph DCDC["Switched-Mode Buck Converter"] + DC_In["Input Voltage (Vin)"] --> DC_Sw["PWM Switch"] + DC_Sw --> DC_Ind["Inductor & Diode"] + DC_Ind --> DC_Out["Output (Vout)"] + end +``` + +[[CAPTION:Figure]] Linear LDO versus switched-mode inductive DC-DC conversion. + +### 2.1 Efficiency Comparison and the Quiescent Current Trap + +Linear regulators exhibit electrical efficiency proportional to the voltage ratio: + +$$\eta_{LDO} = \frac{V_{out}}{V_{in}} \times 100\%$$ + +When regulating a $3.6\,\mathrm{V}$ primary cell down to $1.8\,\mathrm{V}$ for an ultra-low-power MCU core, an LDO operates at only $50\%$ efficiency, dissipating the remaining $50\%$ as waste heat. A synchronous DC-DC buck converter delivers $90\%$ to $95\%$ efficiency under load. + +> [!WARNING] +> While DC-DC buck converters excel at active currents ($> 10\,\mathrm{mA}$), their internal oscillator and switching circuits consume quiescent current ($I_Q$) ranging from $10\,\mu\mathrm{A}$ to $50\,\mu\mathrm{A}$. In an ultra-low-power sensor that sleeps $99.9\%$ of the time, this continuous $I_Q$ draw will drain the battery faster than the radio transmissions! + +**Architectural Recommendation**: Use **Dual-Mode Power Management ICs (PMICs)** or select ultra-low-quiescent buck converters (such as Texas Instruments TPS62840 with $I_Q = 60\,\mathrm{nA}$) that automatically transition to high-efficiency pulse-frequency modulation (PFM) during sleep states. + +## 3. Firmware Duty-Cycling & Low-Power State Execution + +Extending battery lifetime requires the firmware to treat every milliampere-second of active compute as a scarce resource. + +```mermaid +sequenceDiagram + participant RTC as Real-Time Clock + participant MCU as MCU Core & SRAM + participant Sen as Sensor Rail + participant Radio as Cellular / RF Modem + + Note over RTC,Radio: Deep Sleep: Standby Mode (3.2 uA) + RTC->>MCU: Hardware Wakeup Alarm (Every 15 minutes) + Note over MCU: Core Boots from Sleep (8 mA, 12 ms) + MCU->>Sen: Energize P-MOSFET Load Switch + MCU->>Sen: Read Sensor via I2C DMA (3 mA, 20 ms) + MCU->>Sen: De-energize P-MOSFET Load Switch + + alt Threshold Exceeded or Daily Sync Due + MCU->>Radio: Assert Modem Power Key & Wake Line + Note over Radio: Modem Attaches & Transmits (220 mA, 3.8 s) + Radio-->>MCU: Network PUBACK Received + MCU->>Radio: Issue AT+CPSMS / Power Down + else Standard Cycle (No Transmission) + Note over MCU: Telemetry appended to LittleFS flash buffer + end + MCU->>MCU: Enter Deep Sleep / Stop Mode +``` + +[[CAPTION:Figure]] Discrete event timeline for deterministic low-power sensor duty-cycling. + +### 3.1 Firmware Power Optimization Rules + +1. **Tickless RTOS Scheduling**: Standard RTOS kernels maintain a periodic 1 ms tick interrupt that wakes the processor 1,000 times per second. Firmware must enable tickless idle (for example, `configUSE_TICKLESS_IDLE` in FreeRTOS), which programs hardware low-power timers (LPTIM) to sleep for seconds or hours until the next actual software timer expires. +1. **Peripheral Power Rail Gating**: Idle sensors, level shifters, and external memory chips frequently leak $5\,\mu\mathrm{A}$ to $50\,\mu\mathrm{A}$ in standby. Route all peripheral power supplies through a high-side P-channel MOSFET load switch controlled by an MCU GPIO, keeping external circuits completely de-energized during sleep. +1. **Dynamic Voltage and Frequency Scaling (DVFS)**: When executing sensor math or cryptographic hashing, run the core clock at maximum frequency (for example, 80 MHz) to complete computations as fast as possible ("race-to-sleep"). Once computation finishes, immediately drop the clock down to 32 kHz or enter deep sleep. + +## 4. Hardware Reliability & Brownout Mitigation + +When primary batteries degrade or experience cold temperatures, high-current RF transmission bursts induce sharp voltage droops. If the system rail drops below the digital logic threshold, the microcontroller enters a chaotic undefined state. + +```mermaid +flowchart TD + Droop["Battery Voltage Droop (RF Transmission)"] --> Monitor{"Supply Voltage Monitor"} + Monitor -->|VCC > 2.8V| Normal["Normal Operation"] + Monitor -->|2.4V < VCC < 2.8V| PVD["Programmable Voltage Detector (PVD) Interrupt"] + PVD --> Abort["Firmware Gracefully Aborts Radio & Saves State"] + Monitor -->|VCC < 2.4V| BOR["Hardware Brownout Reset (BOR)"] + BOR --> SafeReset["Deterministic MCU Core Reset Held Low"] +``` + +[[CAPTION:Figure]] Multi-tier voltage supervision mitigating catastrophic brownouts. + +### 4.1 Independent Watchdog (IWDG) Architecture + +Firmware can hang due to electrostatic discharge (ESD), infinite loops, or locked external buses. A dedicated **Independent Watchdog Timer (IWDG)** is mandatory: +- The IWDG must be clocked from an independent dedicated internal low-speed RC oscillator (32 kHz), completely decoupled from the main system PLL and crystal oscillators. +- The watchdog timeout must be configured to reset the system if the main loop fails to refresh the watchdog within a bounded window (typically 4 to 8 seconds). +- The watchdog service function must only be called from the primary operational task after verifying that all sub-tasks and IPC queues are healthy. Never refresh a watchdog unconditionally inside a hardware timer interrupt service routine (ISR). diff --git a/content/guides/iot-endpoint-and-gateway-architecture/04-edge-storage-and-local-resilience.md b/content/guides/iot-endpoint-and-gateway-architecture/04-edge-storage-and-local-resilience.md new file mode 100644 index 0000000..659a50e --- /dev/null +++ b/content/guides/iot-endpoint-and-gateway-architecture/04-edge-storage-and-local-resilience.md @@ -0,0 +1,130 @@ +--- +title: "Edge Storage & Local Resilience" +--- + +# Edge Storage & Local Resilience + +Connected devices deployed across harsh field environments encounter unexpected power interruptions, dead battery cutoffs, network brownouts, and physical shocks. Microcontrollers cannot assume clean shutdowns. A device may lose power while committing a database record, writing a telemetry batch or applying a firmware update. Its non-volatile storage must then recover deterministically on reboot, without human intervention. + +This chapter details embedded non-volatile memory architectures, copy-on-write storage engines, offline spooling priority queues, and crash dump persistence. + +## 1. Storage Media Selection & Failure Modes + +Understanding physical storage failure modes is vital to preventing bricked devices in the field: + +```mermaid +flowchart TD + subgraph StorageTypes["Embedded Storage Media"] + NOR["SPI NOR Flash
- Sector Erase: 4 KB / 64 KB
- Page Write: 256 Bytes
- Endurance: 100k cycles
- Zero Bad Blocks"] + NAND["SPI NAND Flash
- Block Erase: 128 KB
- Page Write: 2 KB
- Endurance: 50k cycles
- Bad Block Management Req."] + FRAM["FRAM / EEPROM
- Byte Addressable
- Zero Erase Latency
- Endurance: 10^14 cycles
- High Cost / Small Size"] + end +``` + +[[CAPTION:Figure]] Comparison of non-volatile storage media physical constraints. + +### 1.1 Physical Storage Hazards + +1. **Incomplete Sector Erase**: Erasing a 4 KB NOR flash sector takes 50 ms to 200 ms. If power collapses mid-erase, the sector contains invalid floating charge levels that fail future reads and writes. +1. **Bit Rot & Charge Leakage**: Flash cells store electrons in isolated floating gates. Over years of thermal cycling, charge leaks, turning a programmed `0` bit back into an unprogrammed `1`. High-integrity filesystems must employ CRC32 or ECC (Error Correcting Code) checks on every block. +1. **Write Disturb**: Programming data into one flash page can inadvertently stress adjacent unprogrammed cells in the same block, inducing silent bit flips. + +## 2. Power-Fail-Safe Filesystem: LittleFS + +Standard desktop filesystems (FAT, ext4) are unsuitable for raw microcontroller flash because they cache file allocation tables in RAM and update metadata in place. A power cut during a metadata update renders the entire partition unmountable. + +**LittleFS** is the mandated open-source filesystem for resource-constrained embedded systems: + +```mermaid +flowchart LR + subgraph CoW["Copy-on-Write Atomic Update"] + OldBlock["Original Block A
(Contains Version 1)"] + NewBlock["Target Block B
(Writes Version 2)"] + Ptr["Root Pointer Table"] + end + OldBlock -.->|1. Modify Data| NewBlock + NewBlock -->|2. Verify Integrity| Ptr + Ptr -->|3. Atomic Pointer Flip| NewBlock +``` + +[[CAPTION:Figure]] LittleFS copy-on-write transaction ensuring atomic state commits. + +### 2.1 Core Architectural Guarantees of LittleFS + +1. **Power-Loss Atomicity**: LittleFS uses a copy-on-write log-structured architecture. File modifications are written to freshly erased blocks. Once the new data is fully written and verified, the master metadata pointer is flipped atomically. If power fails at any point prior to the pointer flip, the filesystem cleanly mounts the preceding consistent version. +1. **Dynamic & Static Wear Leveling**: LittleFS tracks cumulative erase counts for every physical block. If a block's erase count significantly exceeds neighboring blocks, LittleFS migrates cold, static data to the worn block and reallocates the fresh block for active writes, ensuring uniform degradation across the entire flash chip. +1. **Zero Dynamic Allocation**: LittleFS operates using bounded static memory buffers, requiring less than 4 KB of RAM regardless of flash partition size. + +## 3. Disconnected Spooling & Priority Queues + +When cellular, satellite, or mesh connectivity is disrupted, endpoints must spool inbound events into local non-volatile queues: + +```mermaid +flowchart TD + Event["Sensor / System Event"] --> Filter{"Severity Classification"} + Filter -->|Critical: Tamper / Fault / BOR| Q0["High Priority Queue (Guaranteed Delivery)"] + Filter -->|Routine: Periodic Telemetry| Q1["Standard FIFO Queue (Spool Buffer)"] + Filter -->|Trace: Verbose Debug Logs| Q2["Ephemeral Ring Buffer (Drop on Overflow)"] + + Q0 --> LittleFS["LittleFS Non-Volatile Storage"] + Q1 --> LittleFS + Q2 --> SRAM["SRAM Circular Buffer"] + + LittleFS --> Dispatcher["Network Cloud Dispatcher"] +``` + +[[CAPTION:Figure]] Prioritized spooling pipeline preventing critical alarm loss. + +### 3.1 Backpressure and Queue Eviction Rules + +When a network outage persists for weeks, local flash capacity will eventually be exhausted: + +1. **Drop Ephemeral Traces First**: Diagnostic debug logs in Queue Q2 are discarded immediately. +1. **Telemetry Downsampling**: When Queue Q1 exceeds 80% capacity, an automated background task consolidates 1-minute telemetry records into hourly aggregates (recording minimum, maximum, mean, and sample count), freeing up to 90% of occupied flash sectors. +1. **Strict FIFO Tail Protection**: If downsampled storage still reaches 98% capacity, the oldest unacknowledged routine records in Queue Q1 are deleted (FIFO head drop). **Queue Q0 (critical security and fault alarms) must never be evicted under any condition**. + +## 4. Crash Reporting & Hardware Fault Persistence + +When an unrecoverable runtime exception occurs (such as an ARM Cortex-M `HardFault`, Memory Management Fault, or Watchdog Timeout), the device must capture forensic diagnostic telemetry before executing a hardware reset. + +```c +// Embedded C: HardFault Forensic Context Dump +typedef struct { + uint32_t r0; + uint32_t r1; + uint32_t r2; + uint32_t r3; + uint32_t r12; + uint32_t lr; // Link Register (Return Address) + uint32_t pc; // Program Counter (Fault Location) + uint32_t psr; // Program Status Register + uint32_t cfsr;// Configurable Fault Status Register + uint32_t hfsr;// HardFault Status Register + uint32_t mmfar; // Memory Management Fault Address + uint32_t bfar; // Bus Fault Address +} crash_dump_t; + +void HardFault_Handler_C(uint32_t *hardfault_args) { + crash_dump_t dump; + dump.r0 = hardfault_args[0]; + dump.r1 = hardfault_args[1]; + dump.r2 = hardfault_args[2]; + dump.r3 = hardfault_args[3]; + dump.r12 = hardfault_args[4]; + dump.lr = hardfault_args[5]; + dump.pc = hardfault_args[6]; + dump.psr = hardfault_args[7]; + dump.cfsr = SCB->CFSR; + dump.hfsr = SCB->HFSR; + dump.mmfar = SCB->MMFAR; + dump.bfar = SCB->BFAR; + + // Write crash dump to dedicated raw backup flash sector (bypassing RTOS / interrupts) + raw_flash_emergency_write(CRASH_SECTOR_ADDR, (uint8_t *)&dump, sizeof(dump)); + + // Force system reboot + NVIC_SystemReset(); +} +``` + +Upon subsequent reboot, the primary boot routine inspects the crash dump sector. If it finds a valid crash record, it formats the dump as an urgent diagnostic event. It uploads the event to the cloud observability platform on the first network connection, and marks the sector as acknowledged. diff --git a/content/guides/iot-endpoint-and-gateway-architecture/05-edge-intelligence-and-processing.md b/content/guides/iot-endpoint-and-gateway-architecture/05-edge-intelligence-and-processing.md new file mode 100644 index 0000000..128b4dd --- /dev/null +++ b/content/guides/iot-endpoint-and-gateway-architecture/05-edge-intelligence-and-processing.md @@ -0,0 +1,121 @@ +--- +title: "Edge Intelligence & Processing" +--- + +# Edge Intelligence & Processing + +As edge devices become equipped with higher-performance microcontrollers and digital signal processing (DSP) extensions, transmitting raw, uncompressed sensor streams to the cloud becomes both economically unviable and technically inefficient. Transmitting high-frequency vibration signals (such as 10 kHz accelerometer samples) over cellular or satellite networks would deplete battery budgets and incur enormous cellular data costs. + +This chapter establishes architectural frameworks for partitioning compute between edge and cloud, executing digital signal processing on microcontrollers, deploying TinyML neural networks, and containerizing software on edge gateways. + +## 1. Compute Partitioning Framework + +Deciding where data processing occurs requires evaluating four core trade-off vectors: + +```mermaid +flowchart LR + subgraph EdgeTier["Edge Tier (Microcontroller)"] + Raw["High-Speed Sensors
(1 kHz - 100 kHz)"] --> DSP["DSP / TinyML Inference"] + DSP --> Anomaly{"Anomaly Detected?"} + end + subgraph CloudTier["Cloud Tier (Enterprise)"] + Model["Global Model Training & Fleet Analytics"] + end + Anomaly -->|Normal: Hourly Summary| CloudTier + Anomaly -->|Critical: Rich Feature Vector| CloudTier + CloudTier -.->|Model Retraining & Updates| DSP +``` + +[[CAPTION:Figure]] Compute partitioning topology filtering raw high-frequency signals into concise telemetry. + +| Vector | Edge Execution (MCU / Gateway) | Cloud Execution | +| :--- | :--- | :--- | +| **Response Latency** | Sub-millisecond deterministic control loops | 100 ms to 2 seconds (network-dependent) | +| **Network Bandwidth**| Transmits only filtered features or anomalies | Requires continuous high-bandwidth uplink | +| **Data Privacy** | Raw audio/video/vibration never leaves device | Sensitive raw data traverses public networks | +| **Compute Scalability**| Strictly bounded by local MCU RAM/Flash | Elastic, virtually unbounded GPU/CPU resources | + +[[CAPTION:Table]] Trade-off comparison between edge and cloud compute execution. + +## 2. On-Device Digital Signal Processing (DSP) + +Prior to invoking machine learning models or writing to flash, microcontrollers should apply classic digital signal processing algorithms to clean raw sensor inputs: + +```mermaid +flowchart TD + Raw["Raw ADC / Accelerometer Samples"] --> AntiAlias["1. Anti-Aliasing & Low-Pass Filter"] + AntiAlias --> EMA["2. Exponential Moving Average (EMA)"] + EMA --> FFT["3. Fast Fourier Transform (CMSIS-DSP)"] + FFT --> Features["4. Spectral Energy & Peak Frequency Extraction"] +``` + +[[CAPTION:Figure]] Digital signal processing pipeline on an embedded microcontroller. + +### 2.1 Exponential Moving Average (EMA) Filter + +For continuous environmental measurements (temperature, pressure, gas concentration), high-frequency electrical noise is eliminated using fixed-point exponential smoothing: + +$$y[n] = \alpha \cdot x[n] + (1 - \alpha) \cdot y[n-1]$$ + +Where $\alpha \in (0, 1]$ represents the smoothing factor. In embedded C, fixed-point arithmetic eliminates floating-point coprocessor power consumption: + +```c +// Embedded C: Integer Fixed-Point EMA Filter (Alpha = 1/8) +uint32_t filter_ema_fixed(uint32_t current_sample, uint32_t previous_filtered) { + // y[n] = (x[n] + 7 * y[n-1]) / 8 + return (current_sample + (7 * previous_filtered) + 4) >> 3; +} +``` + +### 2.2 Frequency-Domain Analysis with CMSIS-DSP + +For predictive maintenance on rotating industrial machinery, time-domain vibration data is transformed into the frequency domain using the **ARM CMSIS-DSP** RFFT (Real Fast Fourier Transform) library. By extracting spectral energy across specific harmonic bins (bearing fault frequencies), the MCU compresses 4,096 raw time-domain samples into a 16-byte frequency health vector. + +## 3. TinyML: Machine Learning on Microcontrollers + +Modern 32-bit microcontrollers (such as ARM Cortex-M4F, Cortex-M33, or Cortex-M55 with Helium vector extensions) can execute quantized deep neural networks directly in SRAM. + +```mermaid +flowchart LR + Dev["Cloud / Workstation Training
(TensorFlow / PyTorch)"] --> Quant["8-Bit Quantization (INT8)
Post-Training Quantization"] + Quant --> TFLM["TensorFlow Lite for Microcontrollers
Static C++ Tensor Arena"] + TFLM --> Silicon["ARM Cortex-M Silicon
(CMSIS-NN Optimized Kernels)"] +``` + +[[CAPTION:Figure]] TinyML model conversion and deployment pipeline. + +### 3.1 Architectural Constraints for TinyML + +1. **Strict INT8 Quantization**: Floating-point models (FP32) must be quantized to 8-bit integers (INT8). Quantization reduces model flash size by 75%, reduces SRAM consumption by 75%, and allows the processor to execute four SIMD multiply-accumulate operations in a single cycle. +1. **Pre-Allocated Tensor Arena**: Dynamic memory allocation (`malloc`) during inference is strictly prohibited. The runtime (such as TensorFlow Lite for Microcontrollers - TFLM) must be assigned a fixed, static byte array (`tensor_arena`) at initialization: + ```cpp + constexpr int kTensorArenaSize = 32 * 1024; // 32 KB Static SRAM Arena + alignas(16) uint8_t tensor_arena[kTensorArenaSize]; + ``` +1. **Inference Watchdog Protection**: Machine learning inference loops must be assigned a bounded execution time slice. If an inference task exceeds its time budget (for example, taking > 50 ms), the scheduler must preempt or terminate the model to prevent starving critical radio or sensor interrupts. + +## 4. Tier 3 Edge Gateway Containerization & Streaming + +On Tier 3 Linux gateways (running Yocto, Debian, or Ubuntu Core on ARM Cortex-A processors), edge intelligence transitions to containerized microservices: + +```mermaid +flowchart TD + subgraph GatewayOS["Tier 3 Linux Gateway (containerd / k3s)"] + Engine["Protocol Ingest (Modbus / BLE / OPC-UA)"] + Broker["Local MQTT Broker / Redis Cache"] + Stream["Stream Analytics Engine (LF Edge eKuiper)"] + ML["Local AI Container (ONNX Runtime / OpenVINO)"] + end + Engine --> Broker + Broker --> Stream + Stream --> ML + ML -->|Actionable Event| Cloud["Enterprise Cloud Ingestion"] +``` + +[[CAPTION:Figure]] Edge gateway containerized processing architecture. + +### 4.1 Edge Container Best Practices + +1. **Lightweight Container Runtimes**: Deploy `containerd` or `crun` rather than full Docker daemons to conserve RAM and CPU cycles on embedded gateways. +1. **Read-Only Root Filesystem**: Gateway operating systems must mount the root filesystem as read-only (`ro`), isolating dynamic container state and telemetry databases to an encrypted, wear-managed data partition. +1. **Edge Stream Processing**: Use lightweight edge stream engines (such as LF Edge eKuiper) to evaluate SQL-like filtering rules on continuous local telemetry streams, transmitting only aggregated metrics or anomalies to cloud brokers. diff --git a/content/guides/iot-endpoint-and-gateway-architecture/index.md b/content/guides/iot-endpoint-and-gateway-architecture/index.md new file mode 100644 index 0000000..49977e2 --- /dev/null +++ b/content/guides/iot-endpoint-and-gateway-architecture/index.md @@ -0,0 +1,35 @@ +--- +title: IoT endpoint and edge gateway architecture +--- + +# IoT endpoint and edge gateway architecture + +## Executive Overview + +Connected device engineering demands a rigorous, cross-disciplinary architectural approach that unifies low-power embedded hardware, real-time operating systems, wireless telecommunications, resilient local storage, and edge intelligence. Modern IoT ecosystems span from micro-watt sensor nodes surviving a decade on a coin cell to multi-core edge gateways executing containerized neural networks on industrial machinery. + +This guide establishes the definitive architecture reference for designing, implementing, and deploying generic connected endpoints and edge gateways across commercial, industrial, municipal, and consumer environments. + +```mermaid +flowchart TD + subgraph Handbook["IoT Endpoint & Edge Gateway Architecture Handbook"] + CH1["Ch 1: Device Tiers & Hardware Architecture"] + CH2["Ch 2: Connectivity & Protocol Trade-Offs"] + CH3["Ch 3: Power Architecture & Duty-Cycling"] + CH4["Ch 4: Edge Storage & Local Resilience"] + CH5["Ch 5: Edge Intelligence & Processing"] + end + Handbook --> Production["Production-Grade Field Systems"] +``` + +[[CAPTION:Figure]] Five architectural pillars of generic IoT endpoint and gateway design. + +## Target Audience & Guide Structure + +This guide is designed for embedded firmware engineers, hardware architects, IoT systems architects, and technical delivery leaders: + +- **[Chapter 1: Device Tiers & Hardware Architecture](01-device-tiers-and-hardware-architecture.md)**: Classifies devices across RFC 7228 tiers, silicon families (Cortex-M, RISC-V, Cortex-A), memory partitioning, and high-reliability bus topologies. +- **[Chapter 2: Connectivity & Protocol Trade-Offs](02-connectivity-and-protocol-tradeoffs.md)**: Comprehensive evaluation of wireless technologies (Cellular LPWAN, BLE, Wi-Fi 6, LoRaWAN) and transport protocols (CoAP, MQTT, gRPC). +- **[Chapter 3: Power Architecture & Duty-Cycling](03-power-architecture-and-duty-cycling.md)**: Power supply topologies, energy harvesting, tickless sleep state machines, and brownout protection. +- **[Chapter 4: Edge Storage & Local Resilience](04-edge-storage-and-local-resilience.md)**: Solid-state memory hierarchies, LittleFS wear leveling, power-fail safety, and priority offline queues. +- **[Chapter 5: Edge Intelligence & Processing](05-edge-intelligence-and-processing.md)**: Partitioning workloads between edge and cloud, TinyML embedded inference, stream filtering, and gateway containerization. diff --git a/content/guides/iot-fleet-operations/01-zero-touch-device-onboarding.md b/content/guides/iot-fleet-operations/01-zero-touch-device-onboarding.md new file mode 100644 index 0000000..2ba1417 --- /dev/null +++ b/content/guides/iot-fleet-operations/01-zero-touch-device-onboarding.md @@ -0,0 +1,85 @@ +--- +title: Zero-Touch Device Onboarding +--- + +# Zero-Touch Device Onboarding + +Deploying connected devices at fleet scale makes manual on-site configuration, keyboard credential entry, or technician pairing economically unviable. An installation technician mounting an asset tracker, air quality monitor, or industrial sensor should simply apply power, close the enclosure, and walk away. + +**Zero-Touch Onboarding** automates the entire authentication, authorisation, cryptographic credential issuance, and cloud assignment sequence upon first boot. + +```mermaid +flowchart TD + Power["1. Technician Powers On Device"] --> FactoryCert["2. Device Presents Factory Birth Certificate"] + FactoryCert --> BootServer["3. Connects to Global Bootstrap Server"] + BootServer --> Validate{"4. Validate Certificate & Whitelist"} + Validate -->|Approved| Issue["5. Issue Operational Certificate via EST"] + Issue --> Redirect["6. Redirect to Regional Operational Broker"] + Redirect --> Active["7. Device Fully Active in Fleet Registry"] +``` + +[[CAPTION:Figure]] End-to-end zero-touch device bootstrapping and onboarding lifecycle. + +## 1. Cryptographic Identity Separation: Birth vs Operational Certificates + +A fundamental principle of secure device onboarding is the strict decoupling of **Birth (Factory) Credentials** from **Operational (Runtime) Credentials**: + +| Credential Type | Issuing Authority | Storage Location | Lifetime | Revocation Mechanism | +| :--- | :--- | :--- | :--- | :--- | +| **Birth Certificate (IDevID)** | Factory Root of Trust CA | Secure Element (Locked Slot) | Lifetime of device hardware | Hardware blacklist in cloud | +| **Operational Certificate (LDevID)**| Enterprise Operational PKI CA | Secure Element or Encrypted Flash | 90 days to 2 years (Rotatable)| Cloud CRL / OCSP revocation | + +[[CAPTION:Table]] IEEE 802.1AR Initial Device Identifier (IDevID) versus Locally Significant Device Identifier (LDevID). + +1. **Initial Device Identifier (IDevID)**: Injected into the Secure Element during factory testing. It proves the physical provenance of the hardware to the global onboarding service. +1. **Locally Significant Device Identifier (LDevID)**: Generated during zero-touch onboarding. It grants access to specific customer tenants, MQTT topics, and enterprise cloud applications, and is rotated periodically throughout the device's operational life. + +## 2. Automated Provisioning Protocols: EST and LwM2M Bootstrap + +To transition a device from its factory state to its operational tenant, the platform deploys standardized certificate enrollment protocols: + +```mermaid +sequenceDiagram + participant Device as IoT Device (IDevID) + participant Boot as Global Bootstrap Service + participant CA as Enterprise Cloud CA + participant Ops as Regional MQTT Broker + + Note over Device,Boot: First Power-On in Field + Device->>Boot: mTLS Handshake using Factory IDevID Certificate + Boot->>Boot: Verify hardware serial against factory MES whitelist + Boot-->>Device: Return Regional Endpoint URL & CA Trust Bundle + + Note over Device,CA: Operational Certificate Enrollment (EST / RFC 7030) + Device->>Device: Generate new operational ECC P-256 keypair in Secure Element + Device->>Boot: POST /cacerts & /simpleenroll (PKCS#10 CSR signed by IDevID) + Boot->>CA: Authorize & Sign CSR + CA-->>Boot: Return signed LDevID Operational Certificate + Boot-->>Device: Deliver LDevID Certificate Chain + + Note over Device,Ops: Operational Connection + Device->>Ops: mTLS Connect using newly acquired LDevID Certificate + Ops-->>Device: Connection Accepted (Assigned to Customer Tenant) +``` + +[[CAPTION:Figure]] Automated certificate enrollment sequence using Enrollment over Secure Transport (EST). + +### 2.1 Enrollment over Secure Transport (EST - RFC 7030) + +EST operates over standard HTTPS/TLS, providing lightweight endpoints for certificate lifecycle management: +- `/simpleenroll`: The device sends a Certificate Signing Request (CSR) generated inside its Secure Element. The private key never leaves the device. The platform signs the CSR and returns an X.509 client certificate. +- `/simplereenroll`: Enables automated certificate rotation prior to expiration without requiring technician intervention. + +## 3. Just-in-Time Registration (JITR / JITP) + +In high-scale cloud platforms (such as AWS IoT Core or Azure Device Provisioning Service - DPS), **Just-in-Time Provisioning (JITP)** streamlines onboarding: +1. The enterprise registers its intermediate CA with the cloud IoT registry once. +1. When a new device connects for the first time presenting a client certificate signed by that CA, the cloud automatically triggers a provisioning hook (such as an AWS Lambda function or Azure Function). +1. The hook validates device ownership, creates the Digital Twin, applies least-privilege security policies, assigns customer metadata tags, and publishes a welcome message back to the device. + +## 4. Carrier-Integrated Provisioning via eSIM (GSMA SGP.32) + +For cellular connected fleets deployed internationally: +- Devices utilise **eSIM (eUICC)** hardware conforming to the **GSMA SGP.32** standard for IoT. +- Upon booting in a new country, the cellular modem connects via a low-cost global bootstrap profile. +- The carrier's Subscription Manager Data Preparation (SM-DP+) server pushes a localized domestic mobile network operator (MNO) profile over the air, switching the device to local commercial cellular data rates without physical SIM replacement. diff --git a/content/guides/iot-fleet-operations/02-scalable-telemetry-ingestion.md b/content/guides/iot-fleet-operations/02-scalable-telemetry-ingestion.md new file mode 100644 index 0000000..7e82f56 --- /dev/null +++ b/content/guides/iot-fleet-operations/02-scalable-telemetry-ingestion.md @@ -0,0 +1,95 @@ +--- +title: Scalable Telemetry Ingestion +--- + +# Scalable Telemetry Ingestion + +When an IoT fleet grows from ten thousand to one million devices, the telemetry pipeline must absorb millions of concurrent TCP connections and millions of messages per second. It must also survive the thundering herd of reconnections after a network outage ends. A naive cloud architecture that writes directly to relational databases will collapse under connection limits and disk I/O bottlenecks. + +This chapter establishes the reference architecture for decoupling ingest brokers, event streaming pipelines, time-series storage, and rate-limiting controls. + +## 1. High-Scale Ingestion Pipeline Architecture + +The ingestion pipeline decouples protocol termination from downstream business processing using a multi-stage streaming pipeline: + +```mermaid +flowchart TD + Devices["1,000,000+ Field IoT Devices"] --> NLB["Network Load Balancers (Layer 4)"] + NLB --> BrokerCluster["Clustered MQTT Broker Fleet
(EMQX / HiveMQ / AWS IoT)"] + BrokerCluster --> Stream["Distributed Event Log
(Apache Kafka / Pulsar)"] + + subgraph Processing["Downstream Consumer Squads"] + Stream --> HotStore["Hot Storage: Time-Series DB
(ClickHouse / TimescaleDB)"] + Stream --> TwinEngine["Digital Twin & Rules Engine"] + Stream --> ColdStore["Cold Storage: Parquet Data Lake
(AWS S3 / GCP GCS)"] + end +``` + +[[CAPTION:Figure]] Decoupled event streaming architecture for high-volume IoT telemetry. + +### 1.1 Ingestion Layers & Responsibilities + +1. **Layer 4 Network Load Balancing**: Distributes incoming mTLS connections across broker nodes without terminating TLS, preserving client certificate authentication down to the application brokers. +1. **Clustered MQTT Broker Tier**: Terminates TLS handshakes, verifies client identities, enforces topic ACLs, and immediately bridges payload bytes into Kafka topics. The broker retains no business logic. +1. **Distributed Event Streaming (Kafka / Pulsar)**: Serves as a persistent, high-throughput buffer that absorbs upstream traffic spikes without dropping messages. Downstream database consumers pull records at their own sustainable processing rate. + +## 2. Kafka Partitioning & In-Order Telemetry Guarantees + +In distributed event streaming, partitioning strategy dictates whether telemetry can be processed in chronological order: + +```mermaid +flowchart LR + Msg["Telemetry Message
Key: Device_UUID_1048"] --> Hash{"Hash(Key) % Partitions"} + Hash -->|Deterministic Route| P2["Partition 2"] + P2 --> Consumer["Dedicated Consumer Worker
(Processes Events Sequentially)"] +``` + +[[CAPTION:Figure]] Key-based partitioning guaranteeing sequential telemetry consumption per physical device. + +### 2.1 Partitioning Rules + +> [!IMPORTANT] +> Messages produced to the event streaming layer must use the unique **Device Hardware UUID** as the Kafka message key. This guarantees that all telemetry emitted by a single physical device is routed to the exact same Kafka partition, ensuring strict in-order processing. + +- Never use random keys or timestamps as partition keys; doing so scatters a device's readings across different consumer workers, causing race conditions in state calculation. +- If a tenant has thousands of devices, prefixing keys with `tenant_id/device_id` preserves per-device ordering while allowing tenant-level stream filtering. + +## 3. Hot vs Cold Time-Series Storage Architecture + +Storing years of high-frequency telemetry in transactional databases incurs unsustainable infrastructure costs. Deploy a tiered storage lifecycle: + +| Storage Tier | Technology | Retention Period | Query Latency | Storage Cost / GB | Intended Purpose | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **Hot Tier** | ClickHouse / TimescaleDB | 30 to 90 days | < 50 ms | High (Fast NVMe SSD) | Operational dashboards, real-time alerting | +| **Warm Tier** | Compressed Parquet on Object Storage | 90 to 365 days | 1 s to 5 s | Low (Standard S3/GCS) | Machine learning training, trend analytics | +| **Cold / Glacier**| Encrypted Parquet Archives | 1 to 10+ years | Minutes to Hours | Ultra-low (Glacier / Deep Archive) | Regulatory compliance, long-term auditing | + +[[CAPTION:Table]] Telemetry data storage tiering strategy. + +### 3.1 ClickHouse Table Optimisation for IoT + +ClickHouse provides extreme compression (often exceeding 80% to 90%) and sub-second aggregation across billions of rows when configured with appropriate primary keys: + +```sql +CREATE TABLE iot_telemetry ( + tenant_id UUID, + device_id UUID, + timestamp DateTime64(3, 'UTC'), + metric_name LowCardinality(String), + val_float Float64, + status_flags UInt32 +) ENGINE = MergeTree() +PARTITION BY toYYYYMM(timestamp) +ORDER BY (tenant_id, metric_name, device_id, timestamp); +``` + +By placing `tenant_id`, `metric_name`, and `device_id` at the beginning of the `ORDER BY` tuple, queries that aggregate specific sensor readings over time span contiguous disk blocks, minimizing disk seeks. + +## 4. Backpressure, Throttling & Thundering Herd Defence + +When a regional cellular tower experiences an outage and comes back online, tens of thousands of sleeping devices wake up and attempt to reconnect simultaneously—creating a **thundering herd**: + +1. **Randomized Exponential Backoff with Jitter**: Embedded devices must never reconnect on fixed intervals. Reconnection attempts must incorporate random jitter: + $$t_{wait} = \min(t_{max}, t_{base} \times 2^{attempt}) + \mathrm{Uniform}(-0.5, +0.5) \times t_{base}$$ +1. **Token Bucket Rate Limiting at Ingest Gateways**: The MQTT broker tier must enforce per-device and per-tenant connection and publish rate limits (for example, maximum 5 connects per minute and 1 publish per second per device). Excess attempts receive MQTT disconnects with reason code `0x9D` (Message rate too high) or `0x89` (Server busy). +1. **Non-Critical Telemetry Shedding**: If the Kafka cluster or cloud consumers experience high lag, the edge gateway can temporarily instruct field devices to downsample or drop verbose diagnostic logs, preserving broker bandwidth for critical alarms. diff --git a/content/guides/iot-fleet-operations/03-digital-twins-and-state-reconciliation.md b/content/guides/iot-fleet-operations/03-digital-twins-and-state-reconciliation.md new file mode 100644 index 0000000..3ac5551 --- /dev/null +++ b/content/guides/iot-fleet-operations/03-digital-twins-and-state-reconciliation.md @@ -0,0 +1,129 @@ +--- +title: "Digital Twins & State Reconciliation" +--- + +# Digital Twins & State Reconciliation + +In large-scale IoT fleets, physical devices operate under intermittent connectivity, while enterprise web consoles, mobile applications, and scheduling microservices require continuous, instant read-and-write access to device settings. Directly issuing synchronous HTTP or RPC requests to physical field hardware fails because devices are frequently asleep, out of coverage, or executing high-priority sensing routines. + +The **Digital Twin** (or **Device Shadow**) architecture decouples operational intent from physical execution by maintaining a persistent cloud state document synchronized with physical hardware. + +```mermaid +flowchart TD + App["Enterprise Web App / API"] -->|1. Set Desired State| Twin["Cloud Digital Twin Document"] + Twin --> Compute["Delta Computation Engine"] + Compute -->|2. Compute Diff| Delta["Delta Event: {interval: 300}"] + Delta --> Broker["MQTT Broker"] + Broker -->|3. Deliver upon Wakeup| Device["Field IoT Device"] + Device -->|4. Apply & Reconcile| Hardware["Microcontroller Peripherals"] + Hardware -->|5. Verify Operation| Device + Device -->|6. Publish Reported State| Twin +``` + +[[CAPTION:Figure]] Desired-to-reported convergence cycle in large-scale digital twins. + +## 1. Digital Twin Document Model + +A production-grade digital twin document partitions device properties into four primary top-level structures: + +```json +{ + "state": { + "desired": { + "reporting_interval_seconds": 300, + "temp_alarm_threshold_celsius": 45.0, + "firmware_target_version": "v3.2.0" + }, + "reported": { + "reporting_interval_seconds": 900, + "temp_alarm_threshold_celsius": 45.0, + "firmware_target_version": "v3.1.4", + "battery_voltage_mv": 3620, + "hardware_sku": "EDGE-SENS-REV2" + }, + "delta": { + "reporting_interval_seconds": 300, + "firmware_target_version": "v3.2.0" + } + }, + "metadata": { + "desired": { + "reporting_interval_seconds": { "timestamp": 1774182000, "updated_by": "operator_alice" } + }, + "reported": { + "reporting_interval_seconds": { "timestamp": 1774170000 } + } + }, + "version": 1284, + "last_updated": 1774182005 +} +``` + +[[CAPTION:Table]] Digital twin JSON state representation. + +1. **Desired State**: Represents the configuration intended by cloud systems or human operators. +1. **Reported State**: Represents the actual, physical reality reported by the device's firmware and onboard sensors. +1. **Delta State**: Dynamically computed by the platform as $\mathrm{Desired} \setminus \mathrm{Reported}$. If an element in `desired` matches `reported`, it is automatically stripped from the delta. +1. **Metadata & Versioning**: Tracks timestamps, updating identities, and a monotonically increasing 64-bit integer `version` used for optimistic concurrency locking. + +## 2. Multi-Tenant Hierarchical Twin Configuration + +In fleets spanning thousands of enterprise customers, managing configuration per individual device is error-prone. The platform must implement **Hierarchical Property Inheritance**: + +```mermaid +flowchart TD + Fleet["Global Fleet Template
(reporting_interval: 900, log_level: WARN)"] + Tenant["Customer Tenant Model
(reporting_interval: 300)"] + Group["Device Group: High-Heat Area
(temp_threshold: 40.0)"] + Device["Specific Device Node
(temp_threshold: 38.5)"] + + Fleet --> Tenant + Tenant --> Group + Group --> Device +``` + +[[CAPTION:Figure]] Hierarchical property inheritance and group-level cascading configuration. + +- Settings cascade downwards: Device-level configurations override Group-level configurations, which in turn override Tenant- and Fleet-level defaults. +- Updating a single property on the Device Group automatically recomputes and dispatches deltas across all member devices upon their next communication session. + +## 3. Concurrency Control & State Conflict Resolution + +When multiple applications or automated workflows attempt to mutate a digital twin simultaneously, race conditions can corrupt system state: + +### 3.1 Optimistic Concurrency Control + +Every shadow update request from a cloud application must include the expected `version` tag. If the document has been modified by another process in the interim, the twin service rejects the transaction with a `409 Conflict` error: + +```mermaid +sequenceDiagram + participant App1 as Web Dashboard + participant App2 as Automated Optimizer + participant Twin as Cloud Shadow Service + + App1->>Twin: Read Shadow (Version: 42) + App2->>Twin: Read Shadow (Version: 42) + + App2->>Twin: Update Desired { interval: 600 } (Expected Version: 42) + Twin->>Twin: Version matches (42 == 42). Commit update, then increment Version to 43. + Twin-->>App2: HTTP 200 OK (New Version: 43) + + App1->>Twin: Update Desired { interval: 300 } (Expected Version: 42) + Twin->>Twin: Version mismatch (42 != 43). Reject! + Twin-->>App1: HTTP 409 Conflict (Current Version is 43) + Note over App1: App1 must reload version 43 and re-apply changes +``` + +[[CAPTION:Figure]] Optimistic concurrency control preventing lost update anomalies. + +### 3.2 Last-Write-Wins (LWW) with Monotonic Timers + +For reported telemetry where devices reconnect after weeks offline, clocks might drift. The twin service uses UTC microsecond timestamps recorded by the ingestion broker to resolve conflicts. If an inbound reported message carries a timestamp older than the current metadata timestamp, the update is dropped as stale. + +## 4. Edge State Machine Reconciliation Rules + +The firmware running on the connected hardware must implement strict validation before acknowledging configuration updates: + +1. **Schema & Range Validation**: If an incoming delta requests `reporting_interval_seconds: -10` or a value outside physical transducer limits, the device must reject the update, report an error flag in `reported.status`, and leave the existing operational configuration intact. +1. **Resource & Battery Verification**: If an update requests high-power operations (such as energizing a cellular modem continuously or activating a heater), the device checks current battery voltage and ambient temperature. If resources are insufficient, the configuration change is rejected. +1. **Atomic Flash Persistence**: Validated configuration must be written to wear-leveled non-volatile storage (LittleFS) before acknowledging the new state in `reported`. This ensures that if the device reboots five seconds later, it boots with the newly applied configuration. diff --git a/content/guides/iot-fleet-operations/04-fleet-scale-ota-campaigns.md b/content/guides/iot-fleet-operations/04-fleet-scale-ota-campaigns.md new file mode 100644 index 0000000..508023d --- /dev/null +++ b/content/guides/iot-fleet-operations/04-fleet-scale-ota-campaigns.md @@ -0,0 +1,111 @@ +--- +title: Fleet-Scale OTA Update Campaigns +--- + +# Fleet-Scale OTA Update Campaigns + +Firmware Over-The-Air (FOTA) updates represent both the most critical operational capability and the single highest-risk procedure in connected device engineering. A bug in cloud software can be rolled back within seconds by an SRE squad. A flawed firmware update dispatched globally can simultaneously brick an entire fleet of field-deployed hardware, requiring catastrophic physical recalls. + +This chapter defines the engineering requirements for managing fleet-scale OTA campaigns, executing phased canary rollouts, minimizing cellular bandwidth via delta compression, and enforcing autonomous bootloader rollbacks. + +## 1. Phased Staged Rollouts & Canary Deployments + +Dispatched updates must never be released to a production fleet simultaneously. Campaigns must progress through five gated phases: + +```mermaid +flowchart LR + P0["Phase 0: Lab HIL Racks
(10 Units, 24 Hours)"] --> P1["Phase 1: Canary Dogfood
(0.1% Fleet, 48 Hours)"] + P1 --> P2["Phase 2: Pilot Fleet
(1% Fleet, 5 Days)"] + P2 --> P3["Phase 3: Broad Staged
(10% Fleet, 7 Days)"] + P3 --> P4["Phase 4: Full Deployment
(100% Fleet)"] +``` + +[[CAPTION:Figure]] Five-phase staged rollout sequence for risk-managed OTA deployments. + +### 1.1 Automated Campaign Circuit Breakers + +Every active campaign must be monitored by automated health supervisors in the cloud: + +> [!CAUTION] +> If the cumulative failure rate (devices reporting download failures, bootloader CRC errors, or watchdog crash loops) exceeds **0.5% of targeted nodes**, the campaign engine must immediately trip an automated circuit breaker, halting further firmware distributions and alerting engineering on-call personnel. + +## 2. Bandwidth Optimisation: Delta Updates (BSDiff / Courgette) + +Sending a full 1 MB or 2 MB firmware image over cellular LPWAN (NB-IoT) to one million devices would use terabytes of data. It would take weeks and drain the batteries. + +The enterprise OTA platform must support **Binary Delta Compression**: + +```mermaid +flowchart TD + Old["Base Firmware (v2.1.0)"] --> Diff{"Binary Diff Engine
(BSDiff / Courgette)"} + New["Target Firmware (v2.2.0)"] --> Diff + Diff --> Delta["Signed Delta Patch (v2.1 -> v2.2)
(Typical Size: 35 KB - 80 KB)"] + Delta --> Device["Edge Device (Runs v2.1.0)"] + Device --> Reconstruct["On-Device Patch Reconstructor"] + Reconstruct --> ActiveBank["Writes Assembled v2.2.0 to Bank B"] +``` + +[[CAPTION:Figure]] Delta compression generating compact patches from existing base images. + +1. **85% to 95% Bandwidth Reduction**: By transmitting only the changed binary symbols and instruction relocations, the payload size drops from ~1 MB down to 40 KB. +1. **Deterministic Verification**: Before applying a delta patch, the edge device validates the SHA-256 hash of its currently executing flash image (Bank A) to ensure the base matches the delta expectations exactly. Once reconstructed in Bank B, the full binary hash is verified against the signed release manifest. + +## 3. Dual-Bank A/B Bootloader Architecture & Automatic Rollback + +Hardware must implement dual-bank flash partitioning (Bank A and Bank B) coordinated by a cryptographically secure bootloader (such as MCUboot or TF-M): + +```mermaid +sequenceDiagram + participant Cloud as OTA Cloud Service + participant App as Active Firmware (Bank A) + participant BL as Secure Bootloader + participant NewBank as Candidate (Bank B) + participant WDT as Hardware Watchdog + + Cloud->>App: Deliver Signed Firmware Binary / Delta + App->>NewBank: Stream chunks into Bank B, then Verify ECDSA P-256 Signature + App->>App: Mark Bank B as "Candidate / Test Run" in NVS + App->>BL: Issue System Reboot + + BL->>BL: Detect "Test Run" flag, then Boot execution from Bank B + BL->>NewBank: Jump to Application Entry Point (Bank B) + + alt Firmware Boots Successfully + NewBank->>WDT: Initialize Watchdog + NewBank->>Cloud: Connect to MQTT Broker & Perform Handshake + NewBank->>NewBank: Mark Bank B as "Permanently Active / Confirmed" + Note over NewBank,BL: Update Permanent! Bank A now available as scratch. + else Firmware Crashes / HardFaults + Note over NewBank: System hangs in infinite loop or HardFault + WDT-->>BL: Watchdog times out (8 seconds), then Triggers Hardware Reset! + BL->>BL: Detect failed trial boot! Clear "Test Run" flag. + BL->>App: Roll back and boot proven image in Bank A! + App->>Cloud: Report FOTA Failure (REASON: CRASH_LOOP_WATCHDOG) + end +``` + +[[CAPTION:Figure]] Dual-bank trial boot sequence ensuring automated self-healing rollback. + +### 3.1 Rules for Trial Boot Confirmation + +1. **Autonomous Rollback**: The new firmware image must be booted in a provisional "trial" state. If the device reboots before confirming stability, the bootloader automatically reverts to the previous known-good bank. +1. **Mandatory Handshake Before Confirmation**: Firmware must never mark itself as permanently confirmed simply because it reached `main()`. Confirmation requires successfully passing three criteria: + - Initializing all sensor buses and peripherals without errors. + - Establishing an encrypted connection with the cloud MQTT broker. + - Successfully receiving an acknowledgment packet from the cloud twin service. + +## 4. Fleet Campaign Lifecycle Metrics + +The fleet operations dashboard must track the real-time state machine of every device targeted by the campaign: + +| Campaign State | Description | Next Permitted State | +| :--- | :--- | :--- | +| **Pending** | Device targeted; waiting for next communication window | Downloading | +| **Downloading** | Chunks currently streaming to Bank B over radio | Validating, Failed | +| **Validating** | Binary received; checking ECDSA signature and SHA-256 | Rebooting, Failed | +| **Rebooting** | Device has scheduled reset into candidate bank | Confirming, Rolled Back | +| **Confirming** | Candidate bank running in trial mode; testing cloud link | Succeeded, Rolled Back | +| **Succeeded** | Candidate permanently confirmed; twin updated | Complete | +| **Rolled Back** | Watchdog reset during trial boot; reverted to prior bank | Failed | + +[[CAPTION:Table]] Finite state machine states during fleet OTA campaigns. diff --git a/content/guides/iot-fleet-operations/05-fleet-observability-and-remote-diagnostics.md b/content/guides/iot-fleet-operations/05-fleet-observability-and-remote-diagnostics.md new file mode 100644 index 0000000..e36f55c --- /dev/null +++ b/content/guides/iot-fleet-operations/05-fleet-observability-and-remote-diagnostics.md @@ -0,0 +1,109 @@ +--- +title: "Fleet Observability & Remote Diagnostics" +--- + +# Fleet Observability & Remote Diagnostics + +Maintaining high reliability across a geographically dispersed fleet of connected devices requires proactive observability. When a field device silently drops connections, exhausts its battery prematurely, or encounters memory corruption, operations teams must detect the failure pattern before end users experience outages. + +However, standard enterprise observability tools (such as heavy APM agents or verbose OpenTelemetry streaming) cannot run directly on microcontrollers due to RAM, compute, and bandwidth constraints. This chapter details observability architectures adapted specifically for constrained IoT fleets. + +## 1. The Three Pillars of Edge Observability + +Observability for connected devices must be structured around three disciplined telemetry streams: + +```mermaid +flowchart TD + subgraph Observability["Edge-Adapted Fleet Observability"] + M["1. Compact Fleet Health Metrics
(Heap Watermark, Battery mV, RSRP, Reconnects)"] + L["2. Forensic Structured Crash Logs
(Captured on Flash; Uploaded upon Reboot)"] + T["3. Distributed W3C Trace Context
(Traceparent Injected in MQTT v5 Headers)"] + end + Observability --> Platform["Unified Cloud Observability Platform
(Grafana / OpenSearch / Prometheus)"] +``` + +[[CAPTION:Figure]] The three pillars of edge fleet observability adapted for constrained systems. + +## 2. Core Fleet Health Metrics + +Every routine telemetry message or diagnostic heartbeat emitted by an endpoint should bundle concise, single-byte health indicators alongside physical sensor data: + +| Metric Category | Specific KPI | Format | Typical Threshold / Warning | +| :--- | :--- | :--- | :--- | +| **RF & Link Quality** | RSRP (dBm) / RSRQ (dB) | Int8 (-140 to -40) | RSRP < -115 dBm (Marginal signal) | +| **RF Coverage Level** | 3GPP CE Level (0, 1, 2) | UInt8 (0, 1, 2) | CE Level 2 (High battery drain alert) | +| **Memory Health** | SRAM Minimum Free Watermark | UInt16 (Bytes) | Free SRAM < 2048 Bytes (Near OOM) | +| **Power Health** | Battery Terminal Voltage | UInt16 (Millivolts) | Drop rate > 50 mV / month | +| **System Uptime** | Reset Reason Code | Enum (POR, BOR, WDT) | Watchdog or BOR reset flags an alert | +| **Network Reliability**| TCP Disconnect Count | UInt8 (Count) | > 5 disconnects per 24 hours | + +[[CAPTION:Table]] Standard embedded diagnostic health indicators. + +```mermaid +xychart-beta + title "Fleet-Wide RF RSRP Signal Distribution (dBm)" + x-axis ["< -120 dBm (Poor)", "-110 to -120", "-100 to -110", "-90 to -100", "> -90 dBm (Excellent)"] + y-axis "Percentage of Fleet" 0 --> 50 + bar [4, 12, 38, 32, 14] +``` + +[[CAPTION:Figure]] Fleet signal distribution identifying units operating in marginal cellular coverage. + +## 3. Remote Diagnostic Harvesting & Crash Dumps + +When an unhandled exception or crash occurs, the device cannot stream logs in real time because the execution core is compromised. + +```mermaid +sequenceDiagram + participant MCU as MCU Hardware / RTOS + participant Flash as Non-Volatile Flash (LittleFS) + participant Cloud as Cloud Fleet Operations + + Note over MCU: HardFault / Memory Violation Occurs + MCU->>Flash: HardFault ISR writes registers, PC, LR & stack trace to crash sector + MCU->>MCU: NVIC_SystemReset() executes + + Note over MCU,Cloud: System Reboots into Clean Operational State + MCU->>Flash: Boot sequence detects unread crash dump in sector + MCU->>Cloud: Establish MQTT session, then Publish /devices/{id}/diagnostics/crash + Cloud-->>MCU: Acknowledge crash receipt + MCU->>Flash: Clear crash dump sector + Note over Cloud: Cloud maps PC to ELF symbols, then Alerts on-call squad! +``` + +[[CAPTION:Figure]] Forensic crash dump harvesting sequence from silicon exception to cloud triage. + +### 3.1 Automated Symbol Demangling in CI/CD + +When the cloud platform ingests a raw crash dump containing hexadecimal register values (`pc: 0x08012A4C`, `lr: 0x08011E20`): +1. The cloud diagnostic service looks up the firmware build manifest matching the device's reported firmware hash. +1. The service invokes `arm-none-eabi-addr2line` against the stored release `.elf` symbol file. +1. The hexadecimal addresses are demangled into human-readable source code locations: + `0x08012A4C -> sensor_driver.c:142 (sensor_read_i2c_dma)`. +1. An issue is automatically logged in the team's defect tracking system, complete with stack trace and hardware telemetry. + +## 4. Distributed Tracing Across Edge and Cloud + +When an operator issues a command from an enterprise web interface (such as triggering an on-demand sensor calibration), tracking the end-to-end transaction latency requires distributed tracing: + +```mermaid +flowchart LR + Web["Web Dashboard
(TraceID: 4bf92f3577b34da6)"] --> API["Cloud API Gateway"] + API --> Broker["MQTT v5 Broker
(Injects Traceparent User Property)"] + Broker --> Device["Field IoT Device
(Extracts Traceparent)"] + Device --> Cloud["Telemetry ACK
(Returns Traceparent)"] +``` + +[[CAPTION:Figure]] W3C Trace Context propagation across MQTT v5 user properties. + +1. **W3C Trace Context**: The cloud API assigns a standard `traceparent` header to the outbound command: + `traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01`. +1. **MQTT v5 User Properties**: The MQTT broker injects the traceparent into the MQTT v5 PUBLISH variable header. +1. **Edge Propagation**: The device logs the traceparent with its local execution timestamp and echoes it in its acknowledgment publish, providing full end-to-end tracing visibility in distributed APM dashboards. + +## 5. Predictive Fleet Health & SLA Monitoring + +By analyzing aggregated diagnostic streams across the entire fleet, machine learning models can detect impending hardware failures before service disruptions occur: +- **Battery Depletion Regression**: Predicts the exact month a cell will reach brownout cutoff based on seasonal temperature fluctuations and discharge slopes. +- **Cell Tower Degradation**: Identifies when a cellular carrier tower experiences hardware faults by detecting sudden correlated drops in RSRP across multiple independent devices in the same geographic cell sector. +- **Flash Wear Warnings**: Alerts operations when high-frequency logging drives a device's LittleFS erase cycles past 80% of rated NOR flash block endurance. diff --git a/content/guides/iot-fleet-operations/index.md b/content/guides/iot-fleet-operations/index.md new file mode 100644 index 0000000..1ee5d90 --- /dev/null +++ b/content/guides/iot-fleet-operations/index.md @@ -0,0 +1,35 @@ +--- +title: IoT fleet operations, telemetry and digital twins +--- + +# IoT fleet operations, telemetry and digital twins + +## Executive Overview + +Operating a production fleet of connected IoT devices across hundreds of thousands or millions of physical nodes presents unique distributed systems challenges. Cloud servers are uniform and sit in temperature-controlled data centres. IoT fleets are in uncontrolled environments, with intermittent connectivity and mixed firmware versions, and engineers cannot reach them when software fails. + +This guide sets out how to run a large IoT fleet through its whole life cycle. It covers zero-touch onboarding in the factory, telemetry ingestion at scale, digital twin reconciliation, fleet-wide OTA campaigns and automated observability. + +```mermaid +flowchart TD + subgraph Operations["IoT Fleet Operations Lifecycle"] + CH1["Ch 1: Zero-Touch Device Onboarding"] + CH2["Ch 2: Scalable Telemetry Ingestion"] + CH3["Ch 3: Digital Twins & State Reconciliation"] + CH4["Ch 4: Fleet-Scale OTA Update Campaigns"] + CH5["Ch 5: Fleet Observability & Remote Diagnostics"] + end + Operations --> FleetScale["1,000,000+ Field-Deployed Nodes"] +``` + +[[CAPTION:Figure]] The five operational pillars of large-scale IoT fleet management. + +## Target Audience & Guide Structure + +This guide is designed for cloud architects, platform engineers, site reliability engineers (SREs), and IoT operations managers: + +- **[Chapter 1: Zero-Touch Device Onboarding](01-zero-touch-device-onboarding.md)**: Automated bootstrapping, cryptographic claim workflows, Enrollment over Secure Transport (EST), and carrier provisioning. +- **[Chapter 2: Scalable Telemetry Ingestion](02-scalable-telemetry-ingestion.md)**: High-throughput broker clusters, Kafka event streaming, time-series storage, partitioning, and rate-limiting. +- **[Chapter 3: Digital Twins & State Reconciliation](03-digital-twins-and-state-reconciliation.md)**: Decoupled shadow architecture, desired versus reported state convergence, conflict handling, and offline synchronization. +- **[Chapter 4: Fleet-Scale OTA Update Campaigns](04-fleet-scale-ota-campaigns.md)**: Phased canary campaigns, blast radius containment, binary delta distribution, and automatic bootloader rollbacks. +- **[Chapter 5: Fleet Observability & Remote Diagnostics](05-fleet-observability-and-remote-diagnostics.md)**: Fleet health metrics, cellular radio diagnostics, crash dump aggregation, distributed tracing, and SLA monitoring. diff --git a/content/guides/scaling-architecture/01-advice-process.md b/content/guides/scaling-architecture/01-advice-process.md new file mode 100644 index 0000000..c6f4aad --- /dev/null +++ b/content/guides/scaling-architecture/01-advice-process.md @@ -0,0 +1,83 @@ +--- +title: The Advice Process +--- + +# The Advice Process + +The foundational engine of conversational architecture is the **Advice Process**. Adapted from organizational governance models and pioneered in software engineering by Andrew Harmel-Law, the Advice Process replaces hierarchical approval chains with empowered peer consultation. + +## 1. The Rule and the Qualifier + +The Advice Process comprises a single rule and one essential qualifier: + +> **The Rule**: Anyone in the engineering organisation can make any architectural decision. + +> **The Qualifier**: Before taking the decision, the decision-taker MUST consult two distinct groups: +> 1. Everyone who will be **meaningfully affected** by the decision. +> 2. People with **demonstrable expertise** in the domain of the decision. + +```mermaid +flowchart TD + A[Engineer / Squad Identifies Architectural Need] --> B[Draft Proposal / ADR Context] + B --> C[Consult Meaningfully Affected Parties] + B --> D[Consult Subject Matter Experts] + C --> E[Actively Solicit Dissenting Perspectives] + D --> E + E --> F[Synthesise Feedback in Decision Record] + F --> G[Decision-Taker Makes Final Call] + G --> H[Publish ADR to Codebase] +``` +[[CAPTION:Figure]] The Advice Process decision workflow. + +### Advice is Not Consensus + +A frequent misunderstanding is equating the Advice Process with seeking consensus or unanimous committee agreement. + +- **Consensus** is slow, stifling, and political; it waters down bold architectural ideas to the lowest common denominator and allows single obstinate voices to veto innovation. +- **The Advice Process** places complete decision-making authority and accountability on the individual or squad taking the call. The decision-taker is under no obligation to agree with the feedback received. However, they must listen actively, consider implications, and formally record the advice in the Architecture Decision Record ([Writing ADRs and RFCs](../decision-records.md)). + +## 2. Who to Consult: Mapping the Advice Scope + +To prevent engineers from feeling overwhelmed by consultation lists, teams maintain clear stakeholder matrices: + +| Architectural Impact Area | Affected Parties to Consult | Subject Matter Experts to Consult | +| :--- | :--- | :--- | +| **API Contract Modification** | Downstream consuming squads, mobile app leads, third-party integration engineers. | Principal API Architect, Integration Tech Lead. | +| **New Cloud Data Store Adoption** | Operations/SRE on-call engineers, billing/FinOps owners. | Cloud Data Architect, Database Administrator. | +| **Microcontroller Hardware Swap** | Firmware developers, manufacturing test engineers, hardware layout team. | Lead Embedded Architect, Hardware Engineering Lead. | +| **Security / Auth Model Change** | All client developers, QA automation team, support desk. | Chief Information Security Officer (CISO), Security Architect. | +| **Personal Identifiable Info (PII)** | Data pipeline engineers, customer operations. | Data Protection Officer, Legal & Compliance Counsel. | + +[[CAPTION:Table]] Advice Process consultation mapping matrix. + +## 3. The Power of Seeking Out Dissent + +High-performing architects do not seek echo chambers; they actively seek out colleagues who are most likely to disagree with their proposed direction. + +When decision-takers know they do not need consensus, fear of disagreement vanishes. Instead of defensive posturing, the interaction becomes an objective exploration of failure modes: +- *"What catastrophic failure mode am I failing to anticipate?"* +- *"If you had to break this design under peak load, how would you do it?"* +- *"What operational friction will your squad experience if we deploy this pattern?"* + +Hearing and documenting dissenting viewpoints surfaces unconsidered edge cases, transforms skeptics into respected contributors, and builds deep institutional trust. + +## 4. Right-Sizing Architectural Decisions + +The Advice Process contains a self-regulating economic forcing function: + +```mermaid +flowchart LR + LARGE[Massive Architecture Change: Requires consulting 40 engineers] -->|Cognitive Load| FORK{Can we right-size?} + FORK -- Yes --> SMALL[Decompose into 3 independent, isolated decisions] + SMALL -->|Each requires consulting only 3-4 peers| FAST[Rapid Execution & Safe Delivery] + FORK -- No --> CONSULT[Proceed with wide consultation for truly enterprise-scale shift] +``` +[[CAPTION:Figure]] Natural right-sizing of architectural decisions under the Advice Process. + +An engineer who sees that an organisation-wide change means consulting twenty teams will naturally pause. They ask: *"Can I decouple this change so it only affects my bounded context?"* + +By right-sizing decisions, teams minimise blast radiuses, limit inter-team dependencies, and accelerate release cadences without descending into chaos. + +## Summary + +The Advice Process decentralizes decision-making while ensuring decisions remain informed and accountable. In Chapter 2, we examine the four organizational alignment mechanisms that support this practice. diff --git a/content/guides/scaling-architecture/02-alignment-mechanisms.md b/content/guides/scaling-architecture/02-alignment-mechanisms.md new file mode 100644 index 0000000..8a85dca --- /dev/null +++ b/content/guides/scaling-architecture/02-alignment-mechanisms.md @@ -0,0 +1,95 @@ +--- +title: The Four Alignment Mechanisms +--- + +# The Four Alignment Mechanisms + +To prevent decentralized decision-making from fracturing into architectural fragmentation, the Advice Process MUST be anchored by four interconnected alignment and learning mechanisms: + +```mermaid +flowchart TD + AP[The Advice Process] + subgraph Mechanisms["The Four Alignment & Learning Mechanisms"] + ADR["1. Lightweight ADRs + Transparent Decision History"] + AAF["2. Architecture Advisory Forum + Open Weekly Conversational Clinic"] + PRIN["3. Team-Sourced Principles + Living Cultural Guardrails"] + RADAR["4. Technology Radar + Lifecycle Tracking: Adopt / Trial / Assess / Hold"] + end + AP <--> ADR + AP <--> AAF + AP <--> PRIN + AP <--> RADAR +``` +[[CAPTION:Figure]] The four alignment mechanisms reinforcing the Advice Process. + +## 1. Lightweight Architecture Decision Records (ADRs) + +Architecture Decision Records ([Writing ADRs and RFCs](../decision-records.md)) capture significant technical choices directly inside the version-controlled codebase. + +### Why ADRs are Critical for Autonomous Teams + +- **Temporal Transparency**: In distributed teams, members inevitably ask: *"Why on earth did we choose SQLite instead of PostgreSQL here?"* Without an ADR, engineers assume past incompetence and rewrite working code. With an ADR, they discover the decision was constrained by a 512 KB microcontroller RAM envelope. +- **Advice Documentation**: The ADR explicitly records who was consulted, what advice was given, and the rationale for accepting or rejecting specific recommendations. +- **Lightweight PR Review**: ADRs are submitted as pull requests. Reviewers comment directly on the Markdown text, creating a verifiable audit trail before merging into the repository `docs/adr/` folder. + +## 2. The Architecture Advisory Forum (ARB) + +The Architecture Advisory Forum transforms the traditional, dreaded Architecture Review Board into an open, collaborative clinic ([Running an architecture review board](../architecture-review-board.md)). + +| Traditional Review Board | Architecture Advisory Forum | +| :--- | :--- | +| Closed committee of senior ivory-tower architects. | Open drop-in clinic attended by developers, tech leads, and specialists. | +| Adversarial: teams "defend" their design against interrogation. | Consultative: teams bring nascent ideas early to solicit constructive advice. | +| Binary verdict: PASS or FAIL gates blocking release pipelines. | Collaborative alignment: refining proposals and connecting disparate squads. | +| Infrequent meetings (monthly); creates delivery bottlenecks. | Frequent, predictable cadence (weekly 60-minute session). | + +[[CAPTION:Table]] Evolution from traditional gatekeeper board to open Advisory Forum. + +Teams use the 23-point Architecture Review Checklist ([Architecture review checklist](../../templates/architecture-review-checklist.md)) not as an administrative barrier, but as a diagnostic checklist to prepare for the session. + +## 3. Team-Sourced Architecture Principles + +Architectural principles cannot be imposed by executive decree. Principles written on laminated posters or hidden in corporate wikis are universally ignored. + +In conversational architecture, principles are **team-sourced**: +- Authored by practicing engineers who live with the consequences of implementation. +- Grounded in real-world engineering fundamentals ([Fundamental Software Architecture Principles](../../standards/software-architecture-principles.md)). +- Regularly reviewed, refined, and retired as organizational maturity evolves. +- Serving as guardrails that guide daily code reviews and pull request discussions. + +## 4. The Technology Radar + +The fourth mechanism is the **Technology Radar**, inspired by ThoughtWorks and curated collaboratively across the engineering estate. + +The radar tracks the maturity lifecycle of technologies across four quadrants and four concentric rings: + +```mermaid +flowchart TD + subgraph Rings["The Four Technology Rings"] + ADOPT["ADOPT: High confidence; standard production choice"] + TRIAL["TRIAL: Low-risk production evaluation in 1-2 squads"] + ASSESS["ASSESS: Research spike, prototyping, POC evaluation"] + HOLD["HOLD: Deprecated; do not use for new projects"] + end + ASSESS --> TRIAL + TRIAL --> ADOPT + ADOPT --> HOLD +``` +[[CAPTION:Figure]] Technology progression across radar rings. + +- **Adopt**: Mature, battle-tested technologies that represent the default production choice (for example, FreeRTOS, PostgreSQL, Docker, C4-PlantUML). +- **Trial**: Promising tools undergoing controlled evaluation in real projects with a documented rollback plan. +- **Assess**: Exploratory spikes to understand emerging paradigms without production risk. +- **Hold**: Deprecated technologies that must not be used for new projects and are actively targeted for decommissioning. + +The complete curated catalogue of 64 technologies is published in the [Personal Technology Radar](../../radar/index.md). + +## Summary: Sustaining Conversational Architecture + +When autonomous squads practice the Advice Process, backed by Lightweight ADRs, an open Advisory Forum, Team-Sourced Principles, and a living Technology Radar, architecture scales gracefully. + +Technical debt is brought into the light, silos dissolve, and the entire engineering organisation marches forward in coherent alignment. diff --git a/content/guides/scaling-architecture/index.md b/content/guides/scaling-architecture/index.md new file mode 100644 index 0000000..d273a98 --- /dev/null +++ b/content/guides/scaling-architecture/index.md @@ -0,0 +1,65 @@ +--- +title: Scaling the practice of architecture +--- + +# Scaling the practice of architecture + +Architecture need not be an authoritarian monologue delivered top-down from a centralized few. This guide describes architecture as a series of structured conversations. They are driven by a decentralised decision-making technique, the **Advice Process**, and anchored by four alignment mechanisms. + +## The Human-Scaling Challenge of Autonomy + +In software delivery environments shifting toward autonomous, value-stream-aligned squads, traditional approaches to software architecture break down. A centralized architecture team attempts to be everywhere at once: reviewing every pull request, attending every stand-up, and approving every framework choice. + +This model inevitably creates one of two pathological outcomes: +1. **The Architecture Bottleneck**: Delivery slows to a crawl as squads wait weeks for scarce architects to schedule reviews and approve design documents. +2. **Architectural Anarchy**: Frustrated squads bypass governance entirely, introducing fragmented technologies, redundant data stores, and incompatible APIs that degrade enterprise coherence. + +```mermaid +flowchart TD + subgraph Traditional["Traditional Centralised Architecture (Bottleneck)"] + ARCH[Central Architecture Committee] + S1[Squad 1] + S2[Squad 2] + S3[Squad 3] + S4[Squad 4] + S1 -->|Awaits Approval| ARCH + S2 -->|Awaits Approval| ARCH + S3 -->|Awaits Approval| ARCH + S4 -->|Awaits Approval| ARCH + end + subgraph Conversational["Conversational Architecture (Autonomous & Aligned)"] + ADV[The Advice Process & 4 Alignment Mechanisms] + CS1[Squad 1: Takes Decisions] <--> ADV + CS2[Squad 2: Takes Decisions] <--> ADV + CS3[Squad 3: Takes Decisions] <--> ADV + CS4[Squad 4: Takes Decisions] <--> ADV + end +``` +[[CAPTION:Figure]] Centralized architectural bottleneck versus decentralized conversational alignment. + +## Core Tenets of Conversational Architecture + +1. **Everyone Can Architect**: Architectural thinking is not the exclusive preserve of an elite title; it is a discipline practiced across all engineering tiers. +2. **Accountability Stays with Implementers**: The team that implements and operates a system MUST own the architectural decisions that shape it. +3. **Decentralized Decision-Making**: Decisions are made rapidly at the edge, constrained by organizational principles and verified through peer advice. +4. **Continuous Shared Understanding**: Alignment is maintained through living conversations, transparent records, and shared visibility. + +## Structure of this Guide + +This guide is organized into two core chapters: + +| Chapter | Title | Key Concepts Covered | +| :--- | :--- | :--- | +| **01** | [The Advice Process](01-advice-process.md) | The rule, the qualifier, seeking out dissent, right-sizing decisions, and scaling accountability. | +| **02** | [The Four Alignment Mechanisms](02-alignment-mechanisms.md) | Lightweight ADRs, Architecture Advisory Forum (ARB), Team-Sourced Principles, and the Technology Radar. | + +[[CAPTION:Table]] Structure of the Scaling Architecture guide. + +## Related Standards and Frameworks + +- [ADR Authoring Guide](../decision-records.md) +- [Architecture Review Board Guide](../architecture-review-board.md) +- [Architecture Review Checklist](../../templates/architecture-review-checklist.md) +- [Fundamental Software Architecture Principles](../../standards/software-architecture-principles.md) +- [Systems and Software Architecture Handbook](../../handbook/index.md) +- [Personal Technology Radar](../../radar/index.md) diff --git a/content/guides/software-design-specification.md b/content/guides/software-design-specification.md new file mode 100644 index 0000000..0033c7a --- /dev/null +++ b/content/guides/software-design-specification.md @@ -0,0 +1,979 @@ +--- +title: Writing a software design specification +--- + +# Writing a software design specification + +A Software Design Specification (SDS) is one of the most important documents produced during a software project. It serves as the definitive technical description of how a system is intended to work, why certain decisions were made, and how various components interact with each other. When maintained well, it forms the foundation for development, testing, onboarding, auditing, and future evolution of the system. + +However, in traditional workflows, SDS documents are often created as Word files or PDFs during early project phases, and then left to gather dust. They become obsolete, disconnected from reality, and difficult to maintain — leading to inconsistencies across design, code, and testing. + +A Docs-as-Code approach fixes this. Instead of static documents, we use markdown, version control (e.g. Git), CI automation, and modular structure to make the SDS a **living artefact**. It stays with the code, evolves with the system, and integrates with your engineering workflows. + +This guide walks through how to structure, write, maintain, and automate an SDS using Docs-as-Code principles, focusing on practical use in embedded, platform, and full-stack software systems. + +## Where the SDS Fits in the Engineering Lifecycle + +The SDS bridges the gap between architectural decision-making and concrete implementation. It complements: + +- RFCs (Requests for Comment) which explore possible changes and solicit feedback +- ADRs (Architecture Decision Records) which capture why decisions were made +- Interface Specifications, source code, test plans, and CI pipelines + +The SDS captures how these decisions are realised in a consistent, understandable, and traceable form. It provides clarity not only for developers, but for stakeholders, testers, maintainers, and even future hires. + +A good SDS doesn’t just list features — it tells the story of how and why the system behaves the way it does. + +> A good SDS is a narrative, not just a checklist — it explains how the system is built and *why* it’s built that way. + +## Example Folder Structure + +Here’s an example structure for storing your SDS as part of a wider docs repository: + +```plaintext +docs/ +├── adr/ # Architectural decision records +├── rfc/ # Requests for comment +├── sds/ # The software design specification +│ ├── index.md +│ ├── architecture.md +│ ├── security.md +│ ├── task-model.md +│ ├── power.md +│ ├── timing.md +│ ├── memory/ +│ │ ├── memory.md # Summary of memory usage: flash, RAM, EEPROM +│ │ ├── nvm-layout.md # Detailed description of config/data NVM +│ ├── updates.md +│ ├── interfaces/ # Internal interfaces +│ │ ├── uart-api.md +│ │ ├── i2c-protocol.md +│ │ ├── nvm-layout.md +│ ├── components/ # Software components/modules +│ │ ├── rtc.md +│ │ ├── lcd.md +│ │ ├── watchdog.md +│ ├── system-modes.md +│ ├── manufacturing.md +│ ├── qa-support.md +│ ├── configuration.md +│ ├── design-patterns.md +│ ├── instrumentation.md +│ ├── limitations.md +│ ├── build-system/ +│ │ ├── build-system.md +│ │ ├── ci-build.md +│ ├── traceability.md +│ ├── glossary.md +│ └── metadata.yaml +├── interfaces/ # External/public APIs +├── diagrams/ +├── scripts/ +``` + +Each file is modular, focused, and reviewed independently. This makes it easier to keep the specification updated and correct. + +## Introduction and Overview (`index.md`) + +This section is the entry point into the SDS and should provide enough background and context to help any reader understand the system being described. It orients new team members, aligns stakeholders, and sets the tone for the level of depth and rigour expected throughout the document. + +Start by introducing the system at a high level. Explain what the system is, what it does, who it's for, and what this document covers. Provide links to related documentation so readers can navigate easily between high-level decisions (ADRs), requirements, and lower-level technical references. + +Consider including: + +- Purpose and Scope: What is the system or component? What does the SDS cover (and not cover)? +- Audience: Who is this document for? Developers? QA? External auditors? +- System Context: Where does this software live — in a product, service, embedded device? +- References: Point to upstream ADRs, RFCs, or requirements. Link to downstream interface specs or test plans. +- System Goals: High-level objectives (e.g. real-time performance, security posture, battery life targets). + +This section should be written in plain English with clarity in mind. Avoid overwhelming detail — the goal is to provide a map and a purpose, not to dive into architecture just yet. + +## Architecture (`architecture.md`) + +The architecture section gives a clear view of how the software system is structured and how it behaves at a high level. It provides both a *static* and *dynamic* understanding of the system: + +- Static: the main building blocks, layers, and data flows +- Dynamic: how those components interact over time (e.g. task lifecycles, signal flows) + +This is often the first place a developer or system integrator will look when trying to understand how things fit together. + +You should include: + +- Visual diagrams: Use layered diagrams (e.g. C4, onion, service interaction) to show how software components are related. Sequence diagrams can help explain runtime behaviours. +- Component layers: Describe how your application, middleware, driver layers, and hardware interact. For example, where does business logic live? What abstracts hardware? +- Runtime interactions: Summarise typical workflows (e.g. "sensor event → processing → transmit via UART"). +- Platform constraints: Note any limitations imposed by the target (e.g. limited memory, no MMU, slow flash writes). +- Security boundaries: Indicate zones of privilege separation, sandboxing, or trust boundaries. + +```mermaid +flowchart TD + App[Application Logic] --> Services[Service Layer] + Services --> Drivers[Drivers & HAL] + Drivers --> Peripherals[Hardware Interfaces] +``` + +> Treat this section as your system's blueprint — not every detail, but enough for someone to understand the structure and find their way around. + +## Security and Security Considerations (`security.md`) + +This section documents the security posture of the software system. It says what threats are considered, how they are mitigated, and what design decisions protect the system and its users. + +Security must be baked into the system from the start, not bolted on at the end. A good SDS explicitly documents the threat model, key assumptions, and concrete mechanisms used to mitigate risks. + +### Security Objectives + +- Protect confidential data stored or transmitted by the device +- Ensure software integrity during updates and boot +- Prevent unauthorised access or misuse of interfaces +- Enable recovery from fault or attack +- Support traceability and audit of changes + +### Threat Model + +Describe the assumptions about potential attackers, their capabilities, and vectors. Define what is in-scope vs out-of-scope. + +#### Example Considerations + +- Physical Access: Does the attacker have physical access to the device? +- Network Exposure: Is the device internet-connected or air-gapped? +- Tampering Scenarios: Battery removal, sensor spoofing, UART sniffing? +- Supply Chain Risks: Malicious firmware injection at factory? +- Replay Attacks: Are command messages signed or timestamped? + +### Boot and Firmware Validation + +Explain how the system ensures that only authorised code is executed: + +- Bootloader Validation: CRCs, digital signatures, hash comparison? +- Key Storage: Where are root-of-trust keys stored? (OTP, secure flash, external secure element) +- Rollback Protection: How are older, vulnerable images prevented from reloading? +- Integrity Checks: Are checks done at boot only, or periodically? + +### Data Protection + +Detail how sensitive data is stored or transmitted: + +- At Rest: Is config/NVM data encrypted? How is key material protected? +- In Transit: Are communications encrypted (TLS, DTLS, AES)? On which links? +- Tamper Evident Storage: Are audit logs or counters protected from erasure? + +> Include how secrets are provisioned — at factory, on first boot, or over-the-air. + +### Interface Hardening + +Document how external and internal interfaces are protected: + +- Authentication/Authorisation: Who can call what APIs or send what commands? +- Rate Limiting: Are repeated attempts (e.g. password, firmware flashing) throttled? +- Error Handling: Are responses generic enough to avoid leaking internal state? +- Interface Disablement: Are debug ports disabled/locked after provisioning? + +### Secure Update Process + +Reiterate or expand on the update mechanism with a focus on security: + +- Manifest Validation: Signature verification before applying? +- Fail-Safe Update: Can update be rolled back or re-attempted without bricking? +- Secure Channels: Is firmware fetched over a trusted path? + +### Logging and Audit + +Capture how events are monitored or logged for later review: + +- Security Logs: Are invalid access attempts or tamper events recorded? +- Forensics: Can retained logs aid post-attack analysis? +- Log Protection: How are logs protected from tampering? + +### Mitigations and Controls + +Summarise protections in place against common embedded threats: + +| Threat | Mitigation | +|--------------------------|------------------------------------------| +| Firmware tampering | Secure boot, signed updates | +| UART console hijack | Passwords, lockout, port disable | +| Flash memory extraction | Encryption, obfuscation, secure storage | +| Replay attacks | Nonces, timestamps, rolling counters | +| Debug access | JTAG/SWD lockout after provisioning | + +### Security Testing + +Describe how the system’s security is validated: + +- Penetration testing (manual or automated) +- Fuzzing tools or static analysis +- Formal review of crypto or secure boot flows + +Mention any compliance standards targeted (e.g. IEC 62443, ENISA guidelines). + +### Maintenance & Response + +Explain how the system stays secure over time: + +- How are CVEs and third-party vulnerabilities tracked? +- How are keys rotated or revoked? +- What’s the process for patching live devices? + +### Documentation and Policy Links + +Reference supporting documents such as: + +- Key management policy +- Secure coding guidelines (e.g. MISRA, CERT) +- Incident response plan +- Vulnerability disclosure process + +> Security must be a living concern — this section should be reviewed regularly and updated alongside changes in the threat landscape or architecture. + +## System Modes (`system-modes.md`) + +Many software systems operate in distinct modes or states, each optimised for a specific set of conditions, use cases, or environmental triggers. Capturing these modes and the rules for transitioning between them is essential for understanding runtime behaviour — especially under error conditions, during startup, or in low-power scenarios. + +This section outlines the operational states of the system and describes how and why transitions occur. It should be used by developers, testers, and system architects to ensure consistent behaviour across all operating conditions. + +Consider including: + +- Defined Modes: Common examples include Boot, Active, Sleep, Maintenance, Recovery, Fault, and Shutdown. +- Triggers and Transitions: What causes a transition? Examples: GPIO wakeup, watchdog timeout, scheduled maintenance, low battery. +- Mode-specific Behaviour: What changes in each mode? Power usage, active tasks, network stack activity, user interface state. +- State Diagrams: These are especially helpful for visualising complex transitions. + +```mermaid +stateDiagram-v2 + [*] --> Boot + Boot --> Active : Initialisation complete + Active --> Sleep : Idle timeout + Sleep --> Active : Interrupt (GPIO/RTC) + Active --> Maintenance : Config change + Active --> Fault : Error detected + Fault --> Recovery : Watchdog reset +``` + +> Use this section to provide a robust mental model of how your system behaves over time. + +## Manufacturing Support (`manufacturing.md`) + +This section describes any software features, modes, or behaviours specifically included to support factory testing, provisioning, or calibration. + +## Why It Matters + +These features are often critical for volume deployment but easily forgotten or misunderstood later. If not documented, changes made for manufacturing (e.g. skipping init steps, ignoring CRCs, bypassing security checks) can leak into field code or be misunderstood in audits. + +## Suggested Contents + +### Manufacturing Modes + +- Does the firmware support a dedicated mode? (e.g. “factory mode”, “golden image boot”) +- How is it triggered? (pin strapping, serial command, fuse setting) + +### Test Points / Interfaces + +- Special I/O exposed for boundary scan, serial diagnostics, or sensor calibration? +- UART/I2C debug protocols used? + +### Provisioning Workflows + +- How are serial numbers, keys, or config data written? +- Describe use of factory fixtures, scripts, or host tools. + +### Bypass Logic + +- Does manufacturing mode skip watchdogs, boot validation, or timers? +- Any backdoors enabled for engineering use (should be removed or gated in release builds)? + +### Logs / Confirmation Output + +- What signals a “pass” vs “fail”? +- Where are results logged — UART, EEPROM, test jig? + +> ⚠️ Be clear about how manufacturing features are locked out, hidden, or disabled in release builds (e.g. `#ifdef MANUFACTURE_MODE` + no release define) + +## QA & Testing Support (`qa-support.md`) + +This section documents special firmware behaviours or interfaces that help with validation, automated testing, or quality gates. + +### Why It’s Important + +These are often the hooks that make factory validation or CI testing practical. Without this, firmware teams forget which debug pins or CLI commands were critical to QA and lose velocity on regression testing. + +### Suggested Contents + +#### Debug/Test CLI or Protocols + +- Are there commands to simulate inputs, dump state, or force transitions? +- Accessible via serial, SWD, CLI shell? + +#### Test Hooks / Mock Modes + +- Can you fake sensor inputs, bypass power checks, etc. for bench testing? +- Support for hardware abstraction that allows mocking? + +#### Assertions, Logs, and Diagnostics + +- Are test assertions enabled? +- Logs routed to test tools or available via postmortem dump? + +#### Test Patterns or Modes + +- Is there a loopback, soak, or burn-in test? +- Stress patterns (e.g. rapid wake/sleep) for endurance checks? + +#### API or Register Access + +- Can testers access internal registers, counters, or performance metrics? +- Memory-mapped or via an RPC/debug protocol? + +#### Integration With QA Tools + +- Jigs or scripts used to validate results? +- Compatibility with TestStand, LabView, or custom Python frameworks? + +## Design Considerations + +This section exists to provide insight into the broader context and trade-offs that shape your system’s design. These considerations often come from business goals, development constraints, or risk management practices — and they help justify why the system is built the way it is. + +Think of it as the foundation layer of your technical rationale. If someone asks "why didn’t you use X?" or "why is it done this way?", this section should help answer that. + +You might want to discuss: + +- Assumptions: What has been assumed about the hardware, users, network environment, third-party components? +- External Dependencies: Include reliance on SDKs, vendor HALs, cloud services, or middleware. +- Coding Standards: Which are followed? (e.g. MISRA C, CERT, AUTOSAR) +- Security Concerns: Discuss expected threat models and mitigations — encrypted channels, secure boot, flash protection. +- Power and Performance Constraints: e.g. CPU utilisation targets, sleep mode constraints, battery sizing. +- Legal and Licensing: GPL risk avoidance, open-source use policies, FIPS compliance. + +> This is the place to capture what influences your design but doesn’t always show up in code. + +## Design Patterns and Anti-Patterns (`design-patterns.md`) + +As your system evolves, recurring architectural or coding practices will emerge. Capturing these as standardised patterns (or anti-patterns) helps your team improve consistency, reduce onboarding time, and avoid costly mistakes. + +This section acts like a miniature design playbook. It highlights what has worked well and what should be avoided — especially in areas like tasking, interface boundaries, or message handling. + +### Recommended Patterns + +- Publish/Subscribe Messaging: Used to decouple components and allow for asynchronous event handling. Useful for telemetry, sensor updates, command dispatch. +- State Machines: Encapsulate mode transitions using modular state patterns, avoiding sprawling switch/case logic. +- Factory or Registry Patterns: To dynamically configure drivers or interfaces based on configuration at boot. +- Command Pattern: Separates logic from actions, helpful in communication protocols and UI abstraction. + +### Known Anti-Patterns + +- God Objects: One module knows and does too much — often seen in monolithic main loops. +- Tight Coupling Across Layers: HALs calling business logic directly, breaking abstraction layers. +- Hidden Global State: Making testing and debugging unpredictable. +- Duplicated Logic Across Tasks: E.g. similar parsing code in both comms and test tasks. + +Where possible, include: + +- Code examples of good vs. bad implementations +- Links to components that exemplify each pattern +- Notes about when a pattern *shouldn’t* be used + +> ⚠️ This section helps encode institutial knowledge, it shoudl evolve over time. + +## Task Model (`task-model.md`) + +In systems using real-time operating systems (RTOS), cooperative scheduling, or even basic super loops, understanding how software is structured into tasks or threads is crucial. This section provides an overview of all concurrent execution contexts, their responsibilities, timing expectations, and interactions with other tasks or interrupts. + +Well-documented task models help avoid timing conflicts, resource contention, and make it easier for new developers to reason about system flow. + +What to include: + +- Task List: Describe each task's responsibility, its period or trigger mechanism, and its priority. +- Stack Size: If stack usage is measured or estimated, note how much each task needs — especially for memory-constrained targets. +- Event Flow: Describe how ISRs signal or wake tasks (e.g. semaphores, message queues). +- Diagrams: Timeline or interaction charts can help visualise timing and sequence of events. + +| Task Name | Priority | Trigger | Stack | Description | +| ---------- | -------- | ----------- | ----- | ------------------------------- | +| SensorTask | High | Timer | 1kB | Polls sensors every 100ms | +| CommTask | Medium | Event queue | 2kB | Handles all communication flows | +| UITask | Low | Button IRQ | 1kB | Manages display and UI updates | + +> 💡 Documenting stack size and priority together makes it easier to identify task starvation or overflow risk during later reviews or audits. + +## Concurrency and Synchronisation + +Multithreaded or multitasking systems require careful coordination to avoid race conditions, deadlocks, or corruption of shared data. This section captures the strategy for how tasks and ISRs interact safely and predictably. + +Describe the mechanisms used for synchronisation and inter-task communication. For example, which tasks publish or consume events? What locks are used, and where? How is starvation or blocking handled? + +Include the following if applicable: + +- Mutexes, Semaphores, Queues: Which ones are used, and in what situations? +- Priority Inversion Mitigation: Does the RTOS support priority inheritance? Are there software strategies in place? +- Deadlock Avoidance: Document timeout strategies or watchdog recovery paths. +- Non-blocking Strategies: In some systems, blocking on shared resources is avoided altogether — capture this as a design choice. + +> ⚠️ This section is often overlooked but becomes critical when diagnosing hard-to-reproduce bugs or unexpected lockups in production. + +## Timing (`timing.md`) + +Time constraints are common in embedded and real-time systems. Whether you’re reading sensors, driving a user interface, or meeting external communication deadlines, understanding your timing model is essential. + +This section documents which tasks or interrupts are time-sensitive and what their constraints are. + +Consider describing: + +- ISR Deadlines: How fast must interrupts complete? What work is deferred? +- Task Budgeting: Expected worst-case execution time for each major task. +- System Tick and Timebase: Describe the base timer, tick rate, and how time is kept. +- Time Sync Mechanisms: Use of RTC, GPS, NTP, or synchronisation with external clocks. + +```yaml +timing-budget: + control-loop: 10ms + display-refresh: 20ms + uart-isr: 2µs +``` + +> ⏲ This information is especially useful when validating designs against real-time constraints or reviewing logs from performance instrumentation. + +## Power (`power.md`) + +For embedded or battery-powered systems, power consumption is often one of the most critical design constraints. The goal of this section is to document how your system manages energy usage across different operating conditions and modes. + +This is particularly important for systems where uptime, efficiency, or thermal limits are affected by energy draw. It also helps verify that your design aligns with battery sizing, regulator capabilities, and customer expectations for device longevity. + +Suggested contents: + +- Supported Low-Power Modes: Enumerate the power states supported by your MCU or platform (e.g. RUN, SLEEP, STOP, STANDBY). Include any specific characteristics (e.g. which peripherals remain active). +- Wakeup Sources and Rules: Describe how the system transitions out of low-power modes — for example, GPIO interrupt, timer, RTC alarm, watchdog. +- Debounce and Filtering: If wake sources involve mechanical switches or user input, include debounce timing strategies to avoid false wakeups. +- Power Budget Estimates: If available, provide power draw figures in each state (measured or estimated), with context for how often the system enters them. +- Peripheral Power Control: Describe how and when peripherals are powered down or gated. E.g. turning off UART transceivers, putting sensors into standby. + +> ⚡ Include a state diagram or table if your system uses multiple modes with complex transitions. + +## Interfaces (`interfaces/`) + +This section documents how software interacts with hardware peripherals or internal drivers. For each interface, provide a dedicated markdown file that describes the protocol or API in full. This helps maintain a clear contract between the software layers and supports portability, testability, and modular design. + +You should break this section into one file per interface. For example: + +- `uart-api.md`: Describes the UART driver API. Include init/config calls, transmit/receive flows, DMA usage (if any), and any line-discipline logic (e.g. software flow control, newline translation). +- `i2c-protocol.md`: Documents I2C commands to and from a specific device. Include register layout, timing requirements, expected acknowledgements, and examples. + +Each interface file should typically include: + +- Overview: Purpose of the interface, what it controls or monitors +- API Summary or Command Set: Function prototypes or I2C/SPI commands +- Parameters: Buffer sizes, error codes, retry logic, blocking/non-blocking behaviour +- Data Structures or Encodings: Register maps, bitfields, enums +- Diagrams: State transitions or bus timing diagrams if relevant +- Examples: Sample usage code or communication trace + +> 📂 You may also include an `interfaces.md` index file if you have many interfaces. It can list and link to each subfile with a short summary, but it’s optional if your structure is already clear from folder organisation. + +## Memory (`memory.md`) + +This section defines how your software uses memory, particularly when there are fixed constraints or persistent data to manage. In embedded systems, memory layout is often tightly coupled to the firmware image and must be coordinated with bootloaders, config storage, or debug tools. + +Document both volatile and non-volatile memory use: + +- Flash Layout: Include memory map showing regions for bootloader, application firmware, configuration pages, user data, and diagnostic logs. If dual-image or failover schemes are used, detail the image switching mechanism. +- RAM Usage Strategy: Outline how RAM is allocated — statically, dynamically, or per-task. Mention stack sizes and heap if applicable. +- Emulated EEPROM or NVM: Describe how configuration data is stored and updated, including wear-leveling, sector rotation, and CRC use. +- Persistent Variables: Note any variables that retain values across resets, using backup registers, battery-backed RAM, or flash. + +```plaintext +0x08000000 - 0x08003FFF : Bootloader (16 KB) +0x08004000 - 0x0801FFFF : App Firmware (96 KB) +0x08020000 - 0x08027FFF : Configuration Data (32 KB) +0x20000000 - 0x20002000 : RAM (8 KB) +``` + +### `nvm-layout.md` + +In addition to the general memory section, a separate file can be created to describe the layout and management of non-volatile memory (NVM). This is useful when configuration or runtime data is stored in flash and managed dynamically. + +Suggested contents: + +- Storage Regions: Define sectors or pages allocated to config, counters, logs, etc. +- Versioning Scheme: Explain how you track versions of stored data, particularly during firmware upgrades. +- Integrity Checks: CRCs, checksums, or signature strategies used to validate contents. +- Update Strategies: Whether you use append-only, copy-on-write, sector rotation, or journaling. +- Error Handling: What happens if a CRC fails or a config is corrupt? Describe the fallback or reset logic. +- Tool Compatibility: Link to any scripts or debug tools that can parse or inspect this memory structure. + +> 📎 Keep this file aligned with memory maps, linker scripts, and config handling logic. + +## Updates (`updates.md`) + +This section describes how the firmware or software running on the device can be updated. Depending on your system architecture, this could include Over-the-Air (OTA) updates, updates via USB, UART flashing, or during production line programming. It should clearly describe what gets updated, how integrity is validated, and what the fallback strategies are in case of partial or failed updates. + +Include the following details: + +- Update Mechanism: Describe supported mechanisms such as OTA (e.g. via LwM2M or MQTT), serial bootloader tools, or SD card updates. +- Bootloader and Validation: + - Does the bootloader validate firmware via CRC, hash, or digital signature? + - What happens on validation failure? + - Include a brief flow: reset → bootloader → validation → app launch. +- Dual-Image Strategy: + - If supported, describe how image switching works. + - Document active vs backup image location. + - Explain the conditions under which rollback happens. +- Update Triggering: + - Who or what initiates an update? (e.g. remote server command, physical button, watchdog failover) + - How is update progress reported? +- Metadata & Versioning: + - Is there a manifest file, checksum table, or version block? + - How is the running firmware version exposed to external systems? + +> 🧪 Include test cases or validation flows to ensure reliability. A diagram or sequence chart can also help visualise the update lifecycle. + +## Components (`components/`) + +This directory contains one markdown file per key software component or module. This helps keep the design documentation modular and scalable. Each file provides the rationale, API summary, interactions, and special considerations for that module. + +Each component file should include: + +- Name and Purpose: What is this component for? +- Key Functions or Interfaces: Public APIs exposed by this component. +- Dependencies: What peripherals or other modules does it rely on? +- Startup/Shutdown Behaviour: What must be initialised? What resets state? +- Resource Usage: Stack size, RAM footprint, exclusive hardware access. +- Constraints and Limitations: Known edge cases or design trade-offs. + +Example: `rtc.md` + +```markdown +## RTC Module + +The RTC module handles timekeeping for the system and provides alarms for timed events. + +### Public API +- `rtc_init()` — Initialise the clock domain +- `rtc_get_time()` — Read current RTC value +- `rtc_set_time()` — Set RTC value +- `rtc_set_alarm()` — Configure future wakeup event + +### Uses +- LSE oscillator (32.768kHz) +- Backup registers for retention across resets + +### Constraints +- Drift may occur during STOP mode without calibration +- Cannot use without LSE availability +``` + +> 📦 Use consistent headers and formatting across all component docs for easier review. + +## Build System (`build-system.md`) + +This section documents how the software is built — what toolchains and configurations are supported, how outputs are structured, and what metadata should be captured to support traceability and CI integration. + +A well-documented build system ensures reproducibility, supports automation, and allows future engineers to rebuild or audit the firmware with confidence. + +### Supported Build Systems + +Describe how the software can be built across environments and by different roles — e.g. developer, CI agent, or release engineer. + +The SDS should explain which build systems are supported and how they interrelate. Typically, this includes: + +- `make` or similar build orchestration for local and CI use +- Generator-based build systems (e.g. `cmake`) if supporting multiple IDEs or cross-platform builds +- IDE-native builds (e.g. IAR project files) for deeply embedded targets +- Scripted or container-based builds for portable, isolated environments + +If multiple systems are supported, note which is canonical. + +### Toolchains + +Capture the expected compiler and toolchain setup used by developers and CI systems. + +Mention whether the toolchain is installed manually, managed by package manager, or included via container. The SDS should list supported toolchains (e.g. GCC, LLVM, IAR), any minimum version requirements, and a pointer to a `.toolchain` file or Docker image if used. + +### Build Configurations + +Define the available build variants, their intended use, and any functional differences. + +Typical configurations include: + +- Debug – enables diagnostics, logs, and debug symbols for development +- Release – optimised firmware, stripped of debugging features, suitable for deployment +- Test – includes mocking, test scaffolding, and additional hooks + +Explain how these configurations affect preprocessor defines, output structure, and included modules. This helps others understand which build type is suitable for specific tasks like flashing, validation, or lab testing. + +### Feature Flags and Defines + +List the key compile-time flags or feature switches used across the system. + +A well-maintained SDS should explain where these are declared (e.g. `build_flags.h`, `CMakeLists.txt`) and what they control — such as optional hardware interfaces, debug modes, or platform-specific adaptations. + +Flags should be consistently named and grouped logically to aid discoverability and maintainability. + +### Output Artefacts + +Detail what artefacts are produced by the build process and where they are placed. + +These commonly include: +- Executables (`.elf`, `.out`) +- Map files (`.map`), listings, and symbol tables +- Optional: memory reports, coverage files, metrics + +The SDS should state how these files are named and structured, especially if they're used in flashing, testing, or deployment workflows. + +### Build Metadata + +Describe the metadata embedded or exported by the build to support traceability and debugging. + +This may include: + +- Git commit, branch, and tag +- Build timestamp +- Build configuration and feature flags +- Toolchain ID or version string + +This metadata should be stored in a header file (`version.h`), embedded in flash, and optionally exported in a structured format (`build.json`) for integration with dashboards or CI pipelines. + +### Firmware Packaging + +Outline how formal release builds are bundled and versioned. + +The SDS should document how release artefacts are packaged for handoff — including: + +- Firmware binaries and supporting files +- Associated map and metadata files +- Optional checksums, signatures, or version manifests +- Directory naming and retention practices (e.g. `/dist/firmware-v1.2.3/`) + +This supports reliable delivery, testing, and rollback of firmware. + +### CI Integration (`ci-build.md`) + +This section documents the expectations for continuous integration (CI) pipelines that validate the design, enforce quality, and protect against regressions. + +A well-integrated CI process ensures every change is verifiable, traceable, and safe to merge. The SDS should make clear what is checked, how failures are detected, and which rules are enforced before code reaches production. + +#### CI Validates + +Summarise what CI pipelines are expected to do. + +A typical embedded software CI system should: + +- Build all configurations (Debug, Release, Test) +- Run static analysis on source code +- Execute unit or integration tests where applicable +- Enforce style and formatting rules +- Validate SDS structure and metadata +- Track binary sizes and alert on unexpected growth + +Mention if different CI stages are split across jobs or merged into a single runner. + +#### Code Quality Metrics + +Define which quality metrics are gathered to evaluate changes over time. + +A good SDS should specify that CI calculates and monitors: + +- Cyclomatic complexity +- Halstead metrics +- Function and file sizes +- Nesting/indentation depth +- Code churn and comment density + +The metrics system should highlight regressions and identify hotspots that may need refactoring. Detail thresholds if they exist. + +#### Binary Size Regression + +The SDS should define how binary size growth is detected and reported. + +CI must compare the size of key artefacts (e.g. `.elf`, `.bin`) to a known baseline and flag regressions. This protects against creeping feature growth, unexpected linker behaviour, or accidental inclusion of unused features. + +Summarise how size trends are tracked (e.g. via diff or dashboard), and when the build should fail. + +#### Incremental Checks + +Document how CI limits scope to affected areas to optimise feedback loops. + +The SDS should note that: + +- Static analysis and linting focus on changed files +- Code metrics and tests are limited to touched modules when possible +- Full system analysis is run nightly or before release + +This keeps feedback fast while still catching critical issues. + +#### Feedback & Merge Protection + +Explain how developers are informed of issues and how quality is enforced. + +CI pipelines should: + +- Post warnings or diffs directly into pull requests +- Annotate files or lines that fail checks +- Block merge if thresholds are exceeded, or key checks fail + +Summarise your merge rules — e.g. static analysis clean, no binary size regression, all tests pass, metadata validated. + +#### Optional Enhancements + +Capture any additional checks required for regulatory, audit, or public release workflows. + +The SDS may include optional CI jobs for: + +- Secret scanning (e.g. env or config leaks) +- Software Bill of Materials (SBOM) generation +- Open source licence validation +- Dependency vulnerability alerts +- Traceability metadata checks (e.g. ADRs linked to features or tests) + +Mention these if your process supports them — or leave as future improvements. + +> 📎 If detailed CI configuration (e.g. job scripts, badge configuration, security policies) is maintained separately, link to a `ci-extras.md` or `.github/workflows/` folder for deeper implementation detail. + +## Instrumentation (`instrumentation.md`) + +Instrumentation allows developers and testers to observe, measure, and validate the behaviour of a running system. Whether it’s to diagnose issues, evaluate performance, or confirm that a state transition occurred as expected, a well-instrumented system offers transparency without requiring invasive access. + +This section should explain how observability is achieved, which tools or techniques are in use, and what runtime flags or modes exist to toggle diagnostic output. + +Recommended contents: + +- Logging Strategy: + - What logging framework is used? (e.g. macros, circular buffers, conditional printfs) + - Describe log levels: `DEBUG`, `INFO`, `WARN`, `ERROR` + - Format: plain text, JSON, binary packets? + - Where does logging go? (UART, memory buffer, external tool?) +- Conditional Compilation: + - Are there flags like `ENABLE_LOGGING`, `DEBUG_MODE`, or `VERBOSE_UART`? + - What impact do they have on binary size or performance? +- Runtime Controls: + - Can instrumentation be enabled/disabled on the fly (e.g. via CLI or config register)? + - Describe diagnostic commands or debug shells (if any) +- Fault Capture: + - What happens on assert or fault? (e.g. stack dump, PC/SP snapshot, reset reason) + - Persistent logs in retained memory or NVM? + - Use of semihosting, debug probes, or test firmware overlays + +> 🧠 Consider documenting how to replay or interpret logs — especially if they use custom formats. + +## Limitations (`limitations.md`) + +This section is a place for honest reflection. It acknowledges areas where the system isn’t perfect — yet. Capturing technical debt, design shortcuts, or areas needing future refactor helps maintain team awareness and supports informed planning. + +You should aim to answer: “What’s broken, missing, or non-ideal — and what are we going to do about it?” + +Include the following where relevant: + +- Unimplemented or Deferred Features: + - APIs that exist in stub form + - Tasks or behaviours described elsewhere but not yet complete +- Known Weaknesses or Risks: + - Use of dynamic memory in ISR context + - Shared global state with minimal locking + - Reliance on outdated or unstable third-party libraries +- Legacy Components: + - Code that is known to be hard to test, slow, or buggy + - Modules earmarked for future replacement +- Refactoring Plans or Proposals: + - Links to backlog items, tickets, or internal RFCs for cleanup + - Prioritisation of technical debt relative to business features + +> ⚠️ Be realistic — don’t list *everything* that could be better, just what is known and likely to impact the system. + +## Traceability (`traceability.md`) + +Traceability ensures your design isn’t floating in isolation — it ties every part of the system to the rationale behind it and the tests that verify it. This is especially valuable in safety-critical, audited, or regulated environments — but it also helps with long-term maintainability in any serious project. + +The goal is to allow someone to trace from: + +- A feature or bug → to the design decision → to the implementation → to the validation + +Document: + +- Linked ADRs: + - Reference architecture decisions that led to this design (e.g. “Use MQTT over CoAP”) +- Linked RFCs: + - Proposals or exploration work that evaluated options (e.g. “rfc-006-iot-messaging.md”) +- Associated Tests: + - Unit, integration, or system tests that verify correctness + - Use consistent test names or IDs to simplify automation +- Metadata Tags: + - Use per-section frontmatter or YAML blocks to define traceability metadata + +Example: + +```yaml +owner: "comm-stack" +linked-adr: ["adr-014-mqtt-integration"] +linked-rfc: ["rfc-006-mqtt-vs-coap"] +linked-tests: + - test_mqtt_reconnect + - test_power_on_retry +last-reviewed: "2025-05-01" +``` + +> 🔍 Good traceability gives confidence during change reviews. It answers “why is this here?”, “who owns it?”, and “how do we know it works?” without needing tribal knowledge. + +## Exporting to PDF or Word for SharePoint + +Although the SDS should be maintained as markdown in version control, there are often cases where you need a formal, static version — for example: + +- Regulatory audits or compliance documentation +- Certification processes +- Customer deliverables +- SharePoint document repositories +- Project release cutoffs + +This section explains how to generate exportable formats (PDF, Word, or HTML) from your markdown-based SDS using tools like **Pandoc** or **MkDocs**. These exports should be versioned and traceable to a specific tagged release or commit to ensure clarity and accountability. + +### Export via MkDocs + +For high-quality, structured exports that include styles, branding, and proper navigation, **MkDocs** (with Material theme and `with-pdf` plugin) is highly recommended. + +MkDocs also supports rendering **inline Mermaid diagrams**, something Pandoc cannot easily do. If your SDS includes architecture diagrams in Mermaid syntax, MkDocs ensures these are fully rendered in the PDF export. + +#### Example MkDocs Configuration + +Here’s a real-world example of an SDS-friendly `mkdocs.yml` setup: + +```yaml +site_name: SDS Documentation +site_author: Engineering Team +site_description: Version-controlled Software Design Spec +copyright: © 2025 Engineering Group + +docs_dir: docs/sds + +theme: + name: material + palette: + - scheme: default + toggle: + icon: material/brightness-7 + name: Switch to dark mode + primary: indigo + accent: pink + - scheme: slate + toggle: + icon: material/brightness-4 + name: Switch to light mode + primary: indigo + accent: pink + features: + - navigation.sections + - navigation.expand + - navigation.top + - toc.integrate + - search.highlight + +markdown_extensions: + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format + - toc: + permalink: true + - codehilite + - admonition + - plantuml_markdown: + server: https://www.plantuml.com/plantuml/svg/ + +extra_javascript: + - https://unpkg.com/mermaid@10.4.0/dist/mermaid.min.js + +plugins: + - gen_nav: + enabled: true + - search + - with-pdf: + cover: true + cover_title: "Software Design Specification" + cover_subtitle: "Auto-generated PDF version" + cover_logo: "assets/cover.png" +``` + +#### Export Commands + +To export the entire SDS as a styled PDF: + +```bash +pip install mkdocs mkdocs-material mkdocs-with-pdf +mkdocs build +mkdocs with-pdf export +``` + +### Alternative: Pandoc + +If you prefer Pandoc (e.g. for lightweight pipelines): + +```bash +pandoc sds/index.md -o SoftwareDesignSpec_v1.2.pdf --from markdown --pdf-engine=xelatex +pandoc sds/index.md -o SoftwareDesignSpec_v1.2.docx +``` + +> 📌 Note: Pandoc **cannot render inline Mermaid diagrams**. If you use these, MkDocs is the better choice. + +### CI Integration Example + +```yaml +- name: Install MkDocs dependencies + run: | + pip install mkdocs mkdocs-material mkdocs-with-pdf + +- name: Export SDS as PDF using MkDocs + run: | + mkdocs build + mkdocs with-pdf export +``` + +Alternatively, for Pandoc: + +```yaml +- name: Export SDS for release using Pandoc + if: github.ref == 'refs/tags/v1.2.0' + run: | + pandoc docs/sds/index.md -o release-artifacts/SDS_v1.2.0.pdf +``` + +> ✨ Whatever tooling you choose, take time to learn it well. Automate as much as you can, and keep it elegant — a polished export process makes your SDS as credible and useful as your code. +> 🚧 Recommendation: Choose **MkDocs** if you rely on diagrams and advanced layout, or **Pandoc** if simplicity and minimal dependencies are key. + +## CI Integration + +To make sure the SDS stays accurate and usable, treat it like code. That means validating its structure, checking metadata, linting markdown formatting, and optionally exporting versioned artefacts. + +Below is a sample CI pipeline that can be included in GitHub Actions, GitLab CI, or any other CI platform: + +```yaml +- name: Validate metadata and frontmatter + run: python scripts/validate_metadata.py + +- name: Check markdown syntax + run: markdownlint docs/sds/**/*.md + +- name: Export SDS to PDF + run: pandoc docs/sds/index.md -o docs/SoftwareDesignSpec.pdf +``` + +You might also add: + +- Spellcheck +- Broken link checker +- Diagram generation from `.mmd` or `.puml` files +- Automatic reviewers for docs-only PRs + +> ⚙️ The goal is to embed SDS health checks into the development lifecycle, ensuring the doc is always ready to ship. + +## Summary of Benefits + +| Docs-as-Code Practice | What it Helps With | +| -------------------------- | ------------------------------------ | +| Modular files | Focused ownership, easier updates | +| Traceable metadata | Clear links to decisions and tests | +| CI integration | Avoids rot, ensures freshness | +| Export to PDF/Word | Works with formal approval workflows | +| Patterns and anti-patterns | Onboarding, review, and consistency | + +A well-structured SDS, maintained using Docs-as-Code, bridges the gap between design and delivery. It gives teams confidence, improves collaboration, and becomes a trusted reference throughout the lifecycle of a system. + +## Final Thoughts + +A well-written SDS helps your team understand the system, justify decisions, and scale the design. When treated as code, it becomes part of your engineering DNA — continuously maintained, traceable, and aligned with the system itself. Build it once, evolve it forever. diff --git a/content/guides/ways-of-working/01-working-agreements.md b/content/guides/ways-of-working/01-working-agreements.md new file mode 100644 index 0000000..4d4f1d8 --- /dev/null +++ b/content/guides/ways-of-working/01-working-agreements.md @@ -0,0 +1,173 @@ +--- +title: Working agreements and team norms +--- + +--- + +title: Working Agreements & Team Norms +status: draft +version: 0.1 +owner: Engineering +last_updated: 2026-01-05 +------------------------ + +# Working agreements and team norms + +This guide defines how we work together day to day. + +Working agreements are explicit, shared expectations about behaviour, communication, and decision-making. They exist to reduce friction, ambiguity, and unspoken assumptions. They are not rules for control, but guardrails that help the team move faster with less stress. + +These agreements apply to everyone equally, regardless of role or seniority. + +## Purpose + +Our working agreements aim to: + +* Create psychological safety and trust +* Make collaboration predictable and fair +* Reduce coordination overhead +* Avoid reliance on heroics or tribal knowledge +* Enable sustainable delivery over time + +If an agreement no longer serves these goals, it should be challenged and changed. + +## Core principles + +* Assume positive intent +* Optimise for team outcomes, not individual convenience +* Make work visible +* Prefer clarity over politeness when trade-offs matter +* Disagree openly, commit clearly + +## Communication norms + +### Default channels + +* Async first where possible +* Written communication preferred for decisions and technical detail +* Meetings are for discussion, alignment, and decision-making, not information broadcast + +### Response expectations + +* Acknowledge messages within one working day, even if the full response comes later +* Say when you are blocked or overloaded +* Silence is not agreement + +### Tone + +* Direct, respectful, and factual +* Critique ideas and behaviours, not people +* Avoid sarcasm in written communication +* Emojis are acceptable if they add clarity, not pressure + +## Meetings + +### General rules + +* Every meeting has an owner, agenda, and intended outcome +* Start on time, end on time +* Decline meetings that do not require your presence +* Capture decisions and actions in writing + +### Typical meeting types + +* Stand-up: coordination and blockers only +* Planning: scope, priority, and trade-offs +* Refinement: clarity and readiness, not commitment +* Review/demo: learning and feedback +* Retrospective: improvement, not blame + +```mermaid +flowchart TD + A[Meeting proposed] --> B{Clear purpose} + B -- No --> C[Do not schedule] + B -- Yes --> D{Agenda shared} + D -- No --> E[Fix agenda first] + D -- Yes --> F[Meeting runs] + F --> G[Decisions and actions recorded] +``` + +## Availability and focus time + +* Core collaboration hours are agreed by the team +* Outside those hours, responses are best-effort +* Use calendar blocks or status messages to protect focus time +* Deep work is respected; interruptions should be intentional + +## Decision-making + +### Decision ownership + +* Decisions should have a clear owner +* Input is encouraged from those affected or with relevant expertise +* Not all decisions require consensus + +### Decision model + +* Reversible decisions: bias for speed +* Irreversible or high-impact decisions: bias for alignment and documentation + +```mermaid +flowchart TD + A[Decision needed] --> B{Reversible} + B -- Yes --> C[Decide quickly] + B -- No --> D[Gather input] + D --> E[Document decision] + E --> F[Decide and communicate] +``` + +## Disagreement and conflict + +* Disagreement is expected and healthy +* Raise concerns early, not after the fact +* Use data, examples, and impact rather than opinion alone +* If consensus cannot be reached, the decision owner decides +* Once a decision is made, we commit and move forward + +## Feedback + +### Giving feedback + +* Be timely +* Be specific and actionable +* Focus on behaviour and impact +* Give feedback in private when personal, in public when it benefits the team + +### Receiving feedback + +* Listen to understand, not to defend +* Ask clarifying questions +* Treat feedback as information, not judgement + +## Ownership and accountability + +* If you pick up work, you own it end to end +* Raise risks and blockers early +* Do not let work silently stall +* Ask for help before you are stuck + +Accountability is about reliability, not blame. + +## Working with code and change + +* Small, incremental changes preferred +* Work is reviewed before merging +* Quality issues are addressed, not deferred indefinitely +* Broken builds are treated as urgent + +(See Code Review Guide and CI/CD Guide for specifics.) + +## Learning and improvement + +* Retrospectives are blameless +* Actions from retros are tracked +* We share learnings openly, including mistakes +* Continuous improvement is part of normal work, not an extra task + + +## Exceptions and evolution + +* These agreements are defaults, not rigid rules +* Exceptions are allowed when explicit and agreed +* This document should be reviewed periodically +* Changes require team discussion and agreement diff --git a/content/guides/ways-of-working/02-onboarding.md b/content/guides/ways-of-working/02-onboarding.md new file mode 100644 index 0000000..2fcc49f --- /dev/null +++ b/content/guides/ways-of-working/02-onboarding.md @@ -0,0 +1,215 @@ +--- +title: Onboarding engineers +--- + +--- + +title: Engineering Onboarding Guide & Playbook +status: draft +version: 0.1 +owner: Engineering +last_updated: 2026-01-05 +------------------------ + +# Onboarding engineers + +This guide exists to help new team members become effective quickly, safely, and confidently. + +Onboarding is not a single event or a checklist to rush through. It is a supported transition from outsider to trusted contributor, with clarity about expectations, systems, and ways of working. + +This playbook defines what *good onboarding* looks like for both the new starter and the team. + +## Objectives + +Effective onboarding should: + +* Reduce time to first meaningful contribution +* Build understanding of the system, not just the code +* Make expectations explicit +* Avoid reliance on tribal knowledge +* Establish trust and psychological safety early + +## Shared responsibility + +Onboarding is a team responsibility. + +### The new starter owns + +* Asking questions early and often +* Following the documented setup and guides +* Making progress visible + +### The team owns + +* Providing time, context, and support +* Answering questions without judgement +* Keeping onboarding documentation accurate + +### The manager owns + +* Clear goals and priorities +* Regular check-ins +* Removing blockers + +## Before day one + +The following should be ready before the new starter arrives: + +* Laptop and required hardware provisioned +* Accounts created (source control, CI, ticketing, documentation) +* Access to repositories and environments +* Onboarding buddy assigned +* Calendar invites for key ceremonies + +If any of these are missing, fix that first. First impressions matter. + +## First day focus + +Day one is about orientation, not productivity. + +### Expected outcomes + +* Knows who is who and how to get help +* Can access core systems +* Understands what the team does and why +* Feels welcome and supported + +### Do not expect + +* Code contributions +* Deep technical understanding +* Full context retention + +## First week focus + +The goal of the first week is *context building*. + +* High-level system overview +* Walkthrough of repositories and services +* Review of Ways of Working guides +* Environment setup completed +* Shadowing reviews, stand-ups, and planning + +```mermaid +flowchart TD + A[Start onboarding] --> B[System overview] + B --> C[Repo and tooling walkthrough] + C --> D[Ways of Working guides] + D --> E[Environment setup] + E --> F[Shadow team ceremonies] +``` + +## Tooling and access checklist + +A new starter should be able to: + +* Clone repositories +* Build and run the system locally (where applicable) +* Run tests +* Raise a pull request +* View CI results +* Access documentation and diagrams + +Any failure here is an onboarding defect, not a personal one. + +## Ways of Working orientation + +Every new starter should be explicitly pointed at: + +* Process guide +* Code Review Guide +* Working Agreements & Team Norms +* Git and branching strategy +* CI/CD expectations +* Documentation and modelling toolchain + +Do not assume prior knowledge, even for experienced engineers. + +## First contribution + +The first contribution should be: + +* Small +* Low risk +* Clearly scoped +* Reviewed with extra care and explanation + +Examples: + +* Documentation fix +* Test improvement +* Minor refactor +* Small bug fix + +The goal is learning the system and workflow, not proving competence. + +```mermaid +flowchart TD + A[Identify starter task] --> B[Clarify scope and context] + B --> C[Implement change] + C --> D[Open pull request] + D --> E[Supportive review] + E --> F[Merge and celebrate] +``` + +## Onboarding buddy role + +The buddy is not a manager. + +They are responsible for: + +* Being the first point of contact +* Explaining how things really work +* Helping navigate unwritten conventions +* Checking in regularly during the first few weeks + +This role should be explicit and time-boxed. + +## 30 / 60 / 90 day expectations + +### First 30 days + +* Comfortable with tooling and workflow +* Understands system at a high level +* Has made several small contributions + +### First 60 days + +* Owns small pieces of work end to end +* Participates actively in reviews and discussions +* Understands key architectural decisions + +### First 90 days + +* Delivers work with minimal supervision +* Identifies improvement opportunities +* Acts as a full team member + +## Check-ins and feedback + +Regular check-ins should cover: + +* What is clear +* What is confusing +* Where documentation or process failed +* What support is missing + +Feedback from onboarding should feed back into improving this guide. + +## Common failure modes + +Watch for: + +* Overloading with information +* Assuming prior context +* Lack of clear starter tasks +* Silent struggling +* Treating onboarding as complete after week one + +If you see these, intervene early. + +## Evolution of this guide + +* This playbook is a living document +* Changes should be proposed via pull request +* Feedback from new starters is prioritised diff --git a/content/guides/ways-of-working/03-development-process.md b/content/guides/ways-of-working/03-development-process.md new file mode 100644 index 0000000..2bfe47e --- /dev/null +++ b/content/guides/ways-of-working/03-development-process.md @@ -0,0 +1,553 @@ +--- +title: The development process, end to end +--- + +# The development process, end to end + +This document describes an end-to-end process flow for a software development team. +It covers **requirements definition**, **feature creation**, **sprint development**, **code quality gates**, **test & verification**, and **system integration testing** — with **feedback loops at every stage**. + +--- + +## Process Overview + +The overall process follows these steps: +1. **Requirements** are created by the Product Manager and reviewed. +2. **Features** are defined by the Architect, reviewed by the team, and accepted. +3. The software team refines features into **user stories**. +4. Delivery happens in **3-week sprints** using feature branches. +5. Before merging a feature branch: + - All stories must be complete. + - The build must compile and pass unit tests and static analysis. + - Code review must be approved. +6. Once merged, a **build is created for T&V**. +7. **Test & Verification (T&V)** and **System Integration Testing (SIT)** provide structured feedback loops. +8. Rework can happen **at any stage** based on review findings, bugs, or changes in upstream requirements. + +--- + +## High-level Flow (Mermaid) + +```mermaid +flowchart TD + PM[Product Manager creates or updates requirements] --> RvwReq[Requirements review] + RvwReq -->|Accepted| Arch[Architect defines features] + RvwReq -->|Changes needed| PM + + Arch --> RvwFeat[Feature refinement & acceptance] + RvwFeat -->|Accepted| Refine[SW team refines features into user stories] + RvwFeat -->|Rework| Arch + + Refine --> Sprint["Sprint cycles (3 weeks)"] + Sprint --> Dev[Develop on feature branches] + Dev --> Gate{Feature merge gate\n- All stories complete\n- Build compiles\n- Tests pass\n- Static analysis clean\n- Code review OK} + Gate -->|Pass| Merge[Merge to main] + Gate -->|Fail| Rework1[Rework dev or tests] + Merge --> Build[Create build for Test & Verification] + Build --> TnV[Test & Verification] + TnV -->|Bugs found| Rework2[Defect fix & retest] + TnV -->|Pass| SIT[System Integration Testing] + SIT -->|Bugs found| Rework3[Fix SIT issues & retest] + SIT -->|Pass| Done[Feature accepted / ready for release] + + Rework1 --> Sprint + Rework2 --> Sprint + Rework3 --> Sprint +``` + +--- + +## Responsibilities & Hand-offs (PlantUML Activity) + +```mermaid +flowchart TD + + %% ===== Nodes ===== + pm_create["PM: Create or update requirements"] + pm_submit["PM: Submit for review"] + + arch_review["Architect: Review requirements"] + arch_clear{"Architect: Requirements clear & acceptable?"} + pm_revise["PM: Revise requirements"] + pm_signoff["PM: Sign-off requirements"] + + arch_define["Architect: Define features from signed-off requirements"] + pm_feat_review["PM: Review feature"] + sw_feat_review["SW Team: Review feature (Developers)"] + feat_ok{"SW Team: Feature accepted?"} + arch_rework_feat["Architect: Rework feature definition"] + + dev_create_us["Developer: Create user stories for feature"] + sw_sprint["SW Team: Sprint planning (3-week)"] + sw_execute["SW Team: Execute sprint(s) to complete user stories"] + dev_develop["Developer: Develop on feature branches"] + dev_local_tests["Developer: Run unit tests + static analysis"] + ci_build["CI: Build & automated checks"] + stories_ci{"CI: Stories complete & CI pass?"} + dev_fix_ci["Developer: Rework to fix CI failures or incomplete stories"] + + rev_review["Reviewer: Code review"] + approved{"Reviewer: Approved?"} + dev_addr_comments["Developer: Address review comments"] + + ci_final["CI: Final CI build & checks"] + ci_pass{"CI: CI passes?"} + dev_merge["Developer: Merge to main"] + ci_tv_build["CI: Create T&V build"] + + tv_run["T&V: Execute Test & Verification"] + tv_bugs{"T&V: Bugs found?"} + sw_log_defects["SW Team: Log defects"] + dev_fix_tv["Developer: Fix defects from T&V"] + + sit_run["SIT: System Integration Testing"] + sit_issues{"SIT: SIT issues found?"} + dev_fix_sit["Developer: Fix SIT issues"] + + rel_ready["Release Manager: Feature accepted / ready for release"] + arch_close_feat["Architect: Mark Feature CLOSED"] + arch_req_all_closed{"Architect: All Features for the Requirement CLOSED?"} + arch_close_req["Architect: Mark Requirement CLOSED"] + + %% ===== Flow ===== + pm_create --> pm_submit --> arch_review + arch_review --> arch_clear + arch_clear -- "no" --> pm_revise --> arch_review + arch_clear -- "yes" --> pm_signoff --> arch_define + + arch_define --> pm_feat_review --> sw_feat_review --> feat_ok + feat_ok -- "no" --> arch_rework_feat --> pm_feat_review + feat_ok -- "yes" --> dev_create_us --> sw_sprint --> sw_execute + + sw_execute --> dev_develop --> dev_local_tests --> ci_build --> stories_ci + stories_ci -- "no" --> dev_fix_ci --> dev_develop + stories_ci -- "yes" --> rev_review --> approved + approved -- "no" --> dev_addr_comments --> rev_review + approved -- "yes" --> ci_final --> ci_pass + ci_pass -- "no" --> dev_fix_ci --> dev_develop + ci_pass -- "yes" --> dev_merge --> ci_tv_build --> tv_run --> tv_bugs + + tv_bugs -- "yes" --> sw_log_defects --> dev_fix_tv --> ci_build --> tv_run + tv_bugs -- "no" --> sit_run --> sit_issues + sit_issues -- "yes" --> dev_fix_sit --> ci_build --> sit_run + sit_issues -- "no" --> rel_ready --> arch_close_feat --> arch_req_all_closed + arch_req_all_closed -- "yes" --> arch_close_req + arch_req_all_closed -- "no" --> arch_define + + +``` + +--- + +## Feature & Requirement Lifecycle (PlantUML State) + +```plantuml +@startuml +[*] --> ReqDraft : Requirement created +ReqDraft --> ReqReview : In review +ReqReview --> ReqUpdate : Changes requested +ReqUpdate --> ReqReview +ReqReview --> ReqAccepted : Approved + +ReqAccepted --> FeatureDraft : Feature defined +FeatureDraft --> FeatureReview : In review +FeatureReview --> FeatureUpdate : Changes requested +FeatureUpdate --> FeatureReview +FeatureReview --> FeatureAccepted : Approved + +FeatureAccepted --> InRefinement : User stories created +InRefinement --> InSprint : Committed to sprint +InSprint --> DevComplete : Stories implemented +DevComplete --> ChecksGreen : Build+Tests+Static OK +ChecksGreen --> InReview : Code review +InReview --> ChangesRequested : Reviewer requests changes +ChangesRequested --> InSprint +InReview --> ReadyToMerge : Review approved +ReadyToMerge --> Merged +Merged --> ForTnV : Build for Test & Verification +ForTnV --> TnVFailed : Bugs found +TnVFailed --> InSprint +ForTnV --> ForSIT : T&V passed +ForSIT --> SITFailed : Integration issues found +SITFailed --> InSprint +ForSIT --> Done : Feature accepted +Done --> [*] +@enduml +``` + +--- + +## CI, T&V and SIT Feedback Loops (Mermaid Sequence) + +```mermaid +sequenceDiagram + participant PM as Product Manager + participant Arch as Architect + participant Dev as Developer + participant Rev as Reviewer + participant CI as CI Pipeline + participant TnV as Test & Verification + participant SIT as System Integration + participant RM as Release Manager + + %% --- Requirement loop --- + loop Until all Features for Requirement are closed + PM->>Arch: Submit requirement for review + Arch-->>PM: Approve & sign-off + + %% --- Multiple features per requirement --- + par One or more Features + Arch->>Arch: Define Feature + Arch->>Dev: Provide feature spec + Arch->>Arch: T-shirt size feature + + %% --- User story loop for this feature --- + loop Until all User Stories for Feature are complete + Dev->>Dev: Create user story + Dev->>Arch: Submit for review + Arch-->>Dev: Approve or request changes + Dev->>Dev: Implement & test + Dev->>CI: Push for CI + CI-->>Dev: Results (pass/fail) + end + + Dev->>Rev: Submit Feature PR for review + loop Until approved + Rev-->>Dev: Review comments / fixes + end + Rev-->>CI: Approve → trigger final CI build + CI-->>TnV: Build for T&V + TnV-->>TnV: Execute verification + alt Bugs found + TnV-->>Dev: Defect tickets + loop Until passed + Dev->>CI: Fix & rebuild + CI->>TnV: Retest + end + end + TnV-->>SIT: Promote to SIT + SIT-->>SIT: Integration testing + alt SIT issues found + SIT-->>Dev: SIT defects + loop Until SIT passes + Dev->>CI: Fix & rebuild + CI->>SIT: Retest + end + end + SIT-->>RM: Feature accepted + Arch->>Arch: Mark Feature closed + and Another feature (parallel) + Arch->>Arch: Define additional Feature + note right of Arch: Same process repeats for each Feature
until all are complete + end + end + + %% --- Requirement closure --- + Arch->>Arch: All Features closed → Requirement closed ✅ + +``` + +```mermaid +sequenceDiagram + participant PM as Product Manager + participant Arch as Architect + participant Dev as Developer + participant Rev as Reviewer + participant VCS as Git Host + participant CI as CI Pipeline + participant TnV as Test & Verification + participant SIT as System Integration + participant RM as Release Manager + + %% Requirements loop + loop Until requirement is accepted + PM->>Arch: Submit requirement for review + alt Needs changes + Arch-->>PM: Feedback / requested changes + PM->>Arch: Revise and resubmit + else Acceptable + Arch-->>PM: Recommend sign-off + end + end + PM->>PM: Requirement SIGN-OFF + + %% Feature definition & review loop + Arch->>Arch: Define features from signed-off requirement + loop Feature review until accepted + PM-->>Arch: PM feature review feedback + Dev-->>Arch: Dev feasibility feedback + alt Rework needed + Arch->>Arch: Rework feature definition + else Accepted + Arch-->>Dev: Approved feature for implementation + end + end + + %% Story creation and sprinting (drop-in) + Arch->>Arch: T-shirt size features + Dev->>Dev: Create user stories for feature + + %% Architect review of user stories + loop Until user stories accepted + Dev->>Arch: Submit user stories for review + alt Changes required + Arch-->>Dev: Feedback / requested changes + Dev->>Dev: Update user stories + else Accepted + Arch-->>Dev: Approve user stories + end + end + + Dev->>Dev: Size user stories (days) + + %% 3-week sprint loop + loop 3-week sprints until all user stories complete + Dev->>Dev: Sprint planning (select stories) + loop During sprint + Dev->>Dev: Design and document + Dev->>Dev: Implement design + Dev->>Dev: Create and run unit tests + Dev->>VCS: Push commits to feature branch + VCS-->>CI: Trigger CI + CI->>CI: Compile + unit tests + static analysis + CI-->>Dev: Results (pass/fail) + alt CI fails + Dev->>Dev: Fix & re-push + end + end + Dev->>Dev: Sprint review & close + end + + %% Review + merge gates (after all user stories complete for the feature) + Dev->>Dev: All user stories for feature complete ✅ + + Dev->>VCS: Open pull request + VCS-->>Rev: Notify for code review + + %% Review loop + loop Until approved + Rev-->>Dev: Review comments / requests + Dev->>VCS: Address comments (push updates) + end + + Rev-->>VCS: Approve + VCS-->>CI: Final pre-merge checks + + %% Pre-merge CI gate + alt Checks pass + Dev->>VCS: Merge to main + else Checks fail + CI-->>Dev: Fix required + Dev->>VCS: Push fixes + VCS-->>CI: Re-run checks + Dev->>VCS: Merge to main + end + %% T&V loop + VCS-->>CI: Create T&V build + CI->>TnV: Deliver build + TnV-->>TnV: Execute tests + alt Bugs found + TnV-->>Dev: Defect tickets + loop Until T&V passes + Dev->>VCS: Fix & push + VCS-->>CI: Rebuild + CI->>TnV: Retest + end + end + + %% SIT loop + TnV-->>SIT: Promote build to SIT + SIT-->>SIT: Integration testing + alt SIT issues found + SIT-->>Dev: SIT defects + loop Until SIT passes + Dev->>VCS: Fix & push + VCS-->>CI: Rebuild + CI->>SIT: Retest + end + end + + %% Acceptance and closure + SIT-->>RM: Feature accepted / ready for release + Arch->>Arch: Close Feature + Arch->>Arch: If all features for requirement closed ⇒ Close Requirement + +``` + +--- + +## Commentary & Rationale + +- **Feedback loops** are intentionally present in *every phase*, not just testing. +- **Requirements & features** have review cycles to ensure downstream teams work with clarity and stable inputs. +- **Feature gates** before merge protect `main` and keep builds stable. +- **T&V** and **SIT** are treated as structured gates, not chaotic bug hunts. +- **Rework is planned**, not a surprise — stories or tasks can be created from findings and re-run through the process. +- This model aligns well with ADO/Jira workflows, CI/CD pipelines, and regulated development environments. + +--- + +## Suggested Practice + +| Stage | Trigger | Feedback Loop | +|-------|---------|---------------| +| Requirement | Ambiguity, scope, missing acceptance | Back to PM for revision | +| Feature | Technical feasibility or unclear scope | Back to Architect | +| Dev | Build, static analysis, unit test failure | Developer fixes before merge | +| Code Review | Review comments | Developer updates | +| T&V | Defects found | Rework through sprint | +| SIT | Integration issues | Rework through sprint | + +--- + +**Author:** Internal Engineering Team +**Version:** 1.0 +**Date:** October 2025 + + +```mermaid +sequenceDiagram + participant PM as Product Manager + participant Arch as Architect + participant Dev as Developer + participant Rev as Reviewer + participant VCS as Git Host + participant CI as CI Pipeline + participant TnV as Test & Verification + participant SIT as System Integration + participant RM as Release Manager + + %% === Requirement lifecycle === + loop Until all Features for Requirement are closed + PM->>Arch: Submit requirement for review + loop Until requirement accepted + alt Needs changes + Arch-->>PM: Feedback / requested changes + PM->>Arch: Revise and resubmit + else Acceptable + Arch-->>PM: Recommend sign-off + end + end + PM->>PM: Requirement SIGN-OFF ✅ + + %% === Feature lifecycle === + par One or more Features + Arch->>Arch: Define Feature + Arch->>Arch: T-shirt size Feature + Arch->>Dev: Provide feature specification + + %% === User story development === + loop Until all User Stories for Feature are complete + par Parallel user stories + Dev->>Dev: Create user story + Dev->>Arch: Submit user story for review + alt Changes required + Arch-->>Dev: Feedback / rework + Dev->>Arch: Resubmit user story + else Accepted + Arch-->>Dev: Approve user story + end + Dev->>Dev: Implement story (develop, unit test, static analysis) + Dev->>VCS: Push to feature branch + VCS-->>CI: Trigger CI build & checks + CI->>CI: Compile + run unit tests + static analysis + CI-->>Dev: Results (pass/fail) + alt CI fails + Dev->>Dev: Fix and re-push + else Pass + Dev-->>Rev: Ready for PR inclusion + end + and Another user story + note right of Dev: Stories can be implemented
in parallel during sprints + end + end + + Dev->>Dev: All user stories for feature complete ✅ + + %% === Review and merge === + Dev->>VCS: Open pull request (feature) + VCS-->>Rev: Notify for code review + loop Until approved + Rev-->>Dev: Review comments / changes + Dev->>Rev: Push updates + end + Rev-->>CI: Approve → trigger pre-merge CI checks + alt Checks pass + Dev->>VCS: Merge to main + else Checks fail + CI-->>Dev: Fix required → re-push + Dev->>VCS: Push fixes + VCS-->>CI: Re-run checks + Dev->>VCS: Merge to main + end + + %% === Test & Verification stage === + VCS-->>CI: Create T&V build + CI->>TnV: Deliver build for testing + TnV-->>TnV: Execute T&V tests + alt Bugs found + TnV-->>Dev: Raise defect tickets + loop Until T&V passes + Dev->>Dev: Diagnose and implement fix + Dev->>VCS: Push fix to feature branch + VCS-->>CI: Trigger CI build & checks + CI->>CI: Compile + unit tests + static analysis + CI-->>Dev: Results (pass/fail) + alt CI fails + Dev->>Dev: Correct issue & re-push + else Pass + Dev->>Rev: Request review for fix + loop Until approved + Rev-->>Dev: Review comments / changes + Dev->>Rev: Push updates + end + Rev-->>CI: Approve → trigger pre-merge checks + CI-->>Dev: Verify checks passed + Dev->>VCS: Merge fix to main + end + VCS-->>CI: Rebuild main + CI->>TnV: Deliver build for retest + TnV-->>TnV: Execute regression tests + end + end + + %% === SIT stage === + TnV-->>SIT: Promote build to SIT + SIT-->>SIT: Execute integration testing + alt SIT issues found + SIT-->>Dev: Raise SIT defect tickets + loop Until SIT passes + Dev->>Dev: Diagnose and implement fix + Dev->>VCS: Push fix to feature branch + VCS-->>CI: Trigger CI build & checks + CI->>CI: Compile + unit tests + static analysis + CI-->>Dev: Results (pass/fail) + alt CI fails + Dev->>Dev: Correct issue & re-push + else Pass + Dev->>Rev: Request review for fix + loop Until approved + Rev-->>Dev: Review comments / changes + Dev->>Rev: Push updates + end + Rev-->>CI: Approve → trigger pre-merge checks + CI-->>Dev: Verify checks passed + Dev->>VCS: Merge fix to main + end + VCS-->>CI: Rebuild main + CI->>SIT: Deliver build for retest + SIT-->>SIT: Execute regression tests + end + end + + SIT-->>RM: Feature accepted / ready for release + Arch->>Arch: Mark Feature closed + and Another feature + note right of Arch: Requirements may have
multiple features progressing
in parallel + end + end + + %% === Requirement closure === + Arch->>Arch: All Features closed → Requirement closed ✅ + +``` diff --git a/content/guides/ways-of-working/04-ci-cd.md b/content/guides/ways-of-working/04-ci-cd.md new file mode 100644 index 0000000..8b70158 --- /dev/null +++ b/content/guides/ways-of-working/04-ci-cd.md @@ -0,0 +1,81 @@ +--- +title: Continuous integration and delivery +--- + +# Continuous integration and delivery + +Continuous Integration (CI) and Continuous Delivery (CD) are a cornerstone of modern development. They are about automating the boring, repetitive tasks so your team can focus on what matters: delivering value. In a multi-disciplinary team, that means more than building and testing code. It means checking every artefact, from firmware to documentation, and giving it a smooth, reliable path to customers. + +This guide outlines key considerations for designing a CI/CD pipeline, aligning with our core philosophy of structure without bureaucracy. + +## The Challenge of a Hybrid Pipeline + +Just as with version control, a CI/CD pipeline for a multi-disciplinary product must handle different types of artifacts and validation needs. + +* **Software and Firmware:** Requires rapid build, lint, and unit test cycles. + +* **Hardware:** Can be validated through simulations, automated tests on physical hardware (Hardware-in-the-Loop or HIL), or even formal design checks. + +* **Documentation:** Should be linted, built into a website, and published automatically. + +The goal is to create a single, unified pipeline that provides fast feedback to every team member, regardless of their discipline. + +## Key Considerations for Your Pipeline + +A good CI/CD pipeline is about enabling flow and reducing friction. Here are some key principles to guide your design. + +### 1. Shift Left and Fail Fast + +The best way to prevent problems is to find them as early as possible. Your pipeline should run a series of automated checks and tests at every stage, starting with the very first commit or pull request. + +* **Continuous Integration (CI):** Every time a change is pushed to a feature branch, the CI pipeline should automatically build and test the code. This gives the developer immediate feedback and prevents them from introducing a bug that could block the entire team. + +* **Automated Checks:** Implement checks for code style, documentation quality, and security vulnerabilities. This ensures consistency and catches common errors before they are ever reviewed by a human. + +* **Decoupling Tests:** Separate your tests into different stages. A fast-running smoke test should run on every commit, while a longer, more comprehensive suite can run on merge requests or nightly builds. This gives your team fast feedback without sacrificing thoroughness. + +### 2. Embrace Automation as an Enabler + +Automation is a force multiplier. It saves time, reduces human error, and ensures consistency. + +* **Build Automation:** Automate the entire build process, from compiling code to generating an executable or a production-ready artefact. This eliminates the "works on my machine" problem. + +* **Automated Testing:** Run unit, integration, and end-to-end tests automatically. This gives you confidence that your changes haven't broken anything. + +* **Automated Deployment:** Automate the process of deploying your application to a staging or production environment. This ensures a consistent, repeatable process and reduces the risk of manual errors. + +### 3. Integrate All Artifacts + +A CI/CD pipeline for a hybrid product should treat all artifacts as first-class citizens. + +* **Docs as Code:** When a developer updates documentation, the CI pipeline should automatically build the website and publish the changes. + +* **Hardware Design Checks:** For hardware, your CI pipeline could run a check that validates a PCB layout against a set of design rules. This catches errors early and prevents costly manufacturing mistakes. + +* **Artefact Traceability:** Ensure that every build artefact (e.g., a software binary, a PCB gerber file) is tagged with a unique version number and a link to the commit that generated it. This provides a clear audit trail and is essential for troubleshooting and compliance. + +## The Pipeline in Practice + +Let's imagine a pull request for a new feature. + +1. **Code Push:** A developer pushes a new commit. The CI pipeline is triggered. + +2. **Lint & Test:** The pipeline runs a linter to check for code style issues and then runs the unit tests. If anything fails, the developer is notified immediately, and the pipeline stops. + +3. **Peer Review:** The developer opens a pull request. A pull request is not just a code review; it's a review of the entire change, including documentation updates, architecture decisions, and tests. + +4. **Merge & Build:** Once reviewed, the code is merged into the `main` branch. The pipeline is triggered again, this time running a more comprehensive suite of tests and checks. + +5. **Deploy:** Once all tests pass, the artefact is automatically deployed to a staging environment for final validation. + +This process provides continuous feedback and ensures that the `main` branch is always in a releasable state, making a "release" a non-event. + +## Tips for Success + +* **Start Small:** Don't try to automate everything at once. Start by automating your build process, then add tests, and then work on deployment. + +* **Treat the Pipeline as Code:** Your CI/CD configuration should live in your repository alongside your code. This ensures it is version-controlled, auditable, and easily shared among your team. + +* **Focus on the Metrics:** Measure what matters. How long does it take for a pull request to be merged? How many bugs are making it to production? Use these metrics to continuously improve your process. + +* **Blameless Postmortems:** When something goes wrong in the pipeline, focus on what failed, not who. Use the failure as a learning opportunity to improve the process for everyone. diff --git a/content/guides/ways-of-working/05-documentation-standards.md b/content/guides/ways-of-working/05-documentation-standards.md new file mode 100644 index 0000000..15de7a2 --- /dev/null +++ b/content/guides/ways-of-working/05-documentation-standards.md @@ -0,0 +1,217 @@ +--- +title: Documentation standards +--- + +--- + +title: Documentation Standards +status: draft +version: 0.1 +owner: Engineering +last_updated: 2026-01-05 +------------------------ + +# Documentation standards + +This guide defines how we write, structure, maintain, and evolve documentation. + +Documentation is part of the product. It is not an afterthought, a side task, or something that happens only when things go wrong. Good documentation reduces risk, accelerates onboarding, supports decision-making, and preserves intent over time. + +These standards apply to all engineering documentation, regardless of format or audience. + +## Purpose + +Our documentation standards exist to: + +* Make intent, decisions, and behaviour explicit +* Reduce reliance on tribal knowledge +* Support safe change and long-term maintenance +* Enable asynchronous collaboration +* Keep documentation trustworthy and current + +If documentation is not used or trusted, it has failed. + +## Core principles + +* Documentation is owned, not abandoned +* Write for future readers, not present authors +* Prefer clarity over completeness +* Keep documentation close to the thing it describes +* Treat documentation as versioned artefacts + +## Documentation as code + +All substantive documentation should: + +* Live in version control +* Be written in plain text formats (Markdown first) +* Be reviewed via pull request +* Change alongside the code or system it describes + +This enables traceability, review, and controlled evolution. + +## Documentation types + +We use different document types for different purposes. Mixing them leads to confusion. + +## Reference documentation + +Reference documentation describes what *is*. + +Examples: + +* API specifications +* Configuration options +* Data schemas +* Interface definitions + +Characteristics: + +* Precise and unambiguous +* Minimal narrative +* Kept strictly in sync with implementation + +## Conceptual documentation + +Conceptual documentation explains *why* things exist and how they fit together. + +Examples: + +* Architecture overviews +* System context diagrams +* Design rationale + +Characteristics: + +* Focus on intent and trade-offs +* Stable over time +* Written for understanding, not step-by-step execution + +## Procedural documentation + +Procedural documentation explains *how* to do something. + +Examples: + +* Build and deployment steps +* Operational runbooks +* Onboarding guides + +Characteristics: + +* Action-oriented +* Clear prerequisites and outcomes +* Regularly validated against reality + +## Decision records + +Architectural and significant technical decisions must be documented explicitly. + +* Use Architecture Decision Records where appropriate +* Capture context, decision, alternatives, and consequences +* Keep records immutable once accepted + +The goal is not approval theatre, but preserved reasoning. + +## Structure and layout + +Documentation should follow a predictable structure. + +* Clear title and purpose +* Audience identified implicitly or explicitly +* Logical sectioning +* Consistent headings +* Summary or orientation at the top for longer documents + +Avoid long, unbroken walls of text. + +## Language and tone + +* Clear, direct, and neutral +* Active voice preferred +* Avoid marketing language and hype +* Avoid unnecessary jargon +* Explain acronyms on first use + +Write as if the reader is capable but unfamiliar with local context. + +## Diagrams + +Diagrams are first-class documentation. + +* Prefer diagrams as code +* Keep diagrams simple and focused +* One diagram per concern +* Diagrams must match reality + +```mermaid +flowchart TD + A[Concept] --> B[Documentation] + B --> C[Implementation] + C --> D[Change] + D --> B +``` + +If a diagram cannot be kept up to date, remove it. + +## Naming and organisation + +* File and folder names should be descriptive and stable +* Avoid vague names like `misc` or `notes` +* Group documents by concern, not by author or date +* Keep paths short and navigable + +Consistency matters more than perfection. + +## Reviews and maintenance + +Documentation changes follow the same review standards as code. + +* Changes reviewed for clarity and correctness +* Reviewers consider the reader’s perspective +* Outdated documentation is treated as a defect + +If documentation is wrong, fix it immediately. + +## Keeping documentation current + +Documentation should be updated: + +* When behaviour changes +* When decisions are made +* When processes evolve +* When onboarding feedback highlights gaps + +Stale documentation is worse than no documentation. + +## Ownership + +Every document should have a clear owner or owning group. + +Ownership means: + +* Keeping it accurate +* Responding to change requests +* Periodic review + +Ownership does not mean exclusive authorship. + +## Common failure modes + +Watch for: + +* Documents written once and never revisited +* Overly long documents no one reads +* Duplication with slight inconsistencies +* Documentation divorced from code +* Optimising for format over usefulness + +If you see these patterns, simplify. + +## Evolution of standards + +* These standards are expected to evolve +* Improvements should be proposed via pull request +* Exceptions are allowed when explicit and justified + +The standard exists to support the work, not constrain it. diff --git a/content/guides/ways-of-working/06-architecture-and-design-documentation.md b/content/guides/ways-of-working/06-architecture-and-design-documentation.md new file mode 100644 index 0000000..2f3d0c3 --- /dev/null +++ b/content/guides/ways-of-working/06-architecture-and-design-documentation.md @@ -0,0 +1,186 @@ +--- +title: Architecture and design documentation +--- + +--- + +title: Architecture & Design Documentation +status: draft +version: 0.1 +owner: Engineering +last_updated: 2026-01-05 +------------------------ + +# Architecture and design documentation + +This guide defines how we document architecture and design decisions so systems remain understandable, evolvable, and safe to change over time. + +Architecture documentation is not about creating perfect diagrams or heavyweight documents. It is about making structure, intent, constraints, and trade-offs explicit at the right level of detail. + +## Purpose + +Architecture and design documentation exists to: + +* Explain how the system is structured +* Make design intent and constraints explicit +* Support reasoning about change and impact +* Enable informed technical decision-making +* Preserve context beyond individual contributors + +If a design decision cannot be explained clearly, it is probably not ready. + +## Scope + +This guide covers: + +* System and software architecture documentation +* Significant design decisions and trade-offs +* Evolution of architecture over time + +It does not replace detailed implementation documentation or code-level comments. + +## Core principles + +* Document decisions, not just diagrams +* Match detail to risk and longevity +* Prefer simple, stable views over exhaustive detail +* Keep documentation close to the code +* Update documentation when the architecture changes + +## Architecture versus design + +We distinguish between architecture and design to avoid confusion. + +* Architecture describes the fundamental structure, boundaries, and responsibilities +* Design describes how a specific problem is solved within those boundaries + +Architecture changes slowly. Design changes more frequently. + +## When to write architecture documentation + +Architecture documentation is required when: + +* Introducing a new system or major component +* Changing system boundaries or responsibilities +* Adding cross-cutting concerns such as security, data persistence, or messaging +* Making decisions that constrain future design choices + +For small, local changes, lightweight design notes are sufficient. + +## Levels of documentation + +We use multiple levels to keep documentation useful and navigable. + +## System context + +The system context shows what the system is and what it interacts with. + +* External systems +* Users and actors +* High-level responsibilities +* Clear system boundary + +```mermaid +flowchart TD + A[Users] --> B[System] + C[External Service] --> B + B --> D[Downstream Systems] +``` + +This view should be stable and easy to understand by non-specialists. + +## Container and component views + +These views explain the internal structure at a high level. + +* Major deployable units +* Responsibilities of each component +* Key communication paths + +Avoid showing every class or function. Focus on responsibility and interaction. + +## Deployment and runtime views + +These views show how the system runs in real environments. + +* Environments and nodes +* Network boundaries +* Data flows +* Scaling and resilience concerns + +These views are critical for operability and incident response. + +## Design documentation + +Design documentation explains how a specific change or feature works. + +A good design document: + +* States the problem clearly +* Describes constraints and assumptions +* Explains the chosen approach +* Documents alternatives considered +* Identifies risks and mitigations + +Design documents should be lightweight and time-bound. + +## Architecture Decision Records + +Significant architectural decisions must be captured explicitly. + +* Use a consistent ADR format +* Record context, decision, and consequences +* Do not rewrite history after acceptance + +ADRs complement diagrams by preserving reasoning. + +## Diagrams as code + +Diagrams should be treated as maintainable artefacts. + +* Prefer text-based diagram formats +* Keep diagrams close to the documentation they support +* Review diagram changes alongside code changes + +If a diagram cannot be kept in sync, remove it. + +## Keeping documentation current + +Architecture documentation must evolve with the system. + +* Update docs when responsibilities change +* Review architecture docs during significant refactors +* Treat outdated architecture documentation as a defect + +Trust in documentation is hard to rebuild once lost. + +## Reviews and governance + +Architecture documentation should be reviewed with intent. + +* Does it clearly explain structure and boundaries? +* Are trade-offs and constraints explicit? +* Is the level of detail appropriate? +* Does it align with existing architecture principles? + +The goal is clarity, not approval theatre. + +## Common failure modes + +Watch for: + +* Overly detailed diagrams that rot quickly +* Missing rationale behind decisions +* Architecture frozen on paper but not in reality +* Documentation that mirrors code structure too closely +* Treating architecture docs as one-off deliverables + +If you see these, simplify and refocus. + +## Ownership and evolution + +* Architecture documentation has clear ownership +* Updates are made via pull request +* Improvements are encouraged + +This guide should evolve as our systems and practices evolve. diff --git a/content/guides/ways-of-working/07-documentation-and-modelling-tools.md b/content/guides/ways-of-working/07-documentation-and-modelling-tools.md new file mode 100644 index 0000000..e188c30 --- /dev/null +++ b/content/guides/ways-of-working/07-documentation-and-modelling-tools.md @@ -0,0 +1,51 @@ +--- +title: Documentation and modelling tools +--- + +# Documentation and modelling tools + +A crucial part of our **Docs as Code** approach is treating documentation just like code. This means we can apply the same development principles-like version control, automated checks, and continuous delivery-to our written artifacts. The result is documentation that's always up-to-date, accurate, and easily accessible. + +This guide outlines the tooling stack that makes this possible and how to integrate it into our workflows. + +## The Power of Markdown + +Markdown is our chosen syntax for all written documentation. It's a lightweight markup language that is both human-readable and can be converted to other formats, like HTML or PDF. + +* **Simplicity and Consistency:** You can write Markdown in any text editor, making it easy for everyone on the team, regardless of their role or technical background, to contribute to and maintain our documentation. This is a huge win for collaboration. + +* **Version Control:** By storing all .md files in our version control system, every change is tracked and auditable. We can see who made a change, when, and why. This is essential for traceability and compliance, especially for regulated products. + +## Static Site Generators + +While Markdown is great for writing, it's not ideal for browsing a large collection of documents. This is where a **Static Site Generator** comes in. Tools like **MkDocs** and **Sphinx** transform our raw Markdown files into a professional, searchable, and navigable website. + +* **Automated Publishing:** Our CI/CD pipeline can automatically build and publish the updated site whenever a documentation change is merged. This ensures our documentation is always live and in sync with our codebase and hardware designs. + +* **Search and Navigation:** These tools automatically create a navigation menu and a powerful search function, making it easy for anyone to find the information they need without digging through file systems. They handle the hard work of organizing content so we can focus on writing. + +## Diagrams as Code + +For complex systems, a picture is worth a thousand words. But a static image can quickly become outdated. Our solution is to use **diagrams as code** with tools like **Mermaid** and **PlantUML**. + +* **Version-Controlled Diagrams:** These tools allow you to write simple, human-readable code that generates a diagram. The code for the diagram is stored directly in the Markdown file, meaning it's version-controlled alongside the rest of the documentation. + +* **Always in Sync:** The diagram is rendered automatically when the static site is built, so it can never fall out of sync with the documentation it's describing. If a change is made to the system, the corresponding diagram code can be updated in the same pull request. + +* **No Special Software:** This approach removes the need for separate diagramming software and external image files, which often leads to diagrams becoming outdated. + +For example, using the **C4 model**, you can create a high-level context diagram and a detailed component diagram all within your Markdown files. This makes it easy for a new team member to understand the architecture without having to hunt for an old PowerPoint slide. + +## Toolchain Integration + +The power of this approach comes from seamlessly integrating these tools into our CI/CD pipeline. Here's a simplified example of how this might work for a documentation job in our pipeline: + +1. **Lint Markdown:** A CI job checks all Markdown files for formatting and syntax errors, ensuring consistency and catching common mistakes before they're merged. + +2. **Validate Diagrams:** The pipeline validates that the Mermaid or PlantUML code is correctly formatted. This prevents broken diagrams from being published. + +3. **Build Site:** The static site generator builds the documentation site from the Markdown files and renders the diagrams. + +4. **Publish:** The updated site is automatically published to a web server for everyone to access. + +This process provides immediate feedback on documentation changes and ensures that our "living artifacts" are always in a pristine and usable state. It transforms documentation from a manual chore into an automated, integrated part of our delivery pipeline. diff --git a/content/guides/ways-of-working/08-metrics-and-feedback.md b/content/guides/ways-of-working/08-metrics-and-feedback.md new file mode 100644 index 0000000..ce6abd4 --- /dev/null +++ b/content/guides/ways-of-working/08-metrics-and-feedback.md @@ -0,0 +1,202 @@ +--- +title: Metrics and feedback +--- + +--- + +title: Metrics & Feedback Loop +status: draft +version: 0.1 +owner: Engineering +last_updated: 2026-01-05 +------------------------ + +# Metrics and feedback + +This guide defines how we use metrics and feedback to understand how delivery is really working and how we improve it over time. + +Metrics exist to support learning and decision-making. They are not targets, performance scores, or tools for individual judgement. Poorly used metrics damage trust and behaviour. Well-used metrics make problems visible early and create space to improve. + +## Purpose + +Our approach to metrics aims to: + +* Make delivery health visible +* Identify bottlenecks and risks early +* Support continuous improvement +* Provide evidence for decisions and trade-offs +* Close the loop between intent and reality + +If a metric does not inform action or learning, it should not be collected. + +## Principles + +* Metrics are for teams, not individuals +* Trends matter more than single data points +* Context always beats raw numbers +* Measure the system, not the people +* Use the smallest useful set of metrics + +## Metric categories + +We group metrics by what they tell us about the system. + +## Flow and delivery + +These metrics describe how work moves through the system. + +* Lead time: time from work starting to work delivered +* Cycle time: time from work in progress to completion +* Work in progress: number of active items +* Throughput: completed items per time period + +These help answer: + +* Are we getting faster or slower? +* Where is work queuing? +* Are we overloading the system? + +## Quality + +These metrics describe how safe and reliable our output is. + +* Defect escape rate +* Test coverage trends (not absolute targets) +* Build and test pass rate +* Flaky test frequency +* Rollback or hotfix frequency + +These help answer: + +* Are we paying down or accumulating risk? +* Where does quality break down? + +## Stability and operability + +These metrics describe how the system behaves in production. + +* Deployment frequency +* Mean time to detect issues +* Mean time to recover +* Incident count and severity +* Alert noise ratio + +These help answer: + +* How resilient is the system? +* How disruptive are changes? + +## Predictability + +These metrics describe how well we meet expectations. + +* Planned versus completed work +* Forecast accuracy over time +* Scope change rate during delivery + +These help answer: + +* How reliable are our plans? +* Where does uncertainty enter the system? + +## People and ways of working + +These signals are often qualitative rather than numeric. + +* Team sentiment from retrospectives +* Onboarding feedback +* Review turnaround times +* Unplanned work frequency + +These help answer: + +* Is the way we work sustainable? +* Where is friction emerging? + +## What we do not measure + +We explicitly avoid: + +* Individual productivity metrics +* Lines of code +* Story points as performance indicators +* Rankings or comparisons between engineers + +These distort behaviour and reduce trust. + +## Feedback loops + +Metrics only matter if they feed back into action. + +```mermaid +flowchart TD + A[Work executed] --> B[Metrics and signals collected] + B --> C[Team reviews data] + C --> D[Insights identified] + D --> E[Actions agreed] + E --> F[Changes applied] + F --> A +``` + +## Review cadence + +Metrics should be reviewed at the right level and frequency. + +* Daily: operational signals and blockers +* Weekly: flow and quality trends +* Sprint or iteration: improvement actions +* Quarterly: systemic issues and investment decisions + +Avoid reviewing everything everywhere. Match cadence to decision type. + +## Using metrics in retrospectives + +In retrospectives, metrics are used to: + +* Ground discussion in evidence +* Validate or challenge perceptions +* Identify experiments to try next + +They are not used to assign blame or justify pre-made decisions. + +## Experiments and learning + +Improvements should be treated as experiments. + +* State the hypothesis +* Define what success looks like +* Use metrics to observe impact +* Keep or revert based on evidence + +Small, frequent experiments beat large, risky changes. + +## Making metrics visible + +Where possible: + +* Dashboards are shared and accessible +* Definitions are documented +* Trends are shown over time +* Context is captured alongside data + +A metric without explanation is a liability. + +## Common failure modes + +Watch for: + +* Too many metrics +* Metrics collected but never discussed +* Optimising one metric while damaging others +* Using metrics to justify pressure rather than improvement +* Loss of qualitative feedback + +If you see these, reset. + +## Ownership and evolution + +* Metric definitions are owned by the team +* Changes to metrics should be explicit +* Retire metrics that no longer drive insight + +This guide should evolve as the team and system evolve. diff --git a/content/guides/ways-of-working/index.md b/content/guides/ways-of-working/index.md new file mode 100644 index 0000000..74fcc69 --- /dev/null +++ b/content/guides/ways-of-working/index.md @@ -0,0 +1,20 @@ +--- +title: Ways of working +--- + +# Ways of working + +How a team works day to day: its agreements, onboarding, the development process, CI/CD, documentation and the metrics it watches. The [Guidelines](../../guidelines/index.md) set the rules; these chapters describe the practice behind them. + +| Chapter | +| --- | +| [Working agreements and team norms](01-working-agreements.md) | +| [Onboarding engineers](02-onboarding.md) | +| [The development process, end to end](03-development-process.md) | +| [Continuous integration and delivery](04-ci-cd.md) | +| [Documentation standards](05-documentation-standards.md) | +| [Architecture and design documentation](06-architecture-and-design-documentation.md) | +| [Documentation and modelling tools](07-documentation-and-modelling-tools.md) | +| [Metrics and feedback](08-metrics-and-feedback.md) | + +[[CAPTION:Table]] The chapters of the ways of working guide. diff --git a/content/handbook/01-introduction-to-architecture.md b/content/handbook/01-introduction-to-architecture.md new file mode 100644 index 0000000..5eed7d8 --- /dev/null +++ b/content/handbook/01-introduction-to-architecture.md @@ -0,0 +1,148 @@ +--- +title: Introduction to architecture +--- + +# Introduction to architecture + +Architecture provides the structural blueprint that guides system engineering from inception through decades of operational deployment. ISO/IEC/IEEE 42010 defines architecture as the fundamental organisation of a system. It is embodied in the system's components and their relationships to each other and to the environment, and in the principles that guide its design and evolution. + +## 1. What is Architecture? + +Martin Fowler describes architecture as "the shared understanding that developers have of the system design". It is also "the decisions that you wish you could get right early in a project because they are difficult to change later." + +Architecture is distinguished from detailed implementation by three core characteristics: +1. **Scope and Consequence**: Architectural decisions dictate system topology, inter-process communication protocols, security boundaries, and persistence mechanisms. Reversing an architectural decision incurs orders of magnitude higher cost than modifying an internal function. +2. **Non-Functional Focus**: While feature engineering focuses on what the system does (functional requirements), architecture focuses on how the system behaves under stress, failure, growth, and security attacks (quality attributes or non-functional requirements). +3. **Abstraction Management**: Architecture abstracts away algorithmic minutiae to provide clear cognitive models of how data, control, and failure domains interact across the entire estate. + +### The Four Architectural Tiers + +System engineering operates across four interconnected architectural tiers: + +| Tier | Primary Focus | Typical Artefacts | Stakeholders | +| :--- | :--- | :--- | :--- | +| **Enterprise Architecture** | Strategic business capabilities, corporate IT landscape, digital transformation, and technical debt reduction across the entire company. | Capability models, strategic roadmaps, technology standards radar. | C-suite, CTO, VP Engineering, Business Unit Leaders. | +| **Solution Architecture** | End-to-end technical solution spanning multiple systems, clouds, edge gateways, and commercial SaaS integrations for a specific product or project. | Solution Design Documents (SDD), High-Level Designs (HLD), interface contracts. | Product Managers, Delivery Leads, Lead Architects. | +| **Software / Application Architecture** | Internal structure of a specific application, microservice, or gateway daemon (classes, modules, threads, databases). | Low-Level Designs (LLD), Software Detailed Specifications ([Writing a software design specification](../guides/software-design-specification.md)), class/container models. | Tech Leads, Senior Software Engineers. | +| **Embedded & Systems Architecture** | Hardware-software co-design, real-time constraints, peripheral buses (I2C, SPI, CAN), memory segmentation, and power budgets. | Hardware-Software Interface specifications, memory layout maps, RTOS task concurrency models. | Firmware Engineers, Hardware Engineers, Embedded Architects. | + +[[CAPTION:Table]] The four architectural tiers in modern engineering organisations. + +```mermaid +flowchart TD + EA[Enterprise Architecture: Strategic Business Capabilities & Standards] + SA[Solution Architecture: End-to-End System Integration & Data Flows] + AA[Application Architecture: Microservices, APIs & Data Stores] + ESA[Embedded Systems Architecture: Hardware/Firmware, RTOS & Sensors] + + EA --> SA + SA --> AA + SA --> ESA + AA <-->|Edge-to-Cloud Interfaces| ESA +``` +[[CAPTION:Figure]] Relationships between the four architectural tiers. + +## 2. The Role of the Modern Architect + +Historically, organisations frequently succumbed to the disconnected, top-down architecture antipattern: individuals producing voluminous, theoretical specification documents without engaging with the engineering teams implementing them. + +Modern engineering requires pragmatic, engaged technical leadership. The modern architect functions as an enabler, mentor, and communicator who scales architectural decision-making across autonomous delivery teams. + +### Core Responsibilities of an Architect + +- **Technical Vision and Tenets**: Defining clear, unambiguous architectural principles that empower delivery teams to make decentralized, coherent decisions (the Advice Process). +- **Managing Technical Risk and Trade-offs**: Evaluating competing priorities (for example, latency versus throughput, battery life versus reporting frequency) through structured trade-off analyses (Architecture Decision Records per [Writing ADRs and RFCs](../guides/decision-records.md)). +- **Governance and Quality Assurance**: Chairing the Architecture Review Board ([Running an architecture review board](../guides/architecture-review-board.md)) and conducting systematic checklist reviews ([Architecture review checklist](../templates/architecture-review-checklist.md)) to maintain consistency across squads. +- **Continuous Hands-On Calibration**: Remaining close to production code and deployment pipelines. Architects who never inspect pull requests or flash hardware lose calibration with real-world developer friction. + +## 3. Core Architecture Frameworks and Standards + +Formal architecture frameworks provide standardised vocabularies, decomposition methodologies, and documentation structures. + +### 3.1 TOGAF (The Open Group Architecture Framework) + +TOGAF is the globally recognized enterprise architecture standard. Its core engine is the **Architecture Development Method (ADM)**, an iterative cycle for creating enterprise architectures: + +```mermaid +flowchart TD + P[Preliminary Phase: Principles & Governance] --> A[Phase A: Architecture Vision] + A --> B[Phase B: Business Architecture] + B --> C[Phase C: Information Systems Architectures - Data & App] + C --> D[Phase D: Technology Architecture] + D --> E[Phase E: Opportunities & Solutions] + E --> F[Phase F: Migration Planning] + F --> G[Phase G: Implementation Governance] + G --> H[Phase H: Architecture Change Management] + H --> A +``` +[[CAPTION:Figure]] TOGAF Architecture Development Method (ADM) iterative cycle. + +TOGAF provides rigor when structuring large multi-year enterprise transformation initiatives, ensuring technology roadmaps align directly with corporate business capabilities. + +### 3.2 The Zachman Framework + +John Zachman formulated a two-dimensional classification matrix providing an exhaustive taxonomy of enterprise architecture. It intersects six fundamental communication questions against six stakeholder perspectives: + +| Perspective \ Focus | What (Data) | How (Function) | Where (Network) | Who (People) | When (Time) | Why (Motivation) | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| **Planner (Scope)** | List of business objects | List of business processes | List of business locations | List of organisations | List of business events | List of business goals | +| **Owner (Business Concept)** | Conceptual data model | Business process model | Logistics network model | Organisational workflow | Business cycle model | Business plan | +| **Designer (System Logic)** | Logical data model | System architecture model | Distributed system topology | Human interface spec | State transition model | Business rule model | +| **Builder (Technology Physics)** | Physical data model | Software detailed design | Network architecture spec | Presentation format | Process timing schedule | Operational rule model | +| **Integrator (Component Config)** | Data definition language | Machine instruction code | Node configuration map | Security permission table | Thread dispatch sequence | Logic condition table | +| **User (Operational Instance)** | Database records | Executing system processes | Live operational network | Authenticated actors | Real-time events | Business operational value | + +[[CAPTION:Table]] The Zachman Enterprise Architecture Framework taxonomy matrix. + +While rarely implemented in its complete 36-cell form today, Zachman remains invaluable as a conceptual checklist to guarantee that no architectural perspective is neglected. + +### 3.3 Kruchten's 4+1 View Model + +Formulated by Philippe Kruchten, the 4+1 View Model structures software architecture into five distinct viewpoints, each addressing concerns of specific stakeholders: + +```mermaid +flowchart TD + subgraph Views["The 4+1 Architectural Views"] + UC[Use Case View: Scenarios & End-User Needs] + LV[Logical View: Domain Classes & Subsystems] + DV[Development View: Packages, Modules & Code Organisation] + PV[Process View: Concurrency, Threads & Run-Time Tasks] + PhV[Physical View: Hardware, Network & Deployment Topology] + + UC --> LV + UC --> DV + UC --> PV + UC --> PhV + end +``` +[[CAPTION:Figure]] Kruchten's 4+1 Architectural View Model. + +- **Logical View**: Concerned with system functionality exposed to end-users (class models, domain objects). +- **Process View**: Addresses runtime concurrency, non-blocking tasks, inter-thread synchronization, and system availability. +- **Development View**: Focuses on the actual software assets, source code repository organisation, packages, and build systems. +- **Physical View**: Details hardware components, network infrastructure, edge processors, and cloud execution environments. +- **Use Case (+1) View**: Ties the four views together by demonstrating how scenarios and operational workflows traverse the architecture. + +### 3.4 The Arc42 Architecture Template + +Arc42 is an open-source, pragmatic documentation template widely adopted across European and global engineering teams. It decomposes software architecture into 12 structured sections: +1. Introduction and Goals (Business goals, requirements, stakeholders) +2. Architecture Constraints +3. Context and Scope (Business and technical context) +4. Solution Strategy +5. Building Block View (Decomposition hierarchy) +6. Runtime View (Scenarios, state machines, sequences) +7. Deployment View (Infrastructure, nodes, network) +8. Cross-cutting Concepts (Security, error handling, logging, testing) +9. Architecture Decisions (ADR records) +10. Quality Requirements (Quality tree, scenarios) +11. Risks and Technical Debt +12. Glossary + +Arc42 maps seamlessly into the Docs-as-Code methodology ([Documentation as code and terminology](../guidelines/documentation-as-code.md)) and Markdown-based documentation repositories. + +## Summary + +Architecture is not an isolated deliverable or an academic exercise. It is the living, evolving set of structural decisions and boundaries that enable engineering teams to move with speed, safety, and autonomy. + +In the subsequent chapter, we will explore how to establish an actionable Architecture Vision, assess engineering competencies using the SFIA framework, and enforce quality attributes through governance. diff --git a/content/handbook/02-architecture-vision-and-principles.md b/content/handbook/02-architecture-vision-and-principles.md new file mode 100644 index 0000000..244d276 --- /dev/null +++ b/content/handbook/02-architecture-vision-and-principles.md @@ -0,0 +1,154 @@ +--- +title: Architecture vision and principles +--- + +# Architecture vision and principles + +A successful engineering practice needs more than individual technical skill. It needs a coherent Architecture Vision and a repeatable governance mechanism. It also needs structured professional development, so that architectural thinking scales across the organisation. + +## 1. Defining the Architecture Vision + +An Architecture Vision articulates the future-state technical capabilities of an organisation, illustrating how technology enables strategic commercial objectives while systematically dismantling technical debt. + +```mermaid +flowchart LR + A[Business Strategy & Market Drivers] --> B[Architecture Vision & Tenets] + B --> C[Target State Architecture] + D[Current State Technical Baseline] --> E[Gap Analysis & Technical Debt] + C --> F[Strategic Transformation Roadmap] + E --> F + F --> G[Iterative Delivery Sprints] +``` +[[CAPTION:Figure]] From business strategy to iterative delivery via Architecture Vision. + +### Elements of an Actionable Architecture Vision + +1. **Guiding Tenets**: A concise set of non-negotiable principles that guide architectural decisions when competing trade-offs arise (for example, "API-first: every service exposes capabilities via standardized contracts before building user interfaces"). +2. **Current State Baseline**: An honest, objective inventory of the existing technical estate, legacy limitations, defect hot-spots, and security vulnerabilities. +3. **Target State Definition**: Clear models (C4 context and container diagrams per [GD-001](../guides/c4-model/index.md)) depicting the desired architecture three to five years in the future. +4. **Transition Roadmap**: Phased milestones (Now, Next, Later) delivering tangible business value at each step while deprecating obsolete subsystems. + +## 2. Professional Capability Frameworks: SFIA + +To build and sustain high-performing architecture and engineering teams, organisations must adopt structured skills frameworks. The **Skills Framework for the Information Age (SFIA)** is the globally recognized taxonomy for digital and technical skills. + +### The Seven SFIA Responsibility Levels + +SFIA defines seven distinct levels of professional responsibility: + +| Level | Title | Guiding Description | Autonomy and Influence | +| :--- | :--- | :--- | :--- | +| **Level 1** | Follow | Works under direct supervision; uses basic skills and routine procedures. | Minimal autonomy; follows established instructions. | +| **Level 2** | Assist | Works under routine supervision; exercises limited discretion. | Participates in work packages; collaborates with peers. | +| **Level 3** | Apply | Performs varied work activities in a range of contexts. | Plans and schedules own work within given deadlines. | +| **Level 4** | Enable | Performs broad and complex activities; provides technical direction to others. | Self-directed; guides junior engineers; resolves escalations. | +| **Level 5** | Ensure / Advise | Influences organisation, customers, and partners; authoritative subject matter expert. | Full accountability for technical outcomes of work streams. | +| **Level 6** | Initiate / Influence | Shapes organizational strategy; leads major technical disciplines. | Champions technical innovation; influences executive leadership. | +| **Level 7** | Set Strategy / Inspire | Directs enterprise technology vision; sets industry benchmarks. | Highest organizational level; accountable for corporate tech strategy. | + +[[CAPTION:Table]] The seven SFIA levels of professional responsibility. + +### Core Architecture and Engineering Skills in SFIA + +SFIA categorises technical competencies into specific skill codes relevant to architecture and systems engineering: + +```mermaid +flowchart TD + subgraph ArchitectureSkills["Core SFIA Architecture & Design Skills"] + ARCH["Solution Architecture (ARCH) + Levels 5 - 7"] + STPL["Enterprise & Business Architecture (STPL) + Levels 5 - 7"] + DESN["Systems Design (DESN) + Levels 4 - 6"] + PROG["Software Development (PROG) + Levels 2 - 5"] + ETMG["Emerging Technology Monitoring (ETMG) + Levels 4 - 6"] + end + STPL --> ARCH + ARCH --> DESN + DESN --> PROG + ETMG --> ARCH +``` +[[CAPTION:Figure]] Key SFIA skills for architecture, design, and software development. + +- **Solution Architecture (ARCH)**: Designing complete solutions that integrate business processes, applications, data, and cloud/hardware infrastructure. +- **Systems Design (DESN)**: Specifying detailed components, hardware-software interfaces, algorithms, and data storage structures. +- **Software Development (PROG)**: Authoring robust, verified production source code conforming to coding standards ([C coding standard](../standards/c-coding-standard.md), [Python for supporting tools](../standards/python.md)). +- **Emerging Technology Monitoring (ETMG)**: Researching, evaluating, and trialing new platforms, tools, and methodologies for the Technology Radar. + +## 3. Architecture Governance and Review Mechanisms + +Architecture governance ensures that projects deliver against the agreed Architecture Vision, comply with corporate security standards, and avoid introducing rogue technical debt. + +```mermaid +flowchart TD + subgraph SquadDelivery["Autonomous Delivery Squads"] + ADR[Author Architecture Decision Record] + SELF[Complete Self-Review Checklist] + end + subgraph Governance["Governance & Assurance"] + ARB[Architecture Review Board] + AUDIT[Automated CI Quality & Security Scans] + end + ADR --> SELF + SELF --> ARB + ARB --> AUDIT + AUDIT --> DEPLOY[Production Deployment] +``` +[[CAPTION:Figure]] Integrated architecture governance workflow. + +Effective governance operates across three complementary layers: + +### 3.1 Architecture Decision Records (ADRs) + +Every significant architectural decision MUST be documented as an ADR committed directly into the codebase ([Writing ADRs and RFCs](../guides/decision-records.md)). ADRs capture the business context, evaluated alternatives, and trade-off consequences at the time the decision was made. + +### 3.2 The Architecture Review Board (ARB) + +The ARB provides central oversight, cross-domain alignment, and compliance review ([Running an architecture review board](../guides/architecture-review-board.md)). Meeting on a regular cadence, the board evaluates proposed High-Level Designs against the 23-point Architecture Review Checklist ([Architecture review checklist](../templates/architecture-review-checklist.md)). + +### 3.3 Automated Architectural Fitness Functions + +Governance cannot rely solely on manual reviews. Modern architectures employ automated tests (fitness functions) executed within CI pipelines: +- Dependency graph analysers enforcing layered isolation (for example, verifying domain logic never imports database or HTTP transport modules per [Software architecture principles](../standards/software-architecture-principles.md)). +- Binary size and memory footprint tracking for embedded microcontroller images. +- Automated API contract linters verifying OpenAPI schemas and RFC 9457 error payload conformance ([API design and error handling](../standards/api-design.md)). + +## 4. Quality Attributes and Quantitative Metrics + +Quality attributes (non-functional requirements) define the behavioural properties of a system. The **ISO/IEC 25010** software product quality model establishes eight primary quality characteristics: + +| Characteristic | Architectural Focus | Quantitative Verification Metric | +| :--- | :--- | :--- | +| **Performance Efficiency** | Response latency, throughput capacity, CPU and RAM resource utilisation. | 99th percentile response time < 200 ms; CPU utilisation < 65% at peak load. | +| **Compatibility** | Interoperability across heterogeneous devices and protocol co-existence. | Full compliance with standard schemas (OpenAPI 3.1, LwM2M Object Registry). | +| **Usability** | Developer experience (DX), intuitive API naming, clear documentation. | Time-to-first-successful-API-call < 15 minutes. | +| **Reliability** | Availability, fault tolerance, MTBF, automatic recovery. | 99.95% monthly availability; MTTR < 5 minutes via automated failover. | +| **Security** | Confidentiality, integrity, non-repudiation, authenticated access control. | Zero high/critical CVEs; TLS 1.3 encryption with Mutual TLS for device auth. | +| **Maintainability** | Modularity, testability, reusability, cyclomatic complexity. | Unit test coverage >= 85%; cyclomatic complexity per function <= 10. | +| **Portability** | Adaptability across hardware variants and cloud deployment environments. | Compilation across ARM GCC and Clang with zero warnings; containerized microservices. | +| **Functional Suitability** | Completeness, correctness, and appropriateness of core business capabilities. | 100% requirements traceability verified against automated system integration tests. | + +[[CAPTION:Table]] ISO/IEC 25010 Quality Characteristics and quantitative engineering metrics. + +### Architecture Trade-off Analysis Method (ATAM) + +Software architecture is fundamentally the management of trade-offs. The SEI Architecture Trade-off Analysis Method (ATAM) provides a structured workshop format to evaluate candidate architectures against competing quality attributes: + +```mermaid +flowchart LR + A[Business Drivers] --> B[Utility Tree Generation] + B --> C[Quality Attribute Scenarios] + C --> D[Architectural Approaches] + D --> E[Trade-off Analysis & Sensitivity Points] + E --> F[Risk Register & Decision Records] +``` +[[CAPTION:Figure]] The Architecture Trade-off Analysis Method (ATAM) flow. + +A **Utility Tree** turns high-level concerns such as "System must be secure" into precise, testable scenarios. An example is "When an unauthorised client sends an unauthenticated MQTT packet, the broker drops the connection within 50 ms and increments a security audit metric". + +## Summary + +Architecture vision, professional capability frameworks, and rigorous governance create an engineering environment where technical quality is built into the culture. In Chapter 3, we turn to the structural patterns that implement these requirements. diff --git a/content/handbook/03-architecture-patterns.md b/content/handbook/03-architecture-patterns.md new file mode 100644 index 0000000..41ad454 --- /dev/null +++ b/content/handbook/03-architecture-patterns.md @@ -0,0 +1,245 @@ +--- +title: Architecture patterns +--- + +# Architecture patterns + +Architectural patterns represent reusable, proven solutions to recurring design challenges across distributed systems, firmware runtimes, and enterprise platforms. Selecting the appropriate architectural pattern establishes the structural foundation for system scalability, maintainability, and resilience. + +## 1. Architectural Styles vs Design Patterns + +While design patterns (such as Factory, Strategy, or Observer) operate at the level of classes, functions, and objects within a single code module, **architectural styles and patterns** govern: +- Coarse-grained subsystem boundaries and execution environments. +- Inter-process and inter-network communication mechanisms. +- Data persistence strategies and consistency boundaries. +- Fault containment and deployment isolation. + +```mermaid +flowchart TD + subgraph Patterns["Spectrum of Architectural Patterns"] + MONO[Monolith & Modular Monolith] + LAYER[Layered / Tiered Architecture] + HEX[Hexagonal / Clean Architecture] + MICRO[Microservices Architecture] + EDA[Event-Driven & Pub/Sub] + EDGE[Edge Computing & IoT Mesh] + end +``` +[[CAPTION:Figure]] Spectrum of core architectural patterns. + +## 2. Monolithic vs Modular Monolith Architectures + +### The Traditional Monolith + +In a classic monolithic architecture, all functional business logic, database queries, and presentation logic are packaged and deployed as a single executable artefact. + +- **Strengths**: Simple initial deployment, straightforward local development, zero inter-process network latency, easy transaction boundaries. +- **Vulnerabilities**: Risk of becoming a "Big Ball of Mud", tight coupling across modules, single point of failure (a crash in one module brings down the entire process), and scaling bottlenecks (entire binary must be scaled horizontally). + +### The Modular Monolith + +A **Modular Monolith** preserves the single-deployment benefit of a monolith while enforcing strict, compile-time module boundaries. Each business domain resides in an isolated package, communicating with other modules strictly through published in-process interfaces rather than querying private database tables directly. + +```mermaid +flowchart TD + subgraph ModularMonolith["Modular Monolith Boundary"] + API[Unified Ingress Gateway / HTTP API] + subgraph ModA["Device Management Module"] + A_Logic[Device Core Logic] + A_DB[(Device Tables)] + end + subgraph ModB["Telemetry Ingestion Module"] + B_Logic[Telemetry Pipeline] + B_DB[(Telemetry Tables)] + end + subgraph ModC["Alerting & Notification Module"] + C_Logic[Alert Dispatcher] + C_DB[(Alert Tables)] + end + API --> A_Logic + API --> B_Logic + API --> C_Logic + A_Logic <-->|Published In-Process API| B_Logic + B_Logic <-->|Internal Event Bus| C_Logic + end +``` +[[CAPTION:Figure]] Modular Monolith architecture with encapsulated private data stores. + +Modular monoliths represent the optimal starting architecture for most products, avoiding the distributed system overhead of microservices until team size or scaling demands warrant separation. + +## 3. Layered vs Hexagonal (Clean) Architecture + +### Traditional Layered Architecture + +Layered architecture divides the software into horizontal tiers (Presentation, Application/Business Logic, Persistence, and Database). + +Traditional layered systems are intuitive, but they have a serious flaw. High-level business logic often depends directly on low-level database schemas or ORM models. That breaks the Dependency Inversion Principle ([Software architecture principles](../standards/software-architecture-principles.md)). + +### Hexagonal Architecture (Ports and Adapters) + +Alistair Cockburn's Hexagonal Architecture places the core business domain at the centre, completely insulated from external technologies, frameworks, hardware peripherals, and databases. + +```mermaid +flowchart LR + subgraph DrivingAdapters["Driving Adapters (Inbound)"] + REST[REST / HTTP Controller] + MQTT[MQTT Subscription Handler] + CLI[Embedded CLI Shell] + end + subgraph Core["Core Business Domain"] + IPorts[Inbound Ports / Use Cases] + DomainModel[Entities & Business Rules] + OPorts[Outbound Ports / Interfaces] + IPorts --> DomainModel + DomainModel --> OPorts + end + subgraph DrivenAdapters["Driven Adapters (Outbound)"] + SQL[PostgreSQL Database Adapter] + FLASH[SPI Flash Storage Driver] + CLOUD[Cloud Event Publisher] + end + REST --> IPorts + MQTT --> IPorts + CLI --> IPorts + OPorts --> SQL + OPorts --> FLASH + OPorts --> CLOUD +``` +[[CAPTION:Figure]] Hexagonal Architecture (Ports and Adapters) decoupling domain logic from infrastructure. + +- **Ports**: Specification of what the domain needs (interfaces). Inbound ports define use cases; outbound ports define required services (repositories, notifiers). +- **Adapters**: Concrete implementations that translate between port interfaces and external technologies (such as translating an outbound `TelemetryStore` port to an SQLite database or SPI NOR flash driver). + +## 4. Microservices Architecture + +Microservices partition a system into small, independently deployable services organized around distinct business capabilities. Each microservice manages its own private datastore. + +| Property | Monolith / Modular Monolith | Microservices Architecture | +| :--- | :--- | :--- | +| **Deployment Unit** | Single executable or container. | Tens to hundreds of independent containers. | +| **Data Storage** | Shared single database instance. | Database-per-Service; decentralized persistence. | +| **Communication** | In-process function calls (< 1 μs). | Network RPC, HTTP, or message buses (1 to 50 ms). | +| **Failure Isolation** | Process crash affects entire application. | Failure contained to individual service. | +| **Operational Overhead** | Low: simple CI/CD and single server host. | High: requires container orchestration, service mesh, tracing. | + +[[CAPTION:Table]] Comparison between Monolithic and Microservices architectural styles. + +Microservices are justified when independent teams require autonomous release cadences or when distinct components require radically divergent scaling profiles (for example, video transcoding versus billing). + +## 5. Event-Driven Architecture (EDA) + +Event-Driven Architecture decouples message producers from consumers. Instead of synchronous request-response calls, components react asynchronously to state change notifications (events). + +```mermaid +sequenceDiagram + participant Sensor as IoT Temperature Sensor + participant Broker as MQTT Broker / Kafka + participant Telemetry as Telemetry Ingestion Service + participant Rules as Anomaly Detection Engine + participant Alert as Notification Service + + Sensor->>Broker: Publish "temperature_exceeded" (28.4°C) + Broker-->>Telemetry: Deliver Event + Broker-->>Rules: Deliver Event + Telemetry->>Telemetry: Store Time-Series Sample + Rules->>Rules: Evaluate Threshold Rule + Rules->>Broker: Publish "alarm_triggered" (Critical Severity) + Broker-->>Alert: Deliver Alarm + Alert->>Alert: Send SMS / Push Notification +``` +[[CAPTION:Figure]] Event-driven choreography across distributed IoT services. + +### Core EDA Tenets + +- **Choreography vs Orchestration**: In choreographed systems, services react independently to events published on a broker without central control. In orchestrated systems, a central workflow coordinator explicitly commands each step. +- **Event Sourcing**: Storing every state mutation as an immutable append-only event log rather than overwriting current state in a relational table. Current state is reconstructed by replaying events. +- **CQRS**: Separating command (write) models from query (read) projections to optimise data schemas for specific access patterns. + +### Message queues and publish-subscribe + +Event-driven systems rest on messaging middleware: a broker that sits between the parts of a system so that they never call each other directly. Two delivery models matter, and most brokers offer both. + +- **Queues (point to point).** A producer sends a message to a queue, and exactly one consumer takes it. Add consumers to share the work. Use a queue for commands and jobs, where each piece of work must be done once, such as processing a payment or generating a report. +- **Topics (publish-subscribe).** A publisher sends a message to a topic, and every subscriber gets its own copy. Publishers do not know who is listening, so a new subscriber can be added without changing them. Use a topic for events that several services care about, such as a device reporting a reading. + +```mermaid +flowchart LR + P[Producer] --> Q[(Queue)] + Q --> C1[Worker 1] + Q -. or .-> C2[Worker 2] + Pub[Publisher] --> T{{Topic}} + T --> S1[Subscriber A] + T --> S2[Subscriber B] +``` + +[[CAPTION:Figure]] A queue delivers each message to one consumer; a topic delivers a copy to every subscriber. + +In connected products the two often appear together. A gateway publishes device readings to an MQTT topic. A bridge forwards them into a streaming platform such as Kafka, where analytics, dashboards and alerting each subscribe independently. Commands to a device go the other way, through a queue or a per-device topic. + +Whichever model you use: + +- **Make consumers idempotent.** Most brokers deliver at least once, so a consumer will sometimes see a message twice. Processing it twice must be harmless. +- **Acknowledge after processing, not before.** A consumer that acknowledges on receipt and then crashes loses the message. +- **Plan for poison messages.** A message that always fails should go to a dead-letter queue after a few attempts, rather than block everything behind it. +- **Name topics deliberately.** A hierarchy such as `site/device-type/device-id/reading` makes routing, access control and wildcards straightforward. +- **Secure the broker.** Authenticate every client, encrypt the connection, and restrict which clients may publish or subscribe to each topic. +- **Watch queue depth and consumer lag.** A growing backlog is the earliest sign that consumers cannot keep up. + +## 6. Edge and Smart Home IoT Network Topologies + +Connected environments (such as smart buildings, industrial factories, and connected homes) require multi-tiered edge architectures that balance local real-time responsiveness with cloud intelligence. + +A production IoT edge topology isolates sensitive infrastructure, segregates untrusted smart devices, and optimizes bandwidth across six distinct network zones: + +```mermaid +flowchart TD + WAN[Internet / Cloud Ingestion Platform] <--> RT[Gateway Router & AP: 10.0.11.29] + + subgraph Zones["Segregated Local Network Zones"] + NI["Network Infra (Zone 1) + Managed Switches, APs, DNS/DHCP"] + SS["Servers & Storage (Zone 2) + Local Home Lab, NAS, Container Hosts"] + SH["Smart Home (Zone 3) + Sensors, Actuators, Matter/Zigbee Hubs"] + MS["Media & Stream (Zone 4) + Smart Displays, Audio Streamers, Cameras"] + CM["Clients & Mobile (Zone 5) + Engineer Workstations, Mobile Devices"] + UN["Unclassified & Quarantine (Zone 6) + New Devices, Untrusted IoT Probes"] + end + + RT --> NI + RT --> SS + RT --> SH + RT --> MS + RT --> CM + RT --> UN + + SH <-->|Local Pub/Sub MQTT| SS + SS -->|Filtered Telemetry| WAN +``` +[[CAPTION:Figure]] Production multi-zone network topology for smart home and edge IoT environments. + +### Zone Isolation Rules + +1. **Zone 1 (Network Infra)**: Core routing, PoE switches, and network management consoles. Access restricted to dedicated administrative credentials. +2. **Zone 2 (Servers & Storage)**: Local Kubernetes/Docker clusters, home lab instances, telemetry aggregators, and Network Attached Storage (NAS). Acts as the local edge compute layer. +3. **Zone 3 (Smart Home)**: Environmental sensors, smart thermostatic radiator valves (TRVs), light switches, and protocol gateways (Matter, Zigbee, Thread). Isolated on a dedicated VLAN with restricted internet egress. +4. **Zone 4 (Media & Stream)**: High-bandwidth multicast devices (IP security cameras, smart TVs, audio streamers). Isolated to prevent buffer-bloat on sensor networks. +5. **Zone 5 (Clients & Mobile)**: Trusted laptops, workstations, and smartphones accessing secure management portals. +6. **Zone 6 (Quarantine / Unclassified)**: Newly provisioned or unverified devices subjected to automated port scanning, behavioural analysis, and containment prior to admission into production VLANs. + +### Edge Store-and-Forward Pattern + +IoT edge nodes must operate reliably during intermittent WAN outages: +- **Local Ring Buffers**: Telemetry is written to local non-volatile storage (flash or embedded SQLite). +- **Graceful Degradation**: Core control loops (such as thermostat heating schedules or motion-triggered lights) execute locally with zero cloud dependence. +- **Batch Upload with Jitter**: When WAN connectivity resumes, edge gateways stream queued data using exponential backoff to avoid overwhelming cloud ingestion pipelines ([API design and error handling](../standards/api-design.md)). + +## Summary + +No single pattern suits every problem. World-class architectures blend Hexagonal Architecture for core software modules, Modular Monoliths or Microservices for service deployment, and multi-tier Edge topologies for connected device fleets. + +In Chapter 4, we examine how these distributed systems communicate via modern API standards. diff --git a/content/handbook/04-api-design-and-management.md b/content/handbook/04-api-design-and-management.md new file mode 100644 index 0000000..8b16b06 --- /dev/null +++ b/content/handbook/04-api-design-and-management.md @@ -0,0 +1,139 @@ +--- +title: API design and management +--- + +# API design and management + +Application Programming Interfaces (APIs) form the digital contracts connecting microservices, embedded edge appliances, mobile applications, and third-party partners. Robust API engineering demands clear protocol selection, disciplined schema governance, and systematic serialisation design. + +## 1. Modern API Paradigms: A Comparative Analysis + +Different communication paradigms optimise for different operational trade-offs (such as latency, payload size, developer ergonomics, and streaming support): + +| Paradigm | Transport | Serialisation | Schema Contract | Ideal Operational Use Cases | +| :--- | :--- | :--- | :--- | :--- | +| **REST** | HTTP/1.1 or HTTP/2 | JSON (RFC 8259) | OpenAPI 3.1 | Public APIs, CRUD management, browser-to-server communications. | +| **gRPC** | HTTP/2 | Protocol Buffers | `.proto` files | High-throughput inter-service microservice RPC, polyglot back-end communications. | +| **GraphQL** | HTTP/1.1 or HTTP/2 | JSON | GraphQL SDL | Complex front-end client queries, aggregations across multiple microservices. | +| **AsyncAPI** | MQTT, Kafka, AMQP | JSON, Protobuf, Avro | AsyncAPI 3.0 | Event-driven telemetry, IoT pub/sub streaming, distributed background workers. | +| **WebSocket** | TCP / WS | Binary / Text | Application-defined | Full-duplex real-time dashboards, financial tickers, live bidirectional device sessions. | + +[[CAPTION:Table]] Comparative analysis of modern API paradigms. + +```mermaid +flowchart TD + subgraph Clients["Consumers"] + WEB[Web Portal] + MOB[Mobile App] + DEV[IoT Edge Devices] + end + subgraph Gateway["API Gateway Layer"] + GW[Unified API Gateway & Ingress Router] + end + subgraph Backends["Backend Microservices"] + REST_SVC[Device Management: REST/JSON] + GRPC_SVC[Analytics Engine: gRPC/Protobuf] + EVENT_SVC[Telemetry Ingestion: AsyncAPI/MQTT] + end + WEB -->|REST / JSON| GW + MOB -->|GraphQL| GW + DEV -->|MQTT / TLS| EVENT_SVC + GW -->|REST| REST_SVC + GW -->|gRPC| GRPC_SVC + REST_SVC <-->|Internal gRPC| GRPC_SVC +``` +[[CAPTION:Figure]] Multi-protocol API ingress and internal microservice communication architecture. + +## 2. Data Serialisation Formats + +The choice of serialisation format directly impacts network bandwidth, CPU utilisation, and memory consumption. In resource-constrained embedded systems and high-throughput cloud streaming pipelines, binary formats provide decisive advantages over text-based JSON. + +### Comparative Serialisation Matrix + +| Format | Type | Schema Required? | Relative Size | CPU Overhead | Embedded Feasibility | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **JSON** | Text | Optional (JSON Schema) | 1.0x (Baseline) | High (String parsing, tokenization) | Feasible on 32-bit MCUs with >= 64 KB RAM; poor for 8/16-bit. | +| **Protobuf** | Binary | Yes (`.proto`) | 0.25 to 0.4x | Minimal (Bit-shift decoding, varints) | Excellent via nanopb library; minimal stack/heap footprint. | +| **CBOR** | Binary | Optional (CDDL) | 0.4 to 0.6x | Low (Type-length-value byte parsing) | Native standard for CoAP and IETF IoT specifications (RFC 8949). | +| **MessagePack** | Binary | Optional | 0.4 to 0.6x | Low | Good general-purpose binary JSON drop-in replacement. | + +[[CAPTION:Table]] Serialisation format comparison across size, performance, and embedded feasibility. + +### Protocol Buffers in Embedded Architecture + +Protocol Buffers (Protobuf) encode strongly-typed data structures into compact binary representations using variable-length integer encoding (varints) and field tags. + +```protobuf +syntax = "proto3"; + +package iot.telemetry.v1; + +message SensorSample { + string device_id = 1; + uint64 timestamp_utc_ms = 2; + float temperature_celsius = 3; + float relative_humidity_pct = 4; + uint32 battery_millivolts = 5; + + enum SensorStatus { + STATUS_UNSPECIFIED = 0; + STATUS_OK = 1; + STATUS_CALIBRATION_NEEDED = 2; + STATUS_HARDWARE_FAULT = 3; + } + SensorStatus status = 6; +} +``` +[[CAPTION:Figure]] Protobuf v3 schema defining strongly-typed IoT sensor telemetry. + +Using lightweight generators like `nanopb`, this schema compiles into plain C structs with static memory allocations, allowing microcontrollers to encode and decode messages in microseconds with zero dynamic memory allocation (`malloc`). + +## 3. API Governance and Management + +As an API estate expands across dozens of squads, centralized governance ensures consistency, security, and backward compatibility. + +### 3.1 The API Gateway Pattern + +An **API Gateway** sits between external clients and internal services, providing cross-cutting operational capabilities: +- **Authentication & Authorisation**: Validating OAuth2 JWT tokens or mTLS client certificates before traffic reaches internal services. +- **Traffic Shaping & Rate Limiting**: Enforcing quota policies (for example, 100 requests/minute per client IP) using token bucket algorithms, returning HTTP `429 Too Many Requests` when exceeded ([API design and error handling](../standards/api-design.md)). +- **Protocol Translation**: Transcoding external REST/JSON HTTP requests into internal high-speed gRPC calls. +- **Circuit Breaking and Fallbacks**: Isolating failing downstream services to prevent cascading failures. +- **Distributed Tracing & Metrics**: Injecting W3C Trace Context headers (`traceparent`) to monitor end-to-end request latencies. + +### 3.2 API Versioning Strategies + +APIs MUST manage evolutionary changes without breaking existing consumer integrations: + +| Versioning Strategy | Syntax Example | Pros | Cons | Recommendation | +| :--- | :--- | :--- | :--- | :--- | +| **URI Path Versioning** | `/api/v1/devices` | Explicit, easily tested in browsers, cache-friendly. | Couples versioning to URL structure; violates pure REST resource modelling. | **Recommended** for public and partner APIs. | +| **Header Versioning** | `Accept: application/vnd.company.v1+json` | Preserves clean URIs; semantically pure content negotiation. | Harder to test manually; proxies and caches may strip or mishandle headers. | Suitable for internal microservices. | +| **Query Parameter** | `/devices?v=1` | Easy to append dynamically. | Can cause cache poisoning or confusion with filtering query parameters. | Discouraged. | + +[[CAPTION:Table]] Comparison of API versioning strategies. + +### 3.3 The API Deprecation Lifecycle + +When introducing breaking changes, teams MUST adhere to the formal deprecation lifecycle: + +```mermaid +flowchart LR + A[Active & Supported] -->|Announce Deprecation| B[Deprecated State] + B -->|Grace Period: 6-12 Months| C[Decommissioned / Retired] +``` +[[CAPTION:Figure]] Formal API deprecation lifecycle. + +During the deprecation window: +1. The API MUST return the standard HTTP `Sunset` and `Deprecation` response headers (RFC 8594): + ```http + Deprecation: @1790000000 + Sunset: Wed, 20 Sep 2027 12:00:00 GMT + Link: ; rel="successor-version" + ``` + [[CAPTION:Figure]] RFC 8594 Deprecation and Sunset headers. +2. The team must monitor API gateway metrics to actively identify consumers still calling deprecated endpoints and assist their migration. + +## Summary + +Disciplined API design ensures systems remain decoupled, discoverable, and performant. In Chapter 5, we explore how cloud platforms provide the computing, streaming, and infrastructure-as-code foundation for these APIs. diff --git a/content/handbook/05-cloud-architecture.md b/content/handbook/05-cloud-architecture.md new file mode 100644 index 0000000..ce415db --- /dev/null +++ b/content/handbook/05-cloud-architecture.md @@ -0,0 +1,150 @@ +--- +title: Cloud architecture +--- + +# Cloud architecture + +Cloud computing transforms software architecture from static server management into dynamic, programmable utility services. Modern cloud architecture demands robust multi-cloud strategies, serverless elasticity, real-time event streaming pipelines, and automated Infrastructure as Code (IaC). + +## 1. Cloud Service Models: Strategic Trade-Offs + +Architects must match system requirements against the appropriate cloud abstraction tier: + +| Cloud Model | Management Responsibility | Architectural Strengths | Trade-Offs & Constraints | +| :--- | :--- | :--- | :--- | +| **IaaS** (Infrastructure as a Service) | OS, Runtime, Middleware, Application, Data. | Complete control; direct kernel and network tuning. | High operational maintenance; manual patching and scaling. | +| **CaaS** (Containers as a Service / K8s) | Application container, Config, Cluster sizing. | Portability; declarative scaling; uniform developer workflow. | Kubernetes operational complexity; control plane overhead. | +| **PaaS** (Platform as a Service) | Application code and configuration only. | Rapid time-to-market; automated patching and scaling. | Platform lock-in; constrained runtime customisation. | +| **Serverless (FaaS)** | Pure functional logic and event bindings. | Zero idle cost; instant scale-to-zero; high developer agility. | Cold-start latency; execution time limits; stateless constraints. | + +[[CAPTION:Table]] Comparison of cloud service delivery models. + +```mermaid +flowchart TD + subgraph CloudHierarchy["Cloud Responsibility Layers"] + direction TB + APP[Application Code & Business Logic] + DATA[Data Schemas & State Storage] + RUN[Runtime & Framework Dependencies] + OS[Operating System & Kernel Patches] + NET[Virtual Network & Firewalls] + PHY[Physical Hardware & Datacenters] + end +``` +[[CAPTION:Figure]] Shared responsibility stack across cloud models. + +## 2. Hybrid and Multi-Cloud Architecture + +Operating across multiple cloud providers (such as AWS, Google Cloud Platform, Microsoft Azure) or bridging on-premises edge clusters with public clouds provides critical strategic advantages: +- **Resilience and High Availability**: Eliminating single-vendor regional outages. +- **Regulatory Compliance and Data Sovereignty**: Retaining sensitive citizen or customer data within local jurisdiction boundaries (such as UK GDPR). +- **Workload Optimisation**: Leveraging specialized vendor strengths (for example, BigQuery and Vertex AI on GCP, edge device provisioning on Azure IoT Hub). + +### Achieving Workload Portability + +To avoid crippling vendor lock-in, multi-cloud architectures employ three design patterns: +1. **Containerized Abstraction**: Packaging services as Open Container Initiative (OCI) images deployed on managed Kubernetes (GKE, EKS, AKS). +2. **Database Decoupling**: Utilising open-source database engines (PostgreSQL, Redis, ClickHouse) supported across all major cloud providers or deployable via managed operators. +3. **Multi-Cloud IaC**: Defining infrastructure using vendor-agnostic declarative tools (Terraform / OpenTofu) with modular abstractions. + +## 3. Serverless Architectures and Patterns + +Serverless computing executes code in ephemeral execution containers triggered by asynchronous events or HTTP requests, scaling down to zero when idle. + +```mermaid +flowchart LR + TRIG[Event Triggers: S3 Upload / Cloud Pub-Sub / Webhook] --> FN[Serverless Function Container] + FN --> DB[(Managed Cloud Database)] + FN --> MSG[Outbound Event Topic] +``` +[[CAPTION:Figure]] Serverless event execution flow. + +### Critical Serverless Engineering Disciplines + +- **Mitigating Cold Starts**: In latency-sensitive APIs, language choice significantly impacts cold-start delays. Compiled binaries (Go, Rust) or optimized C runtimes initiate within 10 to 50 ms, whereas JVM or Python runtimes requiring large dependency trees can incur 500 to 2,500 ms cold starts. +- **Idempotency by Design**: Serverless event triggers guarantee *at-least-once* delivery. Function logic MUST be strictly idempotent, recording processed transaction IDs in a distributed cache (such as Redis or DynamoDB with conditional writes) to discard duplicate events safely. +- **Concurrency Ceilings and Backpressure**: Serverless functions can scale to thousands of concurrent instances within seconds, easily overwhelming downstream relational databases. Serverless tiers must sit behind queue-based rate limiters or connection poolers. + +## 4. High-Throughput Event Streaming: Apache Kafka + +Modern distributed platforms ingest, process, and analyse millions of telemetry events per second. Distributed log-centric streaming architectures (such as Apache Kafka and GCP Pub/Sub) provide the core infrastructure for real-time analytics and event-driven microservices. + +```mermaid +flowchart LR + subgraph Producers["Event Producers"] + P1[IoT Ingestion Gateway] + P2[User Web Service] + end + subgraph KafkaCluster["Kafka Cluster: Telemetry Topic"] + direction TB + Part0["Partition 0: Devices A - H"] + Part1["Partition 1: Devices I - P"] + Part2["Partition 2: Devices Q - Z"] + end + subgraph Consumers["Consumer Groups"] + subgraph Group1["Telemetry Store Group"] + C1[Worker 1] + C2[Worker 2] + end + subgraph Group2["Real-Time Anomaly Group"] + C3[Worker 3] + C4[Worker 4] + end + end + P1 --> Part0 + P1 --> Part1 + P2 --> Part2 + Part0 --> C1 + Part1 --> C2 + Part0 --> C3 + Part1 --> C4 +``` +[[CAPTION:Figure]] Kafka partition distribution and independent consumer group scaling. + +### Key Streaming Concepts + +- **Partitions and Ordering Guarantees**: Messages sent to Kafka with the same partition key (such as `device_id`) are guaranteed to land on the same partition and be consumed in strict temporal order. Different partition keys process in parallel. +- **Consumer Groups and Event Replay**: Multiple independent consumer groups read from the same log independently. If an analytics algorithm requires re-training, consumers can reset their offset to replay weeks of historical data without impacting real-time services. +- **Standardized Envelopes: CloudEvents**: Systems SHOULD package events using the CNCF CloudEvents specification, standardizing metadata attributes (`id`, `source`, `specversion`, `type`, `datacontenttype`, `time`): + +```json +{ + "specversion": "1.0", + "type": "com.company.telemetry.sensor.thresholdExceeded", + "source": "/devices/blackburn/gw-042", + "id": "e84d2b45-1234-5678-9abc-def012345678", + "time": "2026-09-20T12:30:00Z", + "datacontenttype": "application/json", + "data": { + "temperature_c": 31.8, + "threshold_c": 30.0, + "duration_sec": 45 + } +} +``` +[[CAPTION:Figure]] CloudEvents v1.0 standard JSON event envelope. + +## 5. Infrastructure as Code (IaC) and GitOps + +Manual server configuration via cloud web consoles ("ClickOps") is strictly prohibited in production environments. Infrastructure MUST be declared as version-controlled code. + +```mermaid +flowchart LR + DEV[Engineer PR] --> GIT[Git Repository: IaC Code] + GIT --> CI[Automated Plan & Security Scan] + CI --> APPROVE[Peer Review & ARB Sign-Off] + APPROVE --> MERGE[Merge to Main] + MERGE --> GITOPS[GitOps Engine: ArgoCD / Terraform Cloud] + GITOPS --> CLOUD[Production Cloud Resources] +``` +[[CAPTION:Figure]] GitOps declarative infrastructure delivery pipeline. + +### Core IaC Tenets + +1. **Declarative State**: Define the desired end-state infrastructure in code (Terraform, OpenTofu, Bicep); the engine computes the execution graph to transition from current state to desired state. +2. **Immutable Infrastructure**: Cloud servers and container hosts are never patched in-place. Updates deploy newly minted container images or virtual machine images; old instances are terminated. +3. **Drift Detection**: Automated scheduled CI jobs scan live cloud environments against the committed code baseline, immediately alerting or automatically reconciling unapproved out-of-band modifications. + +## Summary + +Cloud architecture provides the elastic compute, robust streaming, and automated deployment foundation necessary for modern connected software systems. In Chapter 6, we delve into the physical edge: Embedded Systems and IoT architecture. diff --git a/content/handbook/06-iot-and-embedded-systems.md b/content/handbook/06-iot-and-embedded-systems.md new file mode 100644 index 0000000..5e59076 --- /dev/null +++ b/content/handbook/06-iot-and-embedded-systems.md @@ -0,0 +1,135 @@ +--- +title: IoT and embedded systems +--- + +# IoT and embedded systems + +Embedded and Internet of Things (IoT) systems inhabit the intersection of physical hardware, real-time operating constraints, and distributed cloud connectivity. Designing robust connected appliances demands deep mastery of hardware-software co-design, real-time scheduling, constrained networking protocols, and cryptographic hardware security. + +## 1. Embedded Runtimes: Microcontrollers to Embedded Linux + +Architects must select the appropriate runtime environment based on processing requirements, memory budgets, power sources, and determinism: + +| Tier | Typical Hardware | Memory Envelope | Operating Runtime | Determinism | Target Use Cases | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **Microcontroller (MCU)** | ARM Cortex-M0+/M3/M4/M33 (for example, Nordic nRF52, STM32) | 32 to 512 KB RAM, 256 KB to 2 MB Flash | Bare-Metal or RTOS (FreeRTOS, Zephyr) | Hard Real-Time (< 10 μs interrupt response) | Battery-powered environmental sensors, BLE beacons, motor control. | +| **Mid-Tier Gateway** | Dual-core Cortex-M7, ESP32-S3 | 2 to 16 MB PSRAM, 4 to 32 MB Flash | Real-Time OS (Zephyr, FreeRTOS) | Soft Real-Time | Local protocol gateways, smart thermostats, Matter/Thread border routers. | +| **Edge Compute Appliance** | ARM Cortex-A53/A72 (for example, NXP iMX8, Raspberry Pi CM4) | 1 to 8 GB RAM, 8 to 64 GB eMMC | Embedded Linux (Yocto, Buildroot) | Non Real-Time (Milliseconds) | Video analytics, local container execution, multi-camera NVRs. | + +[[CAPTION:Table]] Spectrum of embedded computing runtimes and hardware tiers. + +```mermaid +flowchart TD + subgraph Continuum["The IoT Edge-to-Cloud Continuum"] + SENS["Constrained Sensor (FreeRTOS / Zephyr) + Battery-Powered, Microamp Sleep"] + GW["Edge Gateway (Linux / Yocto) + Local Storage, Protocol Translation"] + CLOUD["Cloud IoT Platform + Device Shadows, Kafka Ingestion"] + end + SENS <-->|BLE / Zigbee / Thread| GW + GW <-->|MQTT / LwM2M over Cellular / TLS| CLOUD + SENS -.->|Direct NB-IoT / LwM2M| CLOUD +``` +[[CAPTION:Figure]] The end-to-end IoT Edge-to-Cloud continuum. + +## 2. Real-Time Operating Systems (RTOS) Architecture + +In resource-constrained MCUs, an RTOS provides deterministic multitasking, thread synchronization, and inter-task communication within a footprint under 20 KB. + +### Core RTOS Architecture (FreeRTOS and Zephyr) + +An RTOS replaces complex superloops with prioritized, preemptive tasks: + +```mermaid +flowchart TD + subgraph Tasks["Preemptive Priority Tasks"] + T1["Task 1: Sensor Sampling (Priority: High) + Wakes every 1000ms on timer"] + T2["Task 2: Comms Stack (Priority: Medium) + Blocks on Network RX Queue"] + T3["Task 3: Diagnostics & CLI (Priority: Low) + Executes in idle cycles"] + end + subgraph Sync["RTOS IPC Primitives"] + QUEUE[Thread-Safe Message Queue] + MUTEX[Mutex with Priority Inheritance] + SEM[Counting Semaphore] + end + T1 -->|Post Sample| QUEUE + QUEUE -->|Read Sample| T2 + T2 <-->|Protect SPI Bus| MUTEX +``` +[[CAPTION:Figure]] RTOS preemptive task concurrency and IPC primitives. + +### Critical Embedded Concurrency Disciplines + +1. **Avoid Dynamic Memory Allocation (`malloc`)**: Embedded firmware operating continuously for years MUST avoid `malloc()` and `free()` after system initialisation. Heap fragmentation causes sudden, unpredictable allocation failures. All task stacks, queues, and message buffers must be statically allocated at compile time. +2. **Priority Inversion and Priority Inheritance**: When a low-priority task holds a mutex required by a high-priority task, an intermediate-priority task can preempt the low-priority task, starving the high-priority task indefinitely. Embedded mutexes MUST employ **Priority Inheritance** to temporarily elevate the holder's priority. +3. **Interrupt Service Routine (ISR) Deferred Processing**: ISRs must remain minimal, executing in under 10 μs. The ISR clears the peripheral interrupt flag, captures raw register data, posts the data to an RTOS queue, and exits immediately, deferring processing to an application task. + +## 3. IoT Constrained Communication Protocols + +Connecting embedded edge devices to external networks requires communication protocols designed for low bandwidth, high packet loss, and constrained energy reserves: + +| Protocol | Transport | Architecture | Header Overhead | QoS Levels | Ideal Use Case | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **MQTT** | TCP / TLS | Pub/Sub via Central Broker | 2 Bytes minimum | QoS 0, 1, 2 | Continuous bidirectional telemetry and remote command streaming. | +| **CoAP** | UDP / DTLS | Request/Response (REST-like) | 4 Bytes | Confirmable / Non-Confirmable | Highly constrained networks, battery devices sleeping between transmissions. | +| **LwM2M** | UDP/DTLS via CoAP | Client-Server Object Model | Compact (TLV/CBOR) | Inherits CoAP | Comprehensive device management, firmware updates (FOTA), telemetry. | +| **HTTP/REST** | TCP / TLS | Request/Response | Hundreds of Bytes | None (TCP retransmits) | Unsuitable for constrained MCUs; standard for edge gateways with abundant power. | + +[[CAPTION:Table]] Constrained IoT communication protocol comparison. + +### Lightweight M2M (LwM2M) and the IPSO Object Model + +LwM2M (standardised by the Open Mobile Alliance) provides an interoperable device management layer on top of CoAP: + +```mermaid +flowchart TD + subgraph Objects["Standard LwM2M Object Registry"] + OBJ0["Object 0: Security Credentials"] + OBJ1["Object 1: Server Configuration"] + OBJ3["Object 3: Device Information (Serial, Battery)"] + OBJ5["Object 5: Firmware Update (FOTA)"] + OBJ3303["Object 3303: IPSO Temperature Sensor"] + end + CLIENT[LwM2M Client: Nordic nRF9160] <-->|CoAP / DTLS| SERVER[Cloud LwM2M Server / Leshan] + CLIENT --- Objects +``` +[[CAPTION:Figure]] LwM2M Object model for device management and telemetry, following the LwM2M objects standard. + +Every sensor reading, configuration parameter and firmware state maps to a standard object and resource ID ([Designing LwM2M objects and resources](../standards/lwm2m-objects.md)). For example, `/3303/0/5700` is the current temperature in degrees Celsius. + +## 4. Hardware Security and Cryptographic Chain of Trust + +IoT edge devices are physically accessible to adversaries. Security cannot rely solely on software obfuscation; it requires hardware-enforced protection. + +```mermaid +flowchart TD + subgraph ChainOfTrust["Cryptographic Chain of Trust"] + ROM["Immutable Mask ROM (BootROM) + Hardware Root of Trust"] + BL1["First Stage Bootloader + Validated against vendor public key in eFuse"] + BL2["Second Stage Bootloader (MCUboot) + Validates Firmware Image Header"] + APP["Application Firmware (FreeRTOS / Zephyr) + Executes in Secure Processing Environment"] + ROM -->|Verify Signature| BL1 + BL1 -->|Verify Signature| BL2 + BL2 -->|Verify Signature| APP + end +``` +[[CAPTION:Figure]] Secure Boot cryptographic chain of trust. + +### Essential IoT Security Pillars + +1. **Hardware Root of Trust (RoT)**: Cryptographic public keys and device identity certificates are burned permanently into write-once silicon fuses (eFuses) or stored in external tamper-resistant Secure Elements (such as Microchip ATECC608B or STSAFE). +2. **Secure Boot**: At power-on, immutable Mask ROM verifies the digital cryptographic signature (ECDSA P-256 or RSA-3072) of the first bootloader before executing it. Each stage verifies subsequent stages, guaranteeing that unauthorized or tampered firmware cannot execute. +3. **Dual-Bank FOTA (Firmware Over-The-Air)**: Flash memory is partitioned into two symmetrical banks (Slot 0: Active, Slot 1: Candidate). The new image downloads into Slot 1 and is verified cryptographically. The bootloader marks Slot 1 as pending and boots into it. If the application fails its self-test or crashes before kicking the watchdog, the bootloader automatically rolls back to Slot 0. + +## Summary + +Embedded and IoT systems architecture demands uncompromising discipline: static memory allocation, preemptive concurrency management, protocol optimisation, and hardware-backed cryptographic security. In Chapter 7, we examine the formal security frameworks and international standards that validate these systems. diff --git a/content/handbook/07-security-and-standards.md b/content/handbook/07-security-and-standards.md new file mode 100644 index 0000000..fd095e2 --- /dev/null +++ b/content/handbook/07-security-and-standards.md @@ -0,0 +1,143 @@ +--- +title: Security and standards +--- + +# Security and standards + +Security is an intrinsic quality attribute that must be engineered into systems from initial architectural conception. In modern connected environments, security failures risk customer privacy, regulatory penalties, intellectual property theft, and physical safety hazards. + +## 1. Global Security Frameworks: NIST CSF and OWASP + +Enterprise and systems security architecture aligns against established international frameworks providing structured control catalogs: + +### NIST Cybersecurity Framework (CSF 2.0) + +The NIST CSF organizes security practices across six core functional domains: + +```mermaid +flowchart TD + subgraph NIST["NIST CSF 2.0 Core Functions"] + GV["Govern (GV) + Strategy, Policies & Risk Management"] + ID["Identify (ID) + Asset Inventory & Vulnerability Scans"] + PR["Protect (PR) + Access Control, Encryption & Training"] + DE["Detect (DE) + Anomalies, SIEM & Intrusion Events"] + RS["Respond (RS) + Incident Triage, Containment & Mitigation"] + RC["Recover (RC) + Restoration, Backups & Post-Incident Learning"] + end + GV --> ID + ID --> PR + PR --> DE + DE --> RS + RS --> RC + RC --> GV +``` +[[CAPTION:Figure]] The six core functions of NIST CSF 2.0. + +### OWASP IoT Top 10 Security Vulnerabilities + +The Open Worldwide Application Security Project (OWASP) maintains the authoritative vulnerability index for connected appliances: + +| Rank | Vulnerability Category | Primary Architectural Mitigation | +| :--- | :--- | :--- | +| **I1** | Weak, Guessable, or Hardcoded Passwords | Hardware unique per-device passwords generated at manufacturing; mandatory mTLS certificates. | +| **I2** | Insecure Network Services | Close unnecessary open listening ports (Telnet, unauthenticated web servers); restrict management interfaces to secure local subnets. | +| **I3** | Insecure Ecosystem Interfaces | Enforce OAuth2 / JWT authentication, TLS 1.3 encryption, and input sanitisation across mobile apps, cloud APIs, and web consoles. | +| **I4** | Lack of Secure Update Mechanism | Enforce dual-bank FOTA with cryptographic digital signature verification (ECDSA/RSA) and anti-rollback hardware counters. | +| **I5** | Use of Insecure or Outdated Components | Maintain an automated Software Bill of Materials (SBOM) scanned continuously against the National Vulnerability Database (NVD). | +| **I6** | Insufficient Privacy Protection | Minimise Personal Identifiable Information (PII) on-device; encrypt data at rest in flash using AES-XTS or AES-GCM. | +| **I7** | Insecure Data Transfer and Storage | Enforce TLS 1.3 in transit and hardware-backed cryptographic key storage (Secure Elements / eFuses). | +| **I8** | Lack of Device Management | Implement standardized telemetry, asset tracking, remote wiping, and audit logging via LwM2M ([Designing LwM2M objects and resources](../standards/lwm2m-objects.md)). | +| **I9** | Insecure Default Settings | Ship devices with least-privilege default settings; disable debug ports (JTAG/SWD) before customer dispatch. | +| **I10** | Lack of Physical Hardening | Enforce flash encryption, tamper-detection circuitry, and potting of critical cryptographic components. | + +[[CAPTION:Table]] OWASP IoT Top 10 vulnerabilities and architectural mitigations. + +## 2. IoT Regulatory Standards: ETSI EN 303 645, UK PSTI, and EU CRA + +Global regulators have transitioned from voluntary guidelines to legally enforceable cybersecurity mandates for connected products: + +```mermaid +flowchart LR + ETSI["ETSI EN 303 645 + Technical Baseline Standard"] + PSTI["UK PSTI Act 2022 + UK Mandatory Legal Compliance"] + CRA["EU Cyber Resilience Act + Mandatory CE Mark for EU Market"] + + ETSI -->|Forms technical basis of| PSTI + ETSI -->|Harmonised standard for| CRA +``` +[[CAPTION:Figure]] Relationships between international IoT standards and legislation. + +### 2.1 ETSI EN 303 645: The Global IoT Baseline + +ETSI EN 303 645 establishes 13 baseline security requirements for consumer IoT devices: +1. **No universal default passwords**: Every device must feature a unique pre-set credential or force the user to set a strong password upon initialisation. +2. **Implement a vulnerability disclosure policy**: Manufacturers must maintain a public security reporting endpoint (such as `security.txt`) and act upon reported vulnerabilities. +3. **Keep software updated**: Software updates must be verified, delivered securely, and the manufacturer must publish a defined minimum security support period. +4. **Securely store sensitive security parameters**: Cryptographic private keys and credentials must be stored in hardware-protected non-volatile storage. +5. **Communicate securely**: Encrypt all network interfaces using vetted algorithms (TLS 1.3, DTLS 1.2). +6. **Minimise exposed attack surfaces**: Disable all unused physical interfaces and network services. +7. **Ensure personal data protection**: Adhere to GDPR principles (data minimization and privacy by design). + +### 2.2 UK Product Security and Telecommunications Infrastructure (PSTI) Act + +The UK PSTI Act 2022 legally mandates three core ETSI EN 303 645 provisions for all connected consumer hardware sold in the United Kingdom: +- Elimination of universal default passwords. +- Mandatory public vulnerability disclosure mechanism. +- Published transparent end-of-support statement specifying the minimum duration of security update availability. + +### 2.3 EU Cyber Resilience Act (CRA) + +The European Cyber Resilience Act establishes sweeping mandatory requirements for all hardware and software products entering the European Single Market: +- **Mandatory CE Marking**: Non-compliant products are banned from sale within the EU. +- **Software Bill of Materials (SBOM)**: Manufacturers must deliver machine-readable SBOMs (CycloneDX or SPDX format) detailing all third-party and open-source libraries. +- **24-Hour Vulnerability Reporting**: Actively exploited zero-day vulnerabilities must be formally reported to the EU cybersecurity agency (ENISA) within 24 hours of discovery. +- **Support Lifetime Obligations**: Free security updates must be provided throughout the product's expected operational lifetime (minimum 5 years). + +## 3. The Secure Software Development Lifecycle (SSDLC) + +Security cannot be "tested into" software at the conclusion of development; it must be embedded systematically throughout the engineering lifecycle: + +```mermaid +flowchart LR + REQ[Requirements & Threat Modelling: STRIDE] --> DESIGN[Architecture Review] + DESIGN --> CODE[Static Analysis: SAST / MISRA] + CODE --> TEST[DAST & Fuzz Testing] + TEST --> DEPLOY[Signed Artifact & SBOM Generation] + DEPLOY --> OPS[Continuous Vulnerability Scanning] +``` +[[CAPTION:Figure]] Secure Software Development Lifecycle (SSDLC) pipeline. + +### 3.1 Threat Modelling via the STRIDE Methodology + +During initial architecture design, systems must be analyzed against the six STRIDE threat categories: + +| Threat | Security Property Violated | Definition and Attack Vector | Architectural Mitigation | +| :--- | :--- | :--- | :--- | +| **S**poofing | Authenticity | An adversary pretends to be an authorized device, user, or cloud service. | Mutual TLS (mTLS) with client certificates; OAuth2 JWT signatures. | +| **T**ampering | Integrity | Modifying firmware, network packets, or persistent configuration records. | Cryptographic HMAC / SHA-256 checks; Secure Boot; read-only root filesystems. | +| **R**epudiation | Non-Repudiation | An entity denies performing an action without verifiable proof. | Append-only audit logs with tamper-evident cryptographic timestamps. | +| **I**nformation Disclosure | Confidentiality | Exposing sensitive telemetry, credentials, or proprietary intellectual property. | AES-256-GCM encryption at rest; TLS 1.3 in transit; disabling debug consoles. | +| **D**enial of Service | Availability | Overwhelming CPU, network bandwidth, or memory to prevent legitimate operation. | Rate-limiting token buckets ([API design and error handling](../standards/api-design.md)); hardware watchdog timers; circuit breakers. | +| **E**levation of Privilege | Authorisation | Gaining unauthorized administrative control through software exploits. | Principle of Least Privilege; ARM TrustZone hardware isolation; non-root daemon execution. | + +[[CAPTION:Table]] STRIDE threat modelling matrix and architectural mitigations. + +### 3.2 Automated Security Tooling in CI Pipelines + +The continuous integration pipeline must enforce non-negotiable security gates: +- **Static Application Security Testing (SAST)**: Scanning source code for buffer overflows, memory leaks, and undefined behaviors (Coverity, SonarQube, MISRA C checking per [MISRA C](../standards/misra-c.md)). +- **Software Composition Analysis (SCA)**: Scanning dependencies and third-party C libraries for known Common Vulnerabilities and Exposures (CVEs) and generating CycloneDX SBOMs. +- **Fuzz Testing**: Injecting malformed, randomized packet streams into protocol parsers to uncover edge-case memory corruption faults before production deployment. + +## Summary + +Security architecture combines formal regulatory compliance with practical defence-in-depth engineering. In Chapter 8, we explore the software development best practices, documentation standards, and development methodologies that underpin software delivery excellence. diff --git a/content/handbook/08-software-development-best-practices.md b/content/handbook/08-software-development-best-practices.md new file mode 100644 index 0000000..8c3a55a --- /dev/null +++ b/content/handbook/08-software-development-best-practices.md @@ -0,0 +1,116 @@ +--- +title: Software development practices +--- + +# Software development practices + +Sustainable software systems depend on rigorous engineering practices that foster collaboration, preserve knowledge, and guarantee repeatable delivery. High-performing engineering teams combine proven design principles, pragmatic development methodologies, automated CI/CD pipelines, and Documentation as Code. + +## 1. Foundational Software Engineering Principles + +As detailed in [Fundamental Software Architecture Principles](../standards/software-architecture-principles.md), software design is governed by four timeless engineering tenets: + +- **SOLID Principles**: Single Responsibility (SRP), Open-Closed (OCP), Liskov Substitution (LSP), Interface Segregation (ISP), and Dependency Inversion (DIP). +- **Don't Repeat Yourself (DRY)**: Centralizing business logic and schemas into single, authoritative definitions. +- **Keep It Simple (KISS)**: Prioritising straightforward, readable implementations over speculative, convoluted abstractions. +- **You Aren't Gonna Need It (YAGNI)**: Implementing functionality solely when justified by verified requirements. + +## 2. Development Methodologies and Quality Gates + +Engineering velocity requires clear operational cadence and non-negotiable quality criteria: + +| Methodology | Primary Cadence | Workflow Management | Ideal Operational Fit | +| :--- | :--- | :--- | :--- | +| **Scrum** | Fixed-length iterations (1-3 weeks). | Backlog grooming, sprint planning, daily stand-up, retrospective. | Product feature teams with predictable sprint backlogs. | +| **Kanban** | Continuous flow. | Work-In-Progress (WIP) limits, cycle time optimisation. | Platform engineering, DevOps, support triage, infrastructure teams. | +| **Hybrid Scrumban** | Fixed cadence with continuous pull. | Sprint review with WIP limits on engineering boards. | Fast-paced hardware-software integration teams. | + +[[CAPTION:Table]] Comparison of core agile development methodologies. + +### Enforcing Quality Gates: DoR and DoD + +To prevent defect leakage across sprint boundaries, teams enforce two explicit quality gates: + +```mermaid +flowchart LR + A[Backlog Item] -->|Passes DoR| B[Sprint In-Progress] + B -->|Development & Verification| C[Pull Request Review] + C -->|Passes DoD| D[Done / Released] +``` +[[CAPTION:Figure]] Lifecycle progression through Definition of Ready and Definition of Done. + +- **Definition of Ready (DoR)** ([Definition of Ready](../templates/definition-of-ready.md)): A user story or task cannot enter sprint development until it satisfies explicit preconditions: clear acceptance criteria, identified dependencies, wireframes/schemas provided, and business value articulated. +- **Definition of Done (DoD)** ([Definition of Done](../templates/definition-of-done.md)): A feature cannot be closed or released until it fulfills all technical criteria: passing unit and integration tests, code review sign-off ([Code review checklist](../templates/code-review-checklist.md)), zero static analysis violations, and documentation updated. + +### Collaborative Review and Version Control Hygiene + +Professional engineering teams maintain disciplined version control workflows: +- **Git Branching Strategy**: Trunk-based development or short-lived feature branches merged via Pull Requests ([Git](../guides/git/index.md)). +- **Conventional Commits**: Structuring commit messages using standardized prefixes (`feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `ci:`) to enable automated changelog generation and semantic versioning ([Version control, commits and versioning](../guidelines/version-control-and-versioning.md)). +- **Structured Pull Requests**: Standardized PR descriptions detailing problem context, changes made, and verification steps ([Pull requests and code review](../guidelines/pull-requests-and-review.md)). + +## 3. Documentation as Code and the Diátaxis Framework + +Traditional documentation kept in disconnected corporate wikis rapidly suffers from documentation rot: out-of-sync with code, inaccessible to developers, and unreviewed. + +The **Documentation as Code (Docs-as-Code)** approach ([Documentation as code and terminology](../guidelines/documentation-as-code.md)) treats documentation with the exact same rigor as production source code: +- Stored in plain text (Markdown) within version-controlled repositories. +- Authored alongside code changes within the same Pull Request. +- Validated via automated linters in CI (Vale, markdownlint, link checkers). +- Published automatically to searchable websites and printable PDF/DOCX documents via static site generators (MkDocs Material, Pandoc). + +### The Diátaxis Documentation Framework + +To serve different reader needs, documentation MUST be structured into the four Diátaxis quadrants: + +```mermaid +flowchart TD + subgraph Diataxis["The Diátaxis Documentation Framework"] + TUT["Tutorials + Learning-oriented: Guided lessons for beginners"] + HOWTO["How-To Guides + Task-oriented: Step-by-step problem-solving recipes"] + REF["Reference + Information-oriented: Complete, objective technical facts"] + EXP["Explanation + Understanding-oriented: High-level architectural context & rationale"] + end +``` +[[CAPTION:Figure]] The four Diátaxis documentation modes. + +- **Tutorials**: Practical, step-by-step learning journeys for newcomers (for example, "Getting Started with the Smart Gateway Firmware in 30 Minutes"). +- **How-To Guides**: Direct recipes solving specific real-world tasks (for example, "How to Provision a New LwM2M Object in Zephyr RTOS"). +- **Reference**: Objective, technical specifications (for example, "Complete HTTP Status Code Reference" or "LwM2M Object Registry"). +- **Explanation**: Contextual essays illuminating architectural rationale and design choices (for example, "Why We Migrated from Monolith to Hexagonal Architecture"). + +### Diagrams as Code + +Architectural diagrams MUST NOT be maintained as static binary images (PNG/Visio) that cannot be versioned or reviewed. Systems use declarative text-based diagramming tools: +- **Mermaid**: Native client-side rendering for sequence, state, and flowcharts. +- **PlantUML**: Standard for comprehensive UML and C4 component diagrams. +- **Structurizr DSL**: Single-source-of-truth modelling across all four C4 levels ([GD-001](../guides/c4-model/index.md)). + +## 4. Continuous Integration and Testing Automation + +World-class engineering teams build robust deployment pipelines that automate quality verification from commit to deployment: + +```mermaid +flowchart LR + COMMIT[Git Commit: Conventional Format] --> LINT[Code Linter & Docs Check] + LINT --> UNIT[Unit Tests: Native Host] + UNIT --> HIL[Hardware-in-the-Loop HIL Tests] + HIL --> SAST[Security Scans: Coverity / SCA] + SAST --> BUILD[Binary Artifact & Container Build] + BUILD --> ARTIFACT[Signed Release & SBOM] +``` +[[CAPTION:Figure]] Comprehensive automated Continuous Integration pipeline. + +### The Test Automation Pyramid + +- **Unit Tests (Fast / High Volume)**: Running on native development workstations in milliseconds, validating algorithms and domain logic with isolated mocks. +- **Integration Tests (Medium Volume)**: Verifying communication contracts between microservices or RTOS tasks. +- **Hardware-in-the-Loop (HIL) Tests (Target Hardware)**: Flashing automated test firmware onto physical microcontrollers connected to automated relays, power monitors, and simulated network instruments to verify real-world hardware behaviour. + +## Summary + +Engineering excellence is the culmination of disciplined architecture, collaborative culture, and automated tooling. By adopting the principles, frameworks, and patterns set out across this handbook, teams deliver scalable, resilient, and enduring software systems. diff --git a/content/handbook/index.md b/content/handbook/index.md index 9a44684..de555e7 100644 --- a/content/handbook/index.md +++ b/content/handbook/index.md @@ -5,6 +5,71 @@ author: Mark Bacon # Architecture Handbook -Patterns, principles, API design and governance for software architecture. +Welcome to the Systems and Software Architecture Handbook. This Handbook serves as a comprehensive reference manual for architects, technical leaders, and software engineers designing, building, and operating resilient systems across embedded firmware, edge appliances, and enterprise cloud platforms. -This section is moving in from site-web-content-src's Architecture Handbook and technical-documentation's condensed version. +## Purpose and Scope + +Modern systems rarely operate in isolation. A connected smart device comprises microcontroller firmware, localized mesh networks, edge gateways, cloud ingestion pipelines, and distributed analytics services. + +Designing such systems needs one mental model that spans both ends. At one end are hardware constraints such as CPU cycles, memory and battery budgets. At the other are cloud patterns such as eventual consistency, high availability and API governance. + +This Handbook articulates proven engineering principles, architectural patterns, security standards, and governance models developed across 15+ years of strategic technical leadership. + +## Chapters + +The handbook is organised into eight thematic chapters: + +| Chapter | Title | Primary Focus | +| :--- | :--- | :--- | +| **01** | [Introduction to Architecture](01-introduction-to-architecture.md) | Architecture definitions, architect roles, and frameworks (TOGAF, Arc42, 4+1). | +| **02** | [Architecture Vision and Principles](02-architecture-vision-and-principles.md) | Vision articulation, SFIA capability models, governance, and quality metrics (ATAM). | +| **03** | [Architectural Patterns](03-architecture-patterns.md) | Layered, Hexagonal, Modular Monoliths, Microservices, Event-Driven, and Edge IoT topologies. | +| **04** | [API Design and Management](04-api-design-and-management.md) | REST, gRPC, GraphQL, AsyncAPI, binary serialisation (Protobuf, CBOR), and gateways. | +| **05** | [Cloud Architecture](05-cloud-architecture.md) | Multi-cloud strategy, serverless, event streaming (Kafka), and Infrastructure as Code. | +| **06** | [IoT and Embedded Systems](06-iot-and-embedded-systems.md) | Embedded RTOS (FreeRTOS, Zephyr), edge compute, IoT protocols, and hardware security. | +| **07** | [Security and Standards](07-security-and-standards.md) | Threat modelling, ETSI EN 303 645, ISO/IEC 30141, SSDLC, and regulatory compliance (UK PSTI, EU CRA). | +| **08** | [Software Development Best Practices](08-software-development-best-practices.md) | SOLID, DRY, KISS, YAGNI, Agile governance, Docs-as-Code, and CI/CD pipelines. | + +[[CAPTION:Table]] Architecture Handbook chapter summary and domain mapping. + +```mermaid +flowchart TD + subgraph Foundations["Foundations & Governance"] + C1[Ch 1: Introduction to Architecture] + C2[Ch 2: Vision, SFIA & Governance] + end + subgraph ArchitectureStyles["System & Architectural Styles"] + C3[Ch 3: Architectural Patterns] + C4[Ch 4: API Design & Management] + C5[Ch 5: Cloud Architecture] + C6[Ch 6: IoT & Embedded Systems] + end + subgraph QualityStandards["Assurance & Engineering Excellence"] + C7[Ch 7: Security & Standards] + C8[Ch 8: Software Best Practices] + end + Foundations --> ArchitectureStyles + ArchitectureStyles --> QualityStandards +``` +[[CAPTION:Figure]] Conceptual architecture of the handbook domains. + +## Target Audience + +This Handbook is designed for: +- **Lead and Principal Architects**: Establishing enterprise standards, defining capability frameworks, and chairing Architecture Review Boards. +- **Solution and Cloud Architects**: Designing scalable distributed systems, microservices, and event-driven data streaming architectures. +- **Embedded and Firmware Engineers**: Implementing safety-critical, resource-constrained software on microcontrollers and IoT edge gateways. +- **Engineering Managers and Technical Directors**: Aligning technology initiatives with commercial outcomes and building high-performing teams. + +## Related Guidelines and Standards + +This Handbook operates alongside our core engineering guidelines: +- [ADR Authoring Guide](../guides/decision-records.md) +- [Software Detailed Specification Standard](../guides/software-design-specification.md) +- [Docs-as-Code Standard](../guidelines/documentation-as-code.md) +- [Architecture Review Board Guide](../guides/architecture-review-board.md) +- [Architecture Review Checklist](../templates/architecture-review-checklist.md) +- [Fundamental Software Architecture Principles](../standards/software-architecture-principles.md) +- [API Design and Error Management Standards](../standards/api-design.md) +- [Defect Management and Systematic Debugging Guide](../guides/defect-management-and-debugging.md) +- [C4 Model Guide for Embedded and Connected Systems](../guides/c4-model/index.md) diff --git a/content/radar/directory.md b/content/radar/directory.md new file mode 100644 index 0000000..5ad1562 --- /dev/null +++ b/content/radar/directory.md @@ -0,0 +1,136 @@ +--- +title: Directory +--- + +# Directory + +This directory provides a comprehensive, searchable inventory of all 77 evaluated technologies, patterns, platforms, and utilities tracked on our Technology Radar. Each entry includes its assigned maturity ring, domain tags, architectural rationale, and direct links to engineering guidelines. + +For the visual interactive map, see the **[Interactive Technology Radar](index.md)**. For governance rules and ring criteria, see **[Overview & Methodology](index.md)**. + +--- + +## Radar Distribution Summary + +| Quadrant | Adopt | Trial | Assess | Hold | Total | +| :--- | :---: | :---: | :---: | :---: | :---: | +| **[Languages & Frameworks](languages-and-frameworks.md)** | 11 | 0 | 2 | 0 | **13** | +| **[Platforms & Operations](platforms-and-operations.md)** | 6 | 0 | 1 | 0 | **7** | +| **[Methods & Patterns](methods-and-patterns.md)** | 35 | 0 | 11 | 1 | **47** | +| **[Tools & Utilities](tools-and-utilities.md)** | 7 | 1 | 2 | 0 | **10** | +| **Total** | **59** | **1** | **16** | **1** | **77** | + +[[CAPTION:Table]] Distribution of technologies across quadrants and maturity rings. + +## Languages & Frameworks + +Dedicated quadrant deep-dive: **[Languages & Frameworks](languages-and-frameworks.md)** + +| # | Technology | Ring | Tags | Description & Rationale | Reference | +| :-: | :--- | :---: | :--- | :--- | :--- | +| 8 | **C** | **ADOPT** | `systems-programming`, `embedded-systems`, `performance` | C is a general-purpose, procedural programming language that has been a cornerstone of software development since its creation in the early 1970s. Known for its efficiency and control over system resources, C is particularly useful for system programming, such as operating systems, device drivers, and embedded systems. Its syntax and structure have influenced many other languages, making it a foundational language for software engineers. | [C Coding Standards (GU-009)](../standards/c-coding-standard.md) | +| 6 | **C#** | **ADOPT** | `object-oriented`, `cross-platform`, `microsoft-ecosystem` | C# is a modern, object-oriented programming language developed by Microsoft as part of its .NET initiative. It is designed for building a wide range of applications, from web and mobile apps to enterprise software and cloud-based solutions. C# is known for its ease of use, safety, and versatility, making it a popular choice among developers. | — | +| 7 | **C++** | **ADOPT** | `object-oriented`, `performance-critical`, `systems-programming` | C++ is a general-purpose programming language known for its high performance and efficiency. It supports object-oriented, procedural, and generic programming, making it versatile for a wide range of applications. C++ is particularly useful for systems programming, game development, and applications requiring real-time processing or resource-constrained environments, such as embedded systems. Its ability to directly manipulate hardware resources, coupled with extensive libraries and frameworks, makes it an essential language in areas where performance is critical. | [C Coding Standards (GU-009)](../standards/c-coding-standard.md) | +| 32 | **Go (golang)** | **ADOPT** | `concurrency`, `performance`, `simplicity` | Go, also known as golang, is a statically typed, compiled programming language designed by Google. It is renowned for its simplicity, efficiency, and strong support for concurrent programming, making it a popular choice for developing scalable and high-performance applications. Go's syntax is clean and concise, which reduces the complexity associated with managing large codebases. Its garbage collection, strict dependency management, and built-in testing capabilities enhance developer productivity and code reliability. | — | +| 34 | **GraphQL** | **ADOPT** | `api-design`, `data-fetching`, `flexibility` | GraphQL is a query language for your APIs and a runtime for executing those queries by using a type system you define for your data. It provides an alternative to REST, enabling clients to request exactly the data they need, in the shape they need it. This approach reduces over-fetching and under-fetching of data, a common problem with traditional REST APIs. | — | +| 37 | **JSON** | **ADOPT** | `data-interchange`, `lightweight-format` | JSON (JavaScript Object Notation) is a lightweight data interchange format that is easy for humans to read and write and easy for machines to parse and generate. It is language-independent but uses conventions that are familiar to programmers of the C family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. JSON is widely adopted due to its simplicity and flexibility in representing data structures, making it a staple in web development and various APIs. | — | +| 42 | **Markdown** | **ADOPT** | `documentation`, `markup-language` | Markdown is a lightweight markup language that allows users to format plain text using a simple syntax. It is widely used for creating formatted text using a plain text editor, making it a popular choice for documentation, readme files, blog posts, and more. Its simplicity and ease of use make it accessible to users who may not have extensive technical knowledge, while still being powerful enough for more complex formatting needs. | — | +| 56 | **Python** | **ADOPT** | `programming-language`, `high-level`, `versatile` | Python is a high-level, interpreted programming language known for its simplicity and readability, making it an excellent choice for beginners and experienced developers alike. Its extensive standard library and vibrant ecosystem of third-party packages enable rapid development in various domains, including web development, data analysis, artificial intelligence, and scientific computing. | [Python Standards (GU-016)](../standards/python.md) | +| 60 | **Rust** | **ADOPT** | `systems-programming`, `memory-safety`, `concurrency` | Rust is a systems programming language that aims to provide memory safety without sacrificing performance. It is designed to compete with traditional languages like C and C++ by offering powerful abstractions while ensuring safety through its unique ownership model. Rust is particularly useful in contexts requiring high performance and safety, such as embedded systems, operating systems, and game engines. Its rich type system and emphasis on concurrency make it an excellent choice for developing robust and efficient applications. | — | +| 75 | **YAML** | **ADOPT** | `data-serialization`, `configuration` | YAML, which stands for "YAML Ain't Markup Language," is a human-readable data serialization standard that is commonly used for configuration files and data interchange between programming languages. Its concise and easy-to-read syntax makes it an excellent choice for configuration files and data serialization, particularly in environments where developers and system administrators need to work closely together, such as in DevOps practices. | — | +| 55 | **protobuf** | **ADOPT** | `serialization`, `cross-platform`, `performance` | Protocol Buffers (protobuf) is a language-neutral, platform-neutral, extensible mechanism for serializing structured data. Developed by Google, it is widely adopted for its efficiency in data serialization, especially in scenarios where performance and portability are crucial. Protobuf offers a compact binary format, which reduces the size of the payload and improves the speed of data transmission compared to traditional text-based formats like JSON or XML. | [Constrained Binary Payloads Standard (GU-027)](../standards/constrained-binary-payloads.md) | +| 11 | **CertC** | **ASSESS** | `safety-critical`, `embedded-systems` | CertC is a programming language extension and a set of guidelines designed to enhance the safety and reliability of C code, particularly in safety-critical and embedded system environments. Its main purpose is to address common pitfalls in C programming, such as buffer overflows and undefined behaviors, which can lead to catastrophic failures in critical systems. By providing stricter rules and additional checks, CertC helps developers write more robust and predictable code. | — | +| 46 | **MISRA C++** | **ASSESS** | `safety-critical`, `coding-guidelines` | MISRA C++ is a set of guidelines designed to facilitate the safe and secure use of the C++ programming language in critical systems. Originally developed for the automotive industry, it has been widely adopted in other sectors, such as aerospace and industrial automation, where software reliability is paramount. MISRA C++ provides a collection of best practices that help developers avoid common pitfalls and undefined behaviors in C++ programming, which are crucial in environments where software failure can have severe consequences. | [MISRA C Guidelines (GU-012)](../standards/misra-c.md) | + +[[CAPTION:Table]] Complete directory of Languages & Frameworks blips. + +## Platforms & Operations + +Dedicated quadrant deep-dive: **[Platforms & Operations](platforms-and-operations.md)** + +| # | Technology | Ring | Tags | Description & Rationale | Reference | +| :-: | :--- | :---: | :--- | :--- | :--- | +| 17 | **Docker** | **ADOPT** | `containerization`, `devops`, `microservices` | Docker is a platform that allows developers to automate the deployment of applications inside lightweight, portable containers. These containers include everything needed to run an application, which ensures consistency across different development and production environments. Docker is particularly useful for streamlining development workflows and simplifying operations by providing a consistent environment for running applications. | — | +| 30 | **FreeRTOS** | **ADOPT** | `real-time-operating-system`, `embedded-systems`, `IoT` | FreeRTOS is a real-time operating system kernel designed for embedded devices and microcontrollers. It provides a lightweight and robust platform that supports a wide range of hardware architectures, making it ideal for resource-constrained environments common in IoT (Internet of Things) applications. FreeRTOS enables developers to create reliable, real-time applications with features like task management, inter-task communication, and memory management, all while maintaining a small footprint. | [Architecture Handbook: IoT & Embedded (GD-002)](../handbook/06-iot-and-embedded-systems.md) | +| 33 | **Google Cloud Platform** | **ADOPT** | `cloud` | GCP provides scalable cloud solutions for modern architectures. | [Architecture Handbook: Cloud (GD-002)](../handbook/05-cloud-architecture.md) | +| 49 | **MQTT** | **ADOPT** | `IoT`, `messaging-protocol` | MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe network protocol commonly used for machine-to-machine (M2M) communication and is a cornerstone in the Internet of Things (IoT) ecosystems. It is designed for connections with remote locations where a small code footprint is required, or the network bandwidth is limited. MQTT is particularly useful for its simplicity and efficiency in constrained environments, making it ideal for IoT applications where devices need to communicate reliably over potentially unreliable networks. | [IoT Fleet Operations Handbook (GD-006)](../guides/iot-fleet-operations/index.md) | +| 64 | **Serverless** | **ADOPT** | `cloud-computing`, `scalability`, `cost-efficiency` | Serverless computing is a cloud-computing execution model where the cloud provider dynamically manages the allocation and provisioning of servers. By allowing developers to run code in response to events without the complexity of managing infrastructure, serverless abstracts away the server management, enabling teams to focus more on writing code and less on server maintenance. This approach can lead to significant cost savings, as it inherently scales with demand and charges are based only on the compute time consumed. | — | +| 76 | **Yocto Project** | **ADOPT** | `linux` | Yocto Project helps build custom Linux-based systems for embedded applications. | [Architecture Handbook: IoT & Embedded (GD-002)](../handbook/06-iot-and-embedded-systems.md) | +| 13 | **CloudEvents** | **ASSESS** | `event-driven`, `interoperability` | CloudEvents is a specification for describing event data in a common way, aiming to simplify event-driven systems and improve interoperability between different services and platforms. By standardizing the structure and metadata of events, CloudEvents enables easier integration and processing across diverse environments, reducing the complexity of event-driven architectures. | — | + +[[CAPTION:Table]] Complete directory of Platforms & Operations blips. + +## Methods & Patterns + +Dedicated quadrant deep-dive: **[Methods & Patterns](methods-and-patterns.md)** + +| # | Technology | Ring | Tags | Description & Rationale | Reference | +| :-: | :--- | :---: | :--- | :--- | :--- | +| 1 | **ADRs** | **ADOPT** | `documentation`, `decision-making` | Architectural Decision Records (ADRs) are a proven method for capturing important architectural decisions made during a project. By documenting these decisions, teams can ensure that the rationale behind key architectural choices is recorded for future reference. This practice enhances communication among team members and stakeholders, providing a clear historical context that aids in understanding the evolution of a system's architecture. | [ADR Authoring Guide (GU-005)](../guides/decision-records.md) | +| 2 | **Agile** | **ADOPT** | `software-development`, `project-management` | Agile is a set of principles and practices for software development that emphasizes flexibility, collaboration, and customer satisfaction. It involves iterative development, where requirements and solutions evolve through the collaborative efforts of cross-functional teams. Agile methodologies, such as Scrum and Kanban, focus on delivering small, incremental changes quickly, allowing for frequent reassessment and adjustment. This approach enhances the ability to respond to change effectively, making it a valuable method in dynamic and complex environments. In the methods-and-patterns quadrant, Agile is placed in the adopt ring due to its widespread acceptance and proven success in improving project outcomes and fostering innovation across various industries. | [Agile Requirements & User Stories (GU-024)](../guides/agile-requirements-and-user-stories.md) | +| 4 | **AsyncAPI** | **ADOPT** | `event-driven-architecture`, `api-specification` | AsyncAPI is an open-source initiative that provides a specification for defining event-driven architectures. Similar to how OpenAPI specifies RESTful APIs, AsyncAPI focuses on asynchronous APIs, allowing developers to standardise the communication between microservices and other components in distributed systems. It is highly useful for creating a clear, machine-readable contract for services that utilise messaging systems such as Kafka, MQTT, or RabbitMQ. | — | +| 5 | **Barr Group Coding Guidelines** | **ADOPT** | `embedded-systems`, `safety-critical` | The Barr Group Coding Guidelines are a set of coding standards specifically designed for the development of embedded system software. These guidelines emphasize the importance of writing clear, maintainable, and reliable code to ensure the safety and efficiency of embedded systems, which are often used in critical applications like automotive, medical devices, and industrial control systems. | — | +| 10 | **C4** | **ADOPT** | `software-architecture`, `documentation`, `modeling` | The C4 model is a framework for visualizing the architecture of software systems. It provides a set of hierarchical diagrams, namely Context, Container, Component, and Code, that help teams understand and communicate the structure and design of their software. By using these diagrams, teams can ensure that everyone—from developers to stakeholders—has a clear and consistent understanding of how the system is structured and how its components interact. | [C4 Architecture Modelling Guide (GD-001)](../guides/c4-model/index.md) | +| 9 | **C4 Model** | **ADOPT** | `architecture` | C4 Model provides a simple, hierarchical way to visualize architecture. | [C4 Architecture Modelling Guide (GD-001)](../guides/c4-model/index.md) | +| 12 | **CICD** | **ADOPT** | `automation`, `devops`, `software-development` | Continuous Integration and Continuous Deployment (CICD) is a set of practices aimed at improving the software development lifecycle by automating the process of integrating code changes and deploying them to production. It is particularly useful for ensuring that software is consistently built, tested, and deployed, reducing the risk of human error and speeding up the delivery of new features and bug fixes. | — | +| 22 | **DRY (Don't Repeat Yourself)** | **ADOPT** | `software-design-principle`, `code-quality` | DRY (Don't Repeat Yourself) is a fundamental software design principle aimed at reducing the repetition of code and logic within a system. By emphasizing the elimination of redundancy, DRY encourages developers to create more maintainable, scalable, and efficient codebases. It is particularly useful in preventing the proliferation of bugs, as updates and changes only need to be made in a single location rather than multiple places. This principle is especially relevant in the "methods-and-patterns" quadrant, as it pertains to the overall approach to structuring and organizing code. Being in the "adopt" ring highlights its established importance and widespread acceptance in modern software development practices. DRY's emphasis on clean and maintainable code aligns with best practices for improving code quality and long-term sustainability of software projects. | [Fundamental Software Architecture Principles (GU-019)](../standards/software-architecture-principles.md) | +| 16 | **Diátaxis** | **ADOPT** | `documentation`, `knowledge-management` | Diátaxis is a systematic framework for organizing and writing technical documentation, aimed at improving the clarity and usability of documentation by categorizing content into four distinct types: tutorials, how-to guides, technical references, and explanations. This method helps writers and developers create comprehensive and user-friendly documentation, ensuring that users can find the right information at the right time. | — | +| 19 | **DoD** | **ADOPT** | `software-development`, `agile` | The Definition of Done (DoD) is a key concept in Agile methodologies that refers to a shared understanding among the team about what it means for a piece of work to be considered complete. It is useful because it ensures that everyone has clarity on the quality criteria and the necessary steps to achieve that quality, which helps in delivering consistent and predictable outcomes. In the methods-and-patterns quadrant, DoD is crucial as it provides a framework to guide the development process, ensuring alignment and reducing ambiguity. By adopting DoD, teams can improve their collaboration, enhance the quality of deliverables, and increase stakeholder satisfaction. | [Definition of Done (DoD) Guide (GU-003)](../templates/definition-of-done.md) | +| 18 | **Docs as code** | **ADOPT** | `documentation`, `collaboration`, `version-control` | Docs as code is a methodology that applies software development practices to documentation. This approach involves using tools such as version control systems (e.g., Git) to manage and collaborate on documentation in the same way developers work on code. It encourages the use of plain text formats, such as Markdown or reStructuredText, which can be easily tracked, reviewed, and modified. | [Docs-as-Code Guide (GU-010)](../guidelines/documentation-as-code.md) | +| 20 | **Domain-driven design** | **ADOPT** | `software-architecture`, `design-patterns` | Domain-driven design (DDD) is a strategic approach to software development that emphasizes collaboration between technical and domain experts to create a model that reflects the business domain. It focuses on understanding the core domain, creating a shared language (ubiquitous language), and structuring the codebase around the business concepts to improve alignment between software design and business needs. | — | +| 24 | **ETSI EN 303 645** | **ADOPT** | `IoT-security`, `cybersecurity-standards` | ETSI EN 303 645 is a cybersecurity standard developed by the European Telecommunications Standards Institute (ETSI) for consumer Internet of Things (IoT) devices. It establishes a set of baseline requirements to ensure devices are secure by design, addressing key areas such as data protection, software updates, and vulnerability reporting. This standard is useful as it provides manufacturers with a clear framework to mitigate security risks, enhancing consumer trust and protecting user data. In the methods-and-patterns quadrant, ETSI EN 303 645 is particularly relevant as it sets fundamental guidelines for developing secure IoT devices, emphasizing the need for security considerations during the design phase. The adoption of this standard is encouraged due to the increasing prevalence of IoT devices and the associated security challenges, making it crucial for organizations involved in IoT development to prioritize security best practices. | [Architecture Handbook: Security & Standards (GD-002)](../handbook/07-security-and-standards.md) | +| 23 | **Edge Architectures for IoT** | **ADOPT** | `edge-computing`, `IoT`, `distributed-systems` | Edge Architectures for IoT involve deploying computational resources closer to the data source in IoT ecosystems, rather than relying solely on centralized cloud data centers. This approach aims to reduce latency, improve bandwidth efficiency, and enhance data privacy by processing data locally or at the edge of the network. As IoT devices proliferate, generating vast amounts of data, edge architectures become crucial for real-time analytics and decision-making. By adopting edge architectures, organizations can ensure more responsive and reliable IoT systems, making it a vital method and pattern in the modern technological landscape. The adoption ring signifies its growing importance and maturity in the industry, as businesses increasingly look to optimise their IoT deployments for performance and cost-effectiveness. | [IoT Endpoint & Gateway Architecture Handbook (GD-005)](../guides/iot-endpoint-and-gateway-architecture/index.md) | +| 26 | **Event Driven** | **ADOPT** | `asynchronous`, `scalability`, `reactive-programming` | Event Driven architecture is a design paradigm that emphasizes the production, detection, consumption, and reaction to events. An event can be defined as a significant change in the state of a system, and this approach is particularly useful for building scalable and decoupled systems. By adopting an Event Driven approach, systems can achieve greater responsiveness and flexibility, as components can operate independently and asynchronously, reacting to changes as they occur. | — | +| 29 | **Event Streaming** | **ADOPT** | `real-time-data`, `scalability`, `distributed-systems` | Event Streaming is a data processing paradigm that treats data as a continuous stream of events, allowing for real-time data processing and analysis. This approach is particularly useful for applications that require immediate insights from data, such as fraud detection, real-time analytics, and monitoring systems. By adopting event streaming, organizations can build scalable and responsive systems that react to data changes in real-time, rather than processing data in batches. | — | +| 28 | **Event storming** | **ADOPT** | `collaborative-design`, `domain-driven-design` | Event storming is a workshop-based method used to quickly explore complex business domains by focusing on domain events. It is particularly useful for building a shared understanding among cross-functional teams, including developers, domain experts, and stakeholders. By identifying key events that occur within a system, teams can collaboratively map out processes and uncover insights that lead to better software design and implementation. | — | +| 25 | **Event-Driven Architecture (EDA)** | **ADOPT** | `scalability`, `real-time-processing` | Event-Driven Architecture (EDA) is a design paradigm in which decoupled systems communicate through the production and consumption of events. It is particularly useful for building scalable and responsive systems, as it allows for real-time data processing and asynchronous communication. EDA is increasingly relevant in the methods-and-patterns quadrant due to its ability to handle high volumes of data and its fit with modern distributed systems. By adopting EDA, organizations can create systems that are more agile and responsive to changes, facilitating innovation and improving user experiences. Its relevance in the "adopt" ring suggests that it is a well-established and effective approach that is recommended for widespread use across various industries. | [Architecture Handbook: Patterns (GD-002)](../handbook/03-architecture-patterns.md) | +| 39 | **KISS (Keep It Simple, Stupid)** | **ADOPT** | `simplicity`, `efficiency`, `design-principle` | KISS, an acronym for "Keep It Simple, Stupid," is a design principle that emphasizes simplicity in design and implementation. It advocates for avoiding unnecessary complexity, suggesting that systems function best if they are kept simple rather than made complicated. This principle is applicable across various fields, including software development, engineering, and user interface design. | [Fundamental Software Architecture Principles (GU-019)](../standards/software-architecture-principles.md) | +| 38 | **Kanban** | **ADOPT** | `agile`, `project-management`, `workflow-optimization` | Kanban is a popular workflow management method designed to help visualize work, maximise efficiency, and improve continuously. It is widely adopted in various industries, especially in software development, as part of Agile methodologies. Kanban uses visual cues—typically through a board divided into columns that represent different stages of a process—to track work items as they progress. This visualization allows teams to manage work by balancing demands with available capacity and identifying bottlenecks in the workflow. | — | +| 40 | **Layered Architecture** | **ADOPT** | `architecture-pattern`, `software-design` | Layered Architecture is a widely used architectural pattern in software design that organizes the system into a series of layers, each with a distinct responsibility. This pattern typically includes layers such as presentation, business logic, and data access. The primary advantage of Layered Architecture is its ability to separate concerns, making systems more modular, maintainable, and scalable. Each layer can be developed, tested, and maintained independently, allowing teams to work on different parts of the system simultaneously without significant interference. | — | +| 41 | **Lightweight Architecture Evaluations** | **ADOPT** | `architecture`, `evaluation`, `agile` | Lightweight Architecture Evaluations refer to a streamlined approach to assessing and reviewing software architecture. Unlike traditional, exhaustive architectural reviews, this method focuses on agility, enabling teams to quickly evaluate architectural decisions without extensive documentation or formal processes. This approach is especially useful in agile and fast-paced development environments, where time and resources for detailed analysis may be limited. In the "methods-and-patterns" quadrant, Lightweight Architecture Evaluations help teams stay responsive to change and ensure that architectural decisions align with evolving project needs. The "adopt" ring suggests that this practice has proven benefits and should be considered a standard approach for teams seeking to maintain agility while ensuring sound architectural principles. | — | +| 47 | **MISRA C** | **ADOPT** | `safety-critical`, `coding-standard` | MISRA C is a set of software development guidelines for the C programming language, developed by the Motor Industry Software Reliability Association (MISRA). It is widely adopted in industries where safety and reliability are paramount, such as automotive, aerospace, and medical devices. The guidelines aim to facilitate code safety, portability, and reliability in the embedded systems environment by imposing restrictions on the use of certain C language features that can lead to unsafe or unpredictable behaviour. | [MISRA C Guidelines (GU-012)](../standards/misra-c.md) | +| 45 | **Microservices** | **ADOPT** | `architecture`, `scalability`, `resilience` | Microservices is an architectural style that structures an application as a collection of loosely coupled services, each of which implements a business capability. This approach enhances flexibility and scalability, allowing for the independent deployment of services, which can be developed, tested, and deployed independently by different teams. | [Architecture Handbook: Patterns (GD-002)](../handbook/03-architecture-patterns.md) | +| 48 | **Modular Monolith** | **ADOPT** | `architecture`, `software-design`, `scalability` | The Modular Monolith is a software architecture pattern that emphasizes structuring a monolithic application into distinct modules, each encapsulating a specific domain or functionality. Unlike traditional monoliths, where all components are intertwined, a Modular Monolith promotes a high degree of modularity within a single deployable unit. This approach allows for clearer boundaries, easier maintenance, and the possibility of gradually transitioning to a microservices architecture if needed. | [Architecture Handbook: Patterns (GD-002)](../handbook/03-architecture-patterns.md) | +| 52 | **OWASP** | **ADOPT** | `security`, `best-practices` | The Open Web Application Security Project (OWASP) is an essential resource in the field of web application security. It is a non-profit foundation that works to improve the security of software through community-driven open-source software projects, hundreds of local chapters worldwide, tens of thousands of members, and leading educational and training conferences. OWASP is renowned for its comprehensive resources such as the OWASP Top Ten, a regularly updated list of the most critical security risks to web applications. | — | +| 53 | **PEP 8** | **ADOPT** | `coding-standards`, `python` | PEP 8 is the style guide for Python code, providing conventions on how to format Python code for maximum readability. It is widely adopted in the Python community, ensuring consistency across diverse codebases. PEP 8 covers various aspects, including indentation, naming conventions, and best practices for writing clear and maintainable code. | — | +| 57 | **REST** | **ADOPT** | `web-architecture`, `api-design` | REST (Representational State Transfer) is an architectural style that is widely used for designing networked applications. It leverages standard HTTP methods, such as GET, POST, PUT, and DELETE, to facilitate communication between clients and servers. RESTful services are stateless and typically return data in formats like JSON or XML, making them highly interoperable and scalable. | — | +| 61 | **Scrum** | **ADOPT** | `agile`, `project-management`, `team-collaboration` | Scrum is an agile framework for managing complex projects, primarily focused on software development. It facilitates a flexible, iterative approach where work is broken down into sprints, typically lasting two to four weeks. Scrum emphasizes collaboration, accountability, and continuous improvement, making it an essential tool for teams aiming to deliver high-quality software efficiently. The framework includes defined roles (such as Scrum Master and Product Owner), events (such as daily stand-ups and sprint reviews), and artifacts (like product backlogs) to ensure transparency and enhance productivity. | — | +| 68 | **TOGAF** | **ADOPT** | `enterprise-architecture`, `best-practices` | TOGAF, which stands for The Open Group Architecture Framework, is a comprehensive framework used for developing and managing enterprise architecture. It is widely adopted by organizations to improve business efficiency and ensure that IT aligns with business objectives. As a key player in the methods-and-patterns quadrant, TOGAF provides a structured approach for organizing and governing information technology infrastructure. | — | +| 69 | **UML Modelling** | **ADOPT** | `design` | Unified Modelling Language (UML) is a standard way to visualize software architecture. | — | +| 74 | **YAGNI (You Aren’t Gonna Need It)** | **ADOPT** | `software-development`, `agile-practices` | YAGNI (You Aren’t Gonna Need It) is a fundamental principle of agile software development that encourages developers to avoid adding functionality until it is necessary. By adhering to YAGNI, teams can maintain a lean codebase, reduce complexity, and focus on delivering features that provide immediate value to the user. This principle is particularly useful in iterative development environments where requirements can change rapidly, and maintaining flexibility is crucial. Within the methods-and-patterns quadrant, YAGNI is a foundational practice, promoting efficient workflow and resource use. It is recommended for adoption as it aligns with the modern emphasis on agility and responsiveness in software development. Implementing YAGNI can help organizations avoid over-engineering and concentrate efforts on current needs, ensuring that development remains focused and productive. | [Fundamental Software Architecture Principles (GU-019)](../standards/software-architecture-principles.md) | +| 3 | **arc42** | **ADOPT** | `software-architecture`, `documentation` | arc42 is a template for documenting software and system architectures. It is designed to help architects efficiently communicate design decisions and structural concepts without being overly prescriptive. The template is structured to cover all relevant aspects of an architecture, including requirements, constraints, quality goals, and the rationale behind architectural decisions. | — | +| 35 | **gRPC** | **ADOPT** | `remote-procedure-call`, `microservices` | gRPC is a high-performance, open-source universal remote procedure call (RPC) framework developed by Google. It uses HTTP/2 for transport, Protocol Buffers as the interface description language, and provides features such as authentication, load balancing, and more. gRPC is particularly useful in microservices architectures, where it enables efficient and language-agnostic communication between services. Its ability to support multiple languages makes it a versatile choice for distributed systems. | — | +| 50 | **oauth2** | **ADOPT** | `authentication`, `security` | OAuth2 is an open standard for access delegation commonly used to grant websites or applications limited access to user information on another website without exposing the user's credentials. It is particularly significant in the realm of authentication and authorisation protocols, making it a pivotal method in securing web applications and APIs. By enabling secure token-based authentication, OAuth2 allows developers to build robust security layers, enhancing user privacy and data protection. | — | +| 15 | **CQRS** | **ASSESS** | `architecture`, `design-pattern` | Command Query Responsibility Segregation (CQRS) is a design pattern that separates the read and write operations of a data store into distinct models. This separation is useful in systems that require scalability and have complex business logic, as it allows for optimized query performance and simplifies the management of complex domain logic. CQRS is particularly relevant in the methods-and-patterns quadrant because it provides a strategic approach to handling data operations in software architecture. It is placed in the assess ring, indicating that while it shows promise and can address specific architectural challenges, it should be evaluated carefully for suitability within the context of a project's requirements. CQRS can be especially beneficial when combined with Event Sourcing to provide a complete history of changes to the system’s state. | — | +| 21 | **DoR** | **ASSESS** | `agile`, `project-management` | The Definition of Ready (DoR) is a critical concept in agile project management that ensures a backlog item is prepared for execution by the development team. It sets a checklist of conditions that must be met before work can begin on a particular task, thereby reducing ambiguities and improving the chances of successful delivery. In the context of methods and patterns, DoR plays a significant role in enhancing workflow efficiency and team alignment. By assessing the DoR, teams can ensure that they have a clear understanding of the requirements, acceptance criteria, and necessary resources, ultimately leading to more predictable and manageable development cycles. This makes it an essential practice for teams striving to improve their agile processes and deliver high-quality software consistently. | [Definition of Ready (DoR) Guide (GU-004)](../templates/definition-of-ready.md) | +| 27 | **Event Sourcing** | **ASSESS** | `data-management`, `system-architecture` | Event Sourcing is a method of capturing all changes to the state of an application as a sequence of events. Instead of storing just the current state, every change is recorded as an immutable event, which can be replayed to reconstruct past states or derive new information. This approach is useful for maintaining a clear audit trail, enabling system scalability, and supporting complex business logic by reconstructing states at any point in time. | — | +| 36 | **ISO 30141** | **ASSESS** | `IoT`, `architecture`, `standards` | ISO 30141 is an international standard that provides a reference architecture for the Internet of Things (IoT). It offers a framework to design IoT systems with a focus on interoperability, connectivity, and security. This standard is useful because it establishes a common understanding and approach to developing IoT solutions, which is crucial given the increasing complexity and scale of IoT deployments. | — | +| 58 | **RFC9457** | **ASSESS** | `networking`, `security`, `internet-standards` | RFC9457 is a significant document in the realm of Internet standards, focusing on improving security protocols within networking systems. It provides a comprehensive framework for enhancing secure communications across different platforms and operations. RFC9457 is essential for developers and network architects who need to ensure their systems are compliant with modern security standards, particularly in an era where cyber threats are increasingly sophisticated. | — | +| 62 | **SEI ATAM** | **ASSESS** | `architecture-evaluation`, `software-quality` | The Software Engineering Institute's Architecture Tradeoff Analysis Method (SEI ATAM) is a systematic approach used for evaluating software architectures. It is particularly useful in identifying architectural risks early in the development process and provides a framework for understanding how well an architecture meets its quality attribute requirements, such as performance, modifiability, and security. | — | +| 63 | **SEI CBAM** | **ASSESS** | `architecture-evaluation`, `cost-benefit-analysis` | The SEI Cost Benefit Analysis Method (CBAM) is a structured approach designed to help software architects and decision-makers evaluate the trade-offs among various architectural decisions. By integrating economic considerations with architectural needs, CBAM allows teams to assess the potential costs and benefits associated with different design options. This method is especially relevant in the methods-and-patterns quadrant as it provides a systematic approach to evaluating and justifying architectural choices, ensuring that they align with business goals and constraints. | — | +| 65 | **SOA** | **ASSESS** | `service-oriented-architecture`, `integration-patterns` | Service-Oriented Architecture (SOA) is a design pattern where distinct services are provided to other components by application components through a communication protocol over a network. SOA's primary goal is to allow easy integration between different applications and to facilitate business processes by making these services reusable and interoperable. This architecture style is particularly useful for organizations aiming to create complex systems that are agile and can be modified or extended with minimal impact on existing services. | — | +| 70 | **V-Model** | **ASSESS** | `software-development`, `testing`, `project-management` | The V-Model, also known as the Verification and Validation model, is a software development process that emphasizes the importance of testing at each stage of development. It extends the traditional waterfall model by associating a testing phase with each corresponding development stage. This ensures that validation and verification are integral parts of the development lifecycle, leading to early detection and correction of defects. | [HIL & Automated Testing Guide (GD-007)](../guides/hil-and-automated-testing/index.md) | +| 77 | **Zachman Framework** | **ASSESS** | `enterprise-architecture`, `framework` | The Zachman Framework is a comprehensive enterprise architecture framework for organizing and categorizing the various perspectives involved in enterprise architecture. It provides a structured way to view and define an enterprise's architecture by considering different aspects like data, functionality, network, people, time, and motivation across multiple perspectives, such as business owner, architect, designer, builder, and user. | — | +| 59 | **rRPC** | **ASSESS** | `remote-procedure-call`, `microservices` | rRPC, or Reactive Remote Procedure Call, is an emerging method that enhances the traditional RPC model by incorporating reactive programming principles. This approach is particularly useful in systems where responsiveness and scalability are critical, such as microservices architectures. By leveraging a non-blocking, event-driven paradigm, rRPC enables more efficient use of system resources and improves the overall performance of distributed applications. | — | +| 73 | **Waterfall** | **HOLD** | `project-management`, `software-development` | Waterfall is a traditional software development methodology that follows a linear and sequential approach, akin to a cascading waterfall. Each phase—such as requirements gathering, design, implementation, testing, and maintenance—must be completed before the next one begins. This method is useful in projects where requirements are well-understood and unlikely to change, as it emphasizes thorough documentation and a clear project roadmap. | — | + +[[CAPTION:Table]] Complete directory of Methods & Patterns blips. + +## Tools & Utilities + +Dedicated quadrant deep-dive: **[Tools & Utilities](tools-and-utilities.md)** + +| # | Technology | Ring | Tags | Description & Rationale | Reference | +| :-: | :--- | :---: | :--- | :--- | :--- | +| 43 | **Mermaid** | **ADOPT** | `diagramming`, `visualization`, `documentation` | Mermaid is a JavaScript-based diagramming and charting tool that enables developers and content creators to create complex diagrams from simple text definitions. It is particularly useful for generating flowcharts, sequence diagrams, Gantt charts, and more, using a straightforward syntax that integrates seamlessly into various documentation systems, such as Markdown and static site generators like Jekyll and Docusaurus. | — | +| 51 | **OpenAPI** | **ADOPT** | `API-design`, `documentation` | OpenAPI is a specification for building APIs that provides a standard, language-agnostic interface describing RESTful APIs. It's widely adopted in the industry due to its ability to streamline API development processes. By using OpenAPI, developers can automate the generation of API documentation, client libraries, and server stubs, which reduces the likelihood of errors and accelerates development timelines. | — | +| 54 | **PlantUML** | **ADOPT** | `diagramming`, `visualization`, `documentation` | PlantUML is an open-source tool that allows users to create UML diagrams from a plain text language. It is widely appreciated for its simplicity and the ease with which complex diagrams can be generated without needing to use graphical editing tools. PlantUML supports a variety of UML diagrams, including sequence diagrams, use case diagrams, class diagrams, and more, making it a versatile choice for software architects and developers. | [Diagrams-as-Code Tooling Guide (GD-001)](../guides/c4-model/02-diagrams-as-code-tooling.md) | +| 66 | **SQLite** | **ADOPT** | `embedded-database`, `lightweight-database` | SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. It is the most widely deployed database engine in the world, embedded into all sorts of applications. Its simplicity and ease of use make it an ideal choice for embedded database needs, mobile apps, and small to medium-sized applications that do not require the overhead of a full-scale database server. | — | +| 72 | **VS code** | **ADOPT** | `code-editor`, `IDE`, `developer-productivity` | Visual Studio Code (VS Code) is a widely-used, open-source code editor developed by Microsoft. It provides developers with an efficient, lightweight, and highly customizable environment to write and debug code across various programming languages. VS Code is particularly appreciated for its rich extension ecosystem, enabling users to enhance functionality with plugins for specific languages, integrations, and developer tools. | — | +| 71 | **Visual Studio Code** | **ADOPT** | `editor` | VS Code is a popular editor for development with rich plugin support. | — | +| 31 | **gcc** | **ADOPT** | `compiler`, `open-source`, `C`, `C++` | The GNU Compiler Collection (gcc) is a comprehensive and widely-used set of compilers for various programming languages, most notably C, C++, and Fortran. As an open-source project under the GNU umbrella, gcc is highly valued for its robustness, performance, and cross-platform capabilities. It serves as a critical tool in software development, enabling the translation of high-level code into executable machine code across various architectures. In the tools quadrant and adopt ring, gcc is an essential part of the development toolchain for many organizations and projects due to its maturity, extensive community support, and continuous updates. Its relevance spans from education to enterprise applications, making it a staple in both learning environments and professional software engineering practices. | — | +| 44 | **Mermaid Diagrams** | **TRIAL** | `visualization`, `documentation` | Mermaid is a JavaScript-based diagramming and charting tool that allows for dynamic, code-based visualizations. It's particularly useful for architecture diagrams, flowcharts, and Gantt charts in technical documentation. Mermaid syntax integrates seamlessly into Markdown files for inline visualization. | — | +| 14 | **Coverity** | **ASSESS** | `static-analysis`, `code-quality`, `security` | Coverity is a static analysis tool used for identifying defects in software codebases. It is particularly useful for detecting a wide range of issues such as security vulnerabilities, resource leaks, and concurrency defects before they manifest in production environments. Coverity is relevant in the "tools" quadrant because it provides developers and development teams with a robust mechanism to improve code quality and maintainability through automated code review processes. In the "assess" ring, it reflects its growing importance as organizations increasingly prioritize integrating security and quality assurance early in the software development lifecycle, aligning with practices like DevSecOps. As the landscape of software development continues to evolve, tools like Coverity are integral in ensuring software reliability and security compliance. | — | +| 67 | **Structurizr** | **ASSESS** | `architecture-diagrams`, `software-design`, `cloud` | Structurizr is a tool designed for creating software architecture diagrams based on the C4 model, which emphasizes clarity and simplicity in visualizing software architecture. It is particularly useful for software architects and developers who need to communicate complex systems effectively. Structurizr allows users to define their architecture models using code, enabling version control and integration with existing development workflows. In the tools quadrant, Structurizr stands out as a modern approach to diagramming that aligns with agile methodologies and continuous integration practices. In the assess ring, it is recommended for evaluation as it can potentially enhance documentation and understanding of software systems within organizations, especially those adopting cloud-native technologies. | — | + +[[CAPTION:Table]] Complete directory of Tools & Utilities blips. diff --git a/content/radar/index.md b/content/radar/index.md index 9f962c7..2d412d4 100644 --- a/content/radar/index.md +++ b/content/radar/index.md @@ -5,6 +5,118 @@ author: Mark Bacon # Tech Radar -The technologies, tools and methods I adopt, trial, assess or hold. +The **Technology Radar** is an opinionated, living guide that tracks the technologies, tools, architectural patterns, and operational platforms evaluated, adopted, or retired across our software engineering practice. It provides transparency into our technical stack, guides technology selection for new initiatives, prevents unnecessary architectural fragmentation, and surfaces emerging innovations. -This section is moving in from site-web-content-src's Tech Radar and technical-documentation's radar. +The radar is curated collaboratively by practicing engineers through the Architecture Advisory Forum and Architecture Review Board ([ARB Guide](../guides/architecture-review-board.md)), embodying our conversational architecture approach ([Scaling Architecture Guide](../guides/scaling-architecture/02-alignment-mechanisms.md)). + +--- + + +The radar is published as four quadrant pages and a directory of every entry. On the website there is also an [interactive radar chart](https://markbac.github.io/radar-chart/), with search and filters. + +| Page | Covers | +| --- | --- | +| [Languages and frameworks](languages-and-frameworks.md) | Programming languages and the frameworks around them | +| [Platforms and operations](platforms-and-operations.md) | Operating systems, runtimes, cloud platforms and protocols | +| [Methods and patterns](methods-and-patterns.md) | Architecture styles, practices and standards | +| [Tools and utilities](tools-and-utilities.md) | Development, documentation and analysis tools | +| [Directory](directory.md) | Every entry in one table, with its ring and quadrant | + +[[CAPTION:Table]] The radar pages. + +## Structure of the Radar + +The radar categorises engineering subjects across two fundamental dimensions: **Four Quadrants** (technical domains) and **Four Concentric Rings** (lifecycle maturity). + +```mermaid +flowchart TD + subgraph Quadrants["The Four Quadrants"] + LANG["Languages & Frameworks
Core programming languages, schemas, serialisation formats"] + PLAT["Platforms & Operations
Operating systems, RTOS, containers, clouds, protocols"] + METH["Methods & Patterns
Architecture styles, engineering practices, security standards"] + TOOL["Tools & Utilities
Static analysers, build systems, diagram engines, IDEs"] + end + subgraph Rings["The Four Rings (Maturity Lifecycle)"] + direction LR + ASSESS["ASSESS
Research & spike"] --> TRIAL["TRIAL
Low-risk evaluation"] + TRIAL --> ADOPT["ADOPT
Default standard"] + ADOPT --> HOLD["HOLD
Deprecated / avoid"] + end +``` + +[[CAPTION:Figure]] Conceptual structure of the Technology Radar across quadrants and lifecycle rings. + +--- + +## The Four Rings + +Every technology, tool, or practice on the radar is assigned to one of four maturity rings. Each ring conveys a precise operational recommendation and governance gate. + +```mermaid +stateDiagram-v2 + [*] --> ASSESS: Candidate identified + ASSESS --> TRIAL: Viable proof-of-concept + TRIAL --> ADOPT: Proven in production project + ADOPT --> HOLD: Superseded / Security risk + ASSESS --> HOLD: Rejected during spike + TRIAL --> HOLD: Failed evaluation + HOLD --> [*]: Decommissioned +``` + +[[CAPTION:Figure]] Technology lifecycle transitions through the radar rings. + +### 1. Adopt +- **Definition**: High-confidence, battle-tested technologies that represent the default production choice for all active squads and products. +- **Guidance**: Use whenever appropriate. Choosing an *Adopt* technology requires **zero formal ARB approval**. +- **Examples**: C (C99/C11), FreeRTOS, Docker, PlantUML, Docs-as-Code, C4 Model, Google Cloud Platform. +- **Criteria**: Proven reliability in production, strong community/vendor backing, comprehensive internal expertise, active maintainability, and alignment with corporate security benchmarks. + +### 2. Trial +- **Definition**: Promising technologies that have passed initial research and are undergoing controlled evaluation on low-risk or non-critical production projects. +- **Guidance**: Squads may trial these technologies provided they establish an explicit rollback contingency and document findings in an Architecture Decision Record ([ADR Guide](../guides/decision-records.md)). +- **Examples**: Structurizr DSL, OpenTelemetry, AsyncAPI. +- **Criteria**: Successfully completed a proof-of-concept spike, demonstrated distinct advantages over current defaults, and received endorsement from the Architecture Advisory Forum. + +### 3. Assess +- **Definition**: Emerging or specialized technologies under active investigation, exploration, or technical spike. +- **Guidance**: Worth exploring to understand potential future value. **Strictly prohibited in customer-facing production systems** without explicit ARB dispensation. +- **Examples**: CloudEvents, Cert C, MISRA C++, Zig, Zephyr RTOS. +- **Criteria**: Clear relevance to strategic technical goals, but unproven within our operational environment or lacking internal tooling integration. + +### 4. Hold +- **Definition**: Technologies, practices, or libraries that are deprecated, obsolete, have unacceptable security postures, or generate disproportionate technical debt. +- **Guidance**: **Strictly prohibited for new greenfield projects**. Existing production workloads running *Hold* technologies should plan orderly migration or refactoring. +- **Examples**: Waterfall delivery, legacy monolithic build scripts, unsecured MQTT endpoints, outdated protocol decoders. +- **Criteria**: Superseded by modern alternatives, vendor end-of-life, persistent security vulnerabilities, or fragmentation risks. + +--- + +## The Four Quadrants + +| Quadrant | Domain Description | Representative Technologies | Guideline Alignment | +| :--- | :--- | :--- | :--- | +| **[Languages & Frameworks](languages-and-frameworks.md)** | Core programming languages, domain-specific languages, data serialisation formats, and schemas. | C, C++, C#, Rust, Python, Go, Protobuf, JSON, YAML, GraphQL. | [C Coding Standards](../standards/c-coding-standard.md), [Python Standards](../standards/python.md) | +| **[Platforms & Operations](platforms-and-operations.md)** | Hardware execution environments, embedded RTOS kernels, container runtimes, cloud platforms, and transport protocols. | FreeRTOS, Linux/Yocto, Docker, GCP, MQTT, Serverless, CloudEvents. | [Architecture Handbook: IoT & Embedded](../handbook/06-iot-and-embedded-systems.md) | +| **[Methods & Patterns](methods-and-patterns.md)** | System architecture patterns, software design principles, security specifications, and delivery methodologies. | C4 Model, Clean/Hexagonal, Modular Monolith, EDA, ETSI EN 303 645, OWASP, Agile. | [C4 Model Guide](../guides/c4-model/index.md), [Architecture Principles](../standards/software-architecture-principles.md) | +| **[Tools & Utilities](tools-and-utilities.md)** | Developer tooling, static analysis engines, compilers, diagramming tools, and automated testing suites. | PlantUML, Structurizr DSL, Mermaid, Coverity, GCC, Clang-Format, VS Code. | [Docs-as-Code Guide](../guidelines/documentation-as-code.md), [Diagrams Tooling](../guides/c4-model/02-diagrams-as-code-tooling.md) | + +[[CAPTION:Table]] Summary of the four radar quadrants and their engineering documentation mappings. + +--- + +## Technology Evaluation Lifecycle and Governance + +To introduce a new technology or change an existing ring placement: + +1. **Submission**: Any engineer submits an RFC ([RFC Authoring Guide](../guides/decision-records.md)) detailing the motivation, alternatives evaluated, operational risks, and migration costs. +2. **Exploration (Assess)**: The squad runs a bounded technical spike in a sandbox environment and presents quantitative metrics (binary footprint, latency, memory usage, build times). +3. **Pilot (Trial)**: If endorsed by the Architecture Advisory Forum, the technology is piloted in a scoped, non-critical service with automated test coverage and an ADR ([ADR Guide](../guides/decision-records.md)). +4. **Promotion (Adopt)**: Following successful production deployment and validation of operational runbooks, the ARB promotes the blip to *Adopt*. +5. **Sunset (Hold)**: When a technology is superseded or poses unmaintained dependencies, the ARB issues a deprecation notice and transition roadmap. + +--- + +## Navigating the Radar + +- **[Interactive Technology Radar](index.md)**: Explore the interactive visual radar canvas, filter by ring and quadrant, search blips, and click on any technology to view complete details in a popup modal. +- **[Complete Technology Blip Directory](directory.md)**: Searchable reference catalogue of all evaluated technologies, status badges, and contextual architectural rationale. diff --git a/content/radar/languages-and-frameworks.md b/content/radar/languages-and-frameworks.md new file mode 100644 index 0000000..b7091ae --- /dev/null +++ b/content/radar/languages-and-frameworks.md @@ -0,0 +1,71 @@ +--- +title: Languages and Frameworks +--- + +# Languages and Frameworks + +This quadrant covers core programming languages, schema definition languages, and data serialisation formats used across our embedded firmware, edge gateways, and cloud platforms. + +## Ring: Adopt + +### C +- **Status**: **Adopt** +- **Tags**: `embedded`, `firmware`, `real-time`, `c99`, `misra` +- **Rationale**: C remains the unmatched industry standard for resource-constrained microcontroller development and RTOS kernel programming. With deterministic memory layout and zero runtime overhead, all bare-metal drivers and low-level task code are authored in C99/C11 conforming to [C Coding Standards](../standards/c-coding-standard.md) and [MISRA C Guidelines](../standards/misra-c.md). + +### C++ +- **Status**: **Adopt** +- **Tags**: `systems`, `embedded`, `c++17`, `gateways` +- **Rationale**: Used primarily on mid-tier gateways, edge computing appliances, and complex protocol decoders where object-oriented abstractions, RAII resource management, and templates provide strong type safety without sacrificing performance. Restricted to a safety-conscious subset (avoiding exceptions and heavy RTTI). + +### C# (.NET) +- **Status**: **Adopt** +- **Tags**: `backend`, `cloud`, `enterprise`, `testing` +- **Rationale**: High-productivity language for enterprise services, desktop manufacturing test benches, and high-throughput cloud back-ends running on modern .NET 8+. + +### Rust +- **Status**: **Adopt** +- **Tags**: `systems`, `memory-safety`, `embedded`, `cloud` +- **Rationale**: Adopted for new performance-critical cloud microservices and selected edge components requiring uncompromising memory safety without garbage collection. Its ownership model guarantees thread safety at compile time, eliminating entire classes of concurrency defects. + +### Python +- **Status**: **Adopt** +- **Tags**: `scripting`, `automation`, `testing`, `data-science` +- **Rationale**: The standard language for continuous integration automation, automated hardware-in-the-loop (HIL) test suites, cloud data engineering pipelines, and rapid prototyping. Conforms to [Python Coding Standards](../standards/python.md) and PEP 8. + +### Go (golang) +- **Status**: **Adopt** +- **Tags**: `cloud`, `microservices`, `concurrency`, `cli` +- **Rationale**: The preferred language for high-throughput network daemons, API gateways, and distributed cloud microservices. Its lightweight goroutines and compiled single-binary deployments provide exceptional operational simplicity. + +### Protocol Buffers (Protobuf) +- **Status**: **Adopt** +- **Tags**: `serialization`, `binary`, `grpc`, `iot` +- **Rationale**: Strongly-typed, compact binary serialization format powering all internal microservice gRPC communications and constrained IoT telemetry streams. Reduces payload sizes by 60-75% compared to JSON. + +### JSON +- **Status**: **Adopt** +- **Tags**: `serialization`, `web`, `rest`, `text` +- **Rationale**: The standard data interchange format for public RESTful HTTP APIs and web dashboards. Paired with RFC 9457 for structured error reporting ([GU-020](../standards/api-design.md)). + +### YAML +- **Status**: **Adopt** +- **Tags**: `configuration`, `iac`, `docs-as-code` +- **Rationale**: Standard human-readable configuration language for Kubernetes manifests, GitHub Actions pipelines, and Docs-as-Code publication manifests. + +### GraphQL +- **Status**: **Adopt** +- **Tags**: `api`, `frontend`, `query-language` +- **Rationale**: Adopted for customer-facing web and mobile dashboards where front-end applications require flexible, aggregated data fetching from multiple underlying microservices in a single round-trip. + +## Ring: Assess + +### CERT C +- **Status**: **Assess** +- **Tags**: `security`, `coding-standards`, `embedded` +- **Rationale**: Evaluating SEI CERT C coding rules to complement existing MISRA C compliance, specifically focusing on mitigating cyber-security attack surfaces and integer arithmetic vulnerabilities in connected IoT devices. + +### MISRA C++ +- **Status**: **Assess** +- **Tags**: `safety-critical`, `embedded`, `c++` +- **Rationale**: Evaluating MISRA C++:2023 guidelines for application in next-generation edge gateway runtimes where modern C++ features are adopted under strict safety requirements. diff --git a/content/radar/methods-and-patterns.md b/content/radar/methods-and-patterns.md new file mode 100644 index 0000000..304edb9 --- /dev/null +++ b/content/radar/methods-and-patterns.md @@ -0,0 +1,83 @@ +--- +title: Methods and Patterns +--- + +# Methods and Patterns + +This quadrant covers software architecture styles, engineering methodologies, regulatory security standards, and operational quality practices. + +## Ring: Adopt + +### C4 Architecture Model +- **Status**: **Adopt** +- **Tags**: `architecture`, `modelling`, `c4`, `diagrams` +- **Rationale**: The core architectural abstraction framework used across the company to model software, firmware, and connected systems across Context, Containers, Components, and Code/Deployment ([GD-001](../guides/c4-model/index.md)). + +### Hexagonal Architecture (Ports and Adapters) +- **Status**: **Adopt** +- **Tags**: `architecture`, `patterns`, `clean-architecture`, `domain` +- **Rationale**: Primary structural pattern for core domain logic, decoupling business rules from external databases, UI frameworks, and network protocols. + +### Modular Monolith +- **Status**: **Adopt** +- **Tags**: `architecture`, `patterns`, `monolith`, `modularity` +- **Rationale**: The default architectural pattern for new services, providing independent domain boundaries without the network complexity, eventual consistency challenges, and operational overhead of microservices. + +### Microservices +- **Status**: **Adopt** +- **Tags**: `architecture`, `distributed`, `cloud`, `scaling` +- **Rationale**: Adopted selectively for large, mature domains where independent team delivery cadences and divergent scaling profiles warrant independent containerized services. + +### Event-Driven Architecture (EDA) +- **Status**: **Adopt** +- **Tags**: `architecture`, `eda`, `pubsub`, `asynchronous` +- **Rationale**: Primary communication pattern for IoT telemetry processing and cross-microservice notifications, maximizing decoupling and system resilience. + +### ETSI EN 303 645 & UK PSTI +- **Status**: **Adopt** +- **Tags**: `security`, `compliance`, `iot`, `regulation` +- **Rationale**: Mandatory regulatory security baseline for all connected consumer appliances, enforcing unique per-device passwords, vulnerability disclosure, and secure firmware updates. + +### Documentation as Code +- **Status**: **Adopt** +- **Tags**: `documentation`, `git`, `markdown`, `docs-as-code` +- **Rationale**: Standard practice of authoring technical documentation in Markdown, storing it in Git, validating via linters in CI, and building static documentation sites ([GU-010](../guidelines/documentation-as-code.md)). + +### Architecture Decision Records (ADRs) +- **Status**: **Adopt** +- **Tags**: `governance`, `architecture`, `adr`, `git` +- **Rationale**: Mandatory standard for capturing context, evaluated options, and trade-off consequences of major architectural choices ([GU-005](../guides/decision-records.md)). + +### Agile (Scrum / Kanban) +- **Status**: **Adopt** +- **Tags**: `methodology`, `process`, `delivery` +- **Rationale**: Iterative software delivery methodologies with empirical feedback, clear quality gates (DoD per [GU-003](../templates/definition-of-done.md)), and collaborative retrospectives. + +### MISRA C +- **Status**: **Adopt** +- **Tags**: `embedded`, `standards`, `safety-critical`, `c` +- **Rationale**: Mandatory coding standards subset for all safety-critical embedded C firmware ([GU-012](../standards/misra-c.md)). + +## Ring: Assess + +### CQRS and Event Sourcing +- **Status**: **Assess** +- **Tags**: `architecture`, `patterns`, `data` +- **Rationale**: Evaluating Command Query Responsibility Segregation and Event Sourcing for complex audit-sensitive domains where reconstructing historical state transitions is paramount. + +### Definition of Ready (DoR) +- **Status**: **Assess** +- **Tags**: `methodology`, `process`, `agile` +- **Rationale**: Evaluating structured pre-sprint gates ([GU-004](../templates/definition-of-ready.md)) to prevent ill-defined user stories from entering active sprints. + +### ISO/IEC 30141 (IoT Reference Architecture) +- **Status**: **Assess** +- **Tags**: `iot`, `standards`, `architecture` +- **Rationale**: Assessing the international reference architecture standard for industrial IoT implementations. + +## Ring: Hold + +### Waterfall Delivery Model +- **Status**: **Hold** +- **Tags**: `methodology`, `deprecated` +- **Rationale**: Rigid, linear, big-bang delivery models are deprecated across all engineering squads in favour of iterative, empirical Agile and Docs-as-Code practices. diff --git a/content/radar/platforms-and-operations.md b/content/radar/platforms-and-operations.md new file mode 100644 index 0000000..c8d5566 --- /dev/null +++ b/content/radar/platforms-and-operations.md @@ -0,0 +1,46 @@ +--- +title: Platforms and Operations +--- + +# Platforms and Operations + +This quadrant covers real-time operating systems, embedded build distributions, container runtimes, cloud infrastructure platforms, and messaging protocols. + +## Ring: Adopt + +### FreeRTOS +- **Status**: **Adopt** +- **Tags**: `rtos`, `embedded`, `real-time`, `microcontroller` +- **Rationale**: The core real-time operating system kernel powering our microcontroller firmware. FreeRTOS provides deterministic preemptive task scheduling, minimal RAM overhead ($< 10\,\mathrm{KB}$), and extensive hardware support across ARM Cortex-M and RISC-V architectures. + +### Yocto Project +- **Status**: **Adopt** +- **Tags**: `linux`, `embedded`, `build-system`, `gateways` +- **Rationale**: Standard build automation framework for producing customized, minimal, and fully reproducible embedded Linux board support packages (BSPs) for edge compute gateways. + +### Docker +- **Status**: **Adopt** +- **Tags**: `containers`, `devops`, `cloud`, `ci` +- **Rationale**: Standard containerization runtime for cloud microservices, local developer environments, and automated CI build runners. Guarantees environment parity between development and production. + +### Google Cloud Platform (GCP) +- **Status**: **Adopt** +- **Tags**: `cloud`, `infrastructure`, `analytics`, `bigquery` +- **Rationale**: Primary cloud platform provider hosting our IoT telemetry ingestion pipelines, BigQuery data analytics warehouse, and Google Kubernetes Engine (GKE) clusters. + +### MQTT +- **Status**: **Adopt** +- **Tags**: `iot`, `protocol`, `pubsub`, `messaging` +- **Rationale**: Standard lightweight publish/subscribe protocol for low-bandwidth, high-latency edge-to-cloud device communications. Provides QoS delivery guarantees and keep-alive heartbeats over TLS. + +### Serverless (Cloud Functions / Cloud Run) +- **Status**: **Adopt** +- **Tags**: `cloud`, `serverless`, `faas`, `event-driven` +- **Rationale**: Adopted for asynchronous event processing, third-party webhook receivers, and sporadic background tasks that scale instantly from zero to thousands of instances without dedicated server costs. + +## Ring: Assess + +### CloudEvents +- **Status**: **Assess** +- **Tags**: `standards`, `event-driven`, `cloud`, `cncf` +- **Rationale**: Assessing the CNCF CloudEvents 1.0 specification for standardising event envelopes and metadata across heterogeneous cloud and edge message buses. diff --git a/content/radar/tools-and-utilities.md b/content/radar/tools-and-utilities.md new file mode 100644 index 0000000..8dcd523 --- /dev/null +++ b/content/radar/tools-and-utilities.md @@ -0,0 +1,51 @@ +--- +title: Tools and Utilities +--- + +# Tools and Utilities + +This quadrant covers development environments, diagram engines, static analysis linters, compilers, and local database engines. + +## Ring: Adopt + +### PlantUML +- **Status**: **Adopt** +- **Tags**: `diagrams`, `tools`, `uml`, `c4` +- **Rationale**: The standard text-based diagram generator for complex UML sequence diagrams, component diagrams, and C4 architectural models. Integrated into automated DOCX/PDF document compilation via `createdocs.py`. + +### Mermaid +- **Status**: **Adopt** +- **Tags**: `diagrams`, `tools`, `markdown`, `docs-as-code` +- **Rationale**: Preferred tool for lightweight, client-side rendered flowcharts, state transitions, and sequence diagrams directly inside Markdown documentation pages. + +### OpenAPI Specification (v3.1) +- **Status**: **Adopt** +- **Tags**: `api`, `tools`, `rest`, `specification` +- **Rationale**: Standard machine-readable contract format for describing RESTful HTTP APIs. Powers automated documentation generation, client SDK synthesis, and mock testing. + +### SQLite +- **Status**: **Adopt** +- **Tags**: `database`, `embedded`, `sql`, `edge` +- **Rationale**: Zero-configuration, serverless, self-contained SQL database engine adopted for edge gateway local telemetry caching, desktop testing tools, and embedded data logging. + +### GCC & Clang +- **Status**: **Adopt** +- **Tags**: `compilers`, `c`, `c++`, `toolchains` +- **Rationale**: Standard cross-compilers for ARM Cortex-M and embedded Linux target compilation. Enforcing strict `-Wall -Wextra -Werror` compiler hygiene in CI. + +### Visual Studio Code +- **Status**: **Adopt** +- **Tags**: `ide`, `editor`, `developer-experience` +- **Rationale**: Primary cross-platform integrated development environment with extensions for C/C++, Python, Rust, Docker, PlantUML, and Markdownlint. + +## Ring: Assess + +### Coverity +- **Status**: **Assess** +- **Tags**: `sast`, `security`, `static-analysis`, `quality` +- **Rationale**: Evaluating enterprise-grade deep static analysis for automated detection of concurrency race conditions, memory leaks, and buffer overflows in C/C++ firmware. + +### Structurizr DSL +- **Status**: **Assess** +- **Tags**: `architecture`, `c4`, `modelling`, `dsl` +- **Rationale**: Evaluating Structurizr DSL as a single-source-of-truth textual modelling tool capable of generating synchronized C4 diagrams across all abstraction levels ([GD-001](../guides/c4-model/index.md)). diff --git a/content/standards/api-design.md b/content/standards/api-design.md new file mode 100644 index 0000000..c7148b3 --- /dev/null +++ b/content/standards/api-design.md @@ -0,0 +1,227 @@ +--- +title: API design and error handling +--- + +# API design and error handling + +This guideline defines the engineering standards for designing robust, discoverable Application Programming Interfaces (APIs) and implementing systematic error management across cloud microservices, IoT edge gateways, and client applications. + +## Purpose and Scope + +Distributed systems communicate through interconnected network interfaces. Inconsistent endpoint design, ambiguous status codes, and unstandardised error responses hinder cross-team integration and complicate production troubleshooting. + +This standard applies to: +- Public and partner-facing RESTful HTTP APIs. +- Internal microservice APIs deployed within cloud and on-premises environments. +- Edge-to-cloud device integration APIs and API gateways. + +## 1. RESTful Resource Design Standards + +All HTTP APIs MUST adhere to standard REST architectural constraints: + +### 1.1 URI Structure and Naming Conventions + +URIs represent resources, not actions. HTTP verbs define the operations performed upon those resources: + +| Rule | Requirement | Compliant Example | Non-Compliant Example | +| :--- | :--- | :--- | :--- | +| **Resource Nouns** | Use plural nouns for collections and entities; avoid action verbs. | `/devices`, `/devices/{id}` | `/getDevice`, `/createDevice` | +| **Kebab-Case** | Use lowercase characters and hyphens for multi-word paths. | `/sensor-readings` | `/sensor_readings`, `/sensorReadings` | +| **Hierarchical Subresources** | Nest subordinate resources to denote ownership or containment. | `/devices/{id}/telemetry` | `/getTelemetryForDevice?id=12` | +| **Query Parameters** | Use query parameters exclusively for filtering, sorting, and pagination. | `/devices?status=active&page=2` | `/devices/active/page2` | + +[[CAPTION:Table]] REST URI structure and naming conventions. + +### 1.2 HTTP Method Semantics + +APIs MUST adhere strictly to the idempotency and safety guarantees defined in RFC 9110: + +| Method | Safety | Idempotency | Typical Operational Usage | Expected Success Status | +| :--- | :--- | :--- | :--- | :--- | +| `GET` | Safe | Idempotent | Retrieve a resource or collection representation. | `200 OK` | +| `POST` | Unsafe | Non-idempotent | Create a new subordinate resource or execute an operation. | `201 Created` or `202 Accepted` | +| `PUT` | Unsafe | Idempotent | Replace the target resource in its entirety with the payload. | `200 OK` or `204 No Content` | +| `PATCH` | Unsafe | Non-idempotent | Apply partial modifications to the target resource. | `200 OK` or `204 No Content` | +| `DELETE` | Unsafe | Idempotent | Remove the target resource. | `200 OK` or `204 No Content` | + +[[CAPTION:Table]] HTTP method safety, idempotency, and operational semantics. + +## 2. Comprehensive HTTP Status Code Reference + +HTTP status codes are standardised three-digit integers informing clients of the outcome of their request. APIs MUST NOT return generic `200 OK` responses containing application-level error codes inside the response body. + +### 2.1 1xx Informational Codes + +- **`100 Continue`**: Indicates the server has received request headers and the client should proceed to send the request body (useful for large file or firmware uploads). +- **`101 Switching Protocols`**: The server agrees to switch protocols requested by the client Upgrade header (for example, upgrading from HTTP/1.1 to WebSocket). + +### 2.2 2xx Success Codes + +| Code | Name | Semantic Definition and Usage | +| :--- | :--- | :--- | +| **`200`** | OK | Standard response for successful `GET`, `PUT`, or `PATCH` operations. Response body contains the resource representation. | +| **`201`** | Created | The request succeeded and a new resource was created. The response MUST include a `Location` header containing the URI of the newly created resource. | +| **`202`** | Accepted | The request has been accepted for asynchronous processing (for example, batch firmware provisioning). Processing is incomplete. | +| **`204`** | No Content | The server fulfilled the request successfully and there is no additional content to send in the response body (typical for `DELETE`). | +| **`206`** | Partial Content | The server is delivering only part of the resource due to a `Range` header sent by the client (essential for resumable firmware updates). | + +[[CAPTION:Table]] 2xx Success status codes. + +### 2.3 3xx Redirection Codes + +| Code | Name | Semantic Definition and Usage | +| :--- | :--- | :--- | +| **`301`** | Moved Permanently | The target resource has been assigned a new permanent URI. Clients MUST update bookmarks and caches to the URI in the `Location` header. | +| **`304`** | Not Modified | Indicates that the resource has not been modified since the version specified in request headers (`If-None-Match` or `If-Modified-Since`). Saves network bandwidth. | +| **`307`** | Temporary Redirect | The request should be repeated with another URI, but future requests should continue to use the original URI. The request method MUST NOT be changed. | +| **`308`** | Permanent Redirect | The target resource is now permanently located at another URI. Unlike 301, the request method MUST NOT change upon redirection. | + +[[CAPTION:Table]] 3xx Redirection status codes. + +### 2.4 4xx Client Error Codes + +Client error codes indicate that the client appears to have erred (for example, malformed syntax, unauthenticated request, or invalid parameter). + +| Code | Name | Semantic Definition and Usage | +| :--- | :--- | :--- | +| **`400`** | Bad Request | The server cannot process the request due to malformed request syntax, invalid query parameters, or deceptive routing. | +| **`401`** | Unauthorized | Authentication is required and has failed or has not yet been provided. The response MUST include a `WWW-Authenticate` header. | +| **`403`** | Forbidden | The server understood the request, but the authenticated identity lacks necessary permissions or scope to access the resource. | +| **`404`** | Not Found | The server cannot find the requested resource at the specified URI. | +| **`405`** | Method Not Allowed | The HTTP method used is not supported for the target resource (for example, `POST` on a read-only endpoint). Response MUST include `Allow` header. | +| **`408`** | Request Timeout | The server timed out waiting for the client to complete transmission of the request. | +| **`409`** | Conflict | The request could not be completed due to a conflict with the current state of the target resource (for example, optimistic locking collision). | +| **`410`** | Gone | The target resource is no longer available at the origin server and no forwarding address is known. Permanently removed. | +| **`413`** | Payload Too Large | The request payload exceeds the size ceiling accepted by the server or API gateway. | +| **`415`** | Unsupported Media Type | The format of the request payload (`Content-Type`) is not supported by the endpoint (for example, receiving XML on a JSON-only endpoint). | +| **`422`** | Unprocessable Content | The request syntax is well-formed, but semantic validation errors exist (for example, an invalid email address format or out-of-range sensor calibration). | +| **`429`** | Too Many Requests | The client has sent too many requests in a given time window. The response SHOULD include a `Retry-After` header indicating seconds to wait. | + +[[CAPTION:Table]] 4xx Client Error status codes. + +### 2.5 5xx Server Error Codes + +Server error codes indicate that the server encountered an unexpected condition that prevented it from fulfilling the valid request: + +| Code | Name | Semantic Definition and Usage | +| :--- | :--- | :--- | +| **`500`** | Internal Server Error | An unhandled exception or critical runtime failure occurred within the application code. | +| **`501`** | Not Implemented | The server does not support the functionality required to fulfill the request. | +| **`502`** | Bad Gateway | The server, while acting as a gateway or proxy, received an invalid response from an inbound upstream server. | +| **`503`** | Service Unavailable | The server is currently unable to handle the request due to temporary overload or scheduled maintenance. Response SHOULD include `Retry-After`. | +| **`504`** | Gateway Timeout | The server, while acting as a gateway or proxy, did not receive a timely response from the upstream back-end service. | + +[[CAPTION:Table]] 5xx Server Error status codes. + +## 3. Standardised Error Representation: RFC 9457 Problem Details + +All error responses (4xx and 5xx) returned by HTTP APIs MUST use the RFC 9457 standard (`application/problem+json`). Custom, ad-hoc JSON error structures are prohibited. + +### 3.1 Problem Details Schema + +An RFC 9457 error document consists of standard top-level fields: +- `type` (string): A URI reference identifying the specific error type. Defaults to `about:blank` if no specialised document exists. +- `title` (string): A short, human-readable summary of the problem type. MUST NOT change between occurrences of the error. +- `status` (integer): The HTTP status code generated by the origin server for this occurrence. +- `detail` (string): A human-readable explanation specific to this occurrence of the problem. +- `instance` (string): A URI reference that identifies the specific occurrence of the problem (for example, an audit log identifier or tracing correlation ID). + +### 3.2 RFC 9457 Payload Example + +```json +{ + "type": "https://api.example.com/errors/validation-error", + "title": "Unprocessable Content", + "status": 422, + "detail": "Device configuration parameters failed schema verification.", + "instance": "/errors/txn-9843-f8a1", + "timestamp": "2026-09-20T12:00:00Z", + "invalidParams": [ + { + "name": "sample_interval_sec", + "reason": "Value must be between 10 and 3600 seconds.", + "rejectedValue": 5 + }, + { + "name": "reporting_mode", + "reason": "Must be one of ['periodic', 'on_change', 'alarm'].", + "rejectedValue": "continuous" + } + ] +} +``` +[[CAPTION:Figure]] RFC 9457 compliant error payload with validation extensions. + +## 4. Resilience and Distributed Error Management + +When microservices and edge gateways communicate across unreliable networks, error management must encompass automated retry and isolation patterns. + +```mermaid +flowchart TD + A[Client Request] --> B[API Gateway / Service Client] + B --> C{Circuit Breaker State} + C -- OPEN --> D[Immediate Fail Fast: 503 / Cached Fallback] + C -- CLOSED / HALF-OPEN --> E[Send Request to Upstream Service] + E --> F{Response Received?} + F -- 2xx OK --> G[Return Success to Client] + F -- 4xx Client Error --> H[Return RFC 9457 Problem Details] + F -- 5xx / Network Timeout --> I{Retry Limit Reached?} + I -- No --> J[Exponential Backoff with Full Jitter] + J --> E + I -- Yes --> K["Send Message to Dead Letter Queue (DLQ)"] + K --> L[Trip Circuit Breaker to OPEN] + L --> D +``` +[[CAPTION:Figure]] End-to-end distributed error handling and resilience flow. + +### 4.1 Exponential Backoff with Jitter + +Client libraries MUST NOT retry immediately in a tight loop. Immediate retries create "thundering herd" problems that overwhelm recovering upstream servers. + +Clients MUST use exponential backoff calculated with randomized full jitter: + +$$\mathrm{Sleep\ Time} = \mathrm{random}(0, \min(T_{\max}, T_{\mathrm{base}} \times 2^{\mathrm{attempt}}))$$ + +```python +import random +import time + +def execute_with_retry(operation, max_attempts=5, base_delay_sec=0.5, max_delay_sec=10.0): + for attempt in range(max_attempts): + try: + return operation() + except (ConnectionError, TimeoutError) as exc: + if attempt == max_attempts - 1: + raise exc + # Calculate exponential backoff with full jitter + ceiling = min(max_delay_sec, base_delay_sec * (2 ** attempt)) + sleep_duration = random.uniform(0.0, ceiling) + time.sleep(sleep_duration) +``` +[[CAPTION:Figure]] Exponential backoff implementation with full jitter in Python. + +### 4.2 Circuit Breaker Pattern + +To protect downstream services from cascading failure, distributed clients MUST implement a Circuit Breaker with three operating states: +1. **Closed**: All requests flow normally. The circuit records failed calls. If the failure rate exceeds a threshold (for example, 50% over 10 seconds), the circuit trips to *Open*. +2. **Open**: Requests fail immediately without attempting upstream network communication, returning HTTP `503 Service Unavailable` or a cached fallback. +3. **Half-Open**: After a cooldown period, a single probe request is permitted through. If it succeeds, the circuit resets to *Closed*. If it fails, the circuit returns to *Open*. + +### 4.3 Dead-Letter Queue (DLQ) Pattern + +For asynchronous event-driven architectures (MQTT, Kafka, AMQP), messages that fail processing after exhausting all configured retry policies MUST be routed to a Dead-Letter Queue (DLQ). + +DLQ payloads MUST include: +- The original unmutated message payload. +- Error diagnostic headers (exception type, error message, stack trace summary). +- Ingestion timestamp and total retry attempt count. +- The originating service identifier. + +## Related Documents + +- [Software Detailed Specification Standard](../guides/software-design-specification.md) +- [LwM2M Objects Specification](lwm2m-objects.md) +- [Architecture Review Board (ARB) Guide](../guides/architecture-review-board.md) +- [Architecture Review Checklist](../templates/architecture-review-checklist.md) +- [Fundamental Software Architecture Principles](software-architecture-principles.md) +- [Defect Management and Systematic Debugging Guide](../guides/defect-management-and-debugging.md) diff --git a/content/standards/c-coding-standard.md b/content/standards/c-coding-standard.md new file mode 100644 index 0000000..a8e109a --- /dev/null +++ b/content/standards/c-coding-standard.md @@ -0,0 +1,971 @@ +--- +title: C coding standard +--- + +# C coding standard + +This document outlines the coding standards for C projects, focusing on readability, maintainability, and consistency. Adhering to these guidelines will help ensure high-quality code and facilitate collaborative development. + +This standard assumes a C99 or later compiler. + +## File Organisation + +A file should consist of logically separated sections, each identified by a blank line and an optional comment. + +### File Names + +- New file names should be all lowercase. +- Use `.h` for header files and `.c` for C source files. +- File names should clearly reflect the functionality defined or implemented within them. +- Files with a logical connection (e.g., header and source pairs) should reflect this in their names. +- Files belonging to the same module should use a short, unique prefix followed by an underscore (e.g., `module_filename.c`). + +Existing files may retain historical naming conventions to avoid unnecessary renaming. + +**Examples:** + +- `io_base.h`, `io_base.c` +- `server.c`, `server_ioplug.h`, `server_ioplug.c` + +### Source Files (`.c`) + +C source files should follow this ordering: + +1. Beginning comments (file header). +1. Version information. +1. `#include` statements. +1. `#define`s. +1. Local data type definitions. +1. Local (static) function prototypes. +1. Local (static) data. +1. Externally visible functions. +1. Local (static) functions. + +**Example File Header for `.c`:** + +``` c +/** + * @file template.c + * @brief Brief description of contents of file. + * @details Long description of the file's purpose and functionality. + * @date date of last edit @todo to be auto added + * @version CVS $Header$ field + * @copyright + * COPYRIGHT NOTICE: ® 20?? All rights reserved.
+ * Unauthorized copying of this file, via any medium is strictly prohibited + * Proprietary and confidential + * @addtogroup name-of-library-or-application-or-module + * @{ + */ +``` + +### Header Files (`.h`) + +All header files must begin with a comment. + +**Example File Header for `.h`:** + +```c +/** + * @file template.h + * @brief Brief description of contents of file. + * @details Long description of the header file's purpose and contents. + * @date date of last edit @todo to be auto added + * @version CVS $Header$ field + * @copyright + * COPYRIGHT NOTICE: ® 20?? All rights reserved.
+ * Unauthorized copying of this file, via any medium is strictly prohibited + * Proprietary and confidential + * @addtogroup name-of-library-or-application-or-module + * @{ + */ +``` + +To prevent multiple inclusions, header file contents must be protected by an `#ifndef ... #endif` pair: + +```c +#ifndef NAMEOFHEADERFILEINCAPITALS_H +#define NAMEOFHEADERFILEINCAPITALS_H 1 + +// ...body of header file... + +#endif /* NAMEOFHEADERFILEINCAPITALS_H */ +``` + +**General Rule:** Header files should generally *not* include system header files directly. Instead, document which system header files they require. + +## Indentation & Line Length + +### Indentation + +- Use **four spaces** as the unit of indentation. +- Do not use tabs as they can lead to inconsistent formatting. + +### Line Length + +- Avoid lines longer than **80 characters** for better readability across various terminals and tools. +- Note: Examples in documentation should be shorter, generally no more than 70 characters. + +### Wrapping Lines + +When an expression exceeds a single line, break it according to these principles: + +- Break after a comma. +- Break before an operator. +- Prefer higher-level breaks to lower-level breaks. +- Align the new line with the beginning of the expression at the same level on the previous line. +- If the above rules lead to confusing or squished code, indent by **8 spaces** instead. + +**Examples of Function Call Breaking:** + +```c +Function(longExpression1, longExpression2, longExpression3, + longExpression4, + longExpression5); + +var = Function(longExpression1, + Function2(longExpression2, + longExpression3)); +``` + +**Examples of Arithmetic Expression Breaking:** + +```c +LongName1 = LongName2 * (LongName3 + LongName4 - LongName5) + + 4 * longname6; /* PREFERRED */ + +LongName1 = LongName2 * (LongName3 + LongName4 +LongName5) + 4 * Longname6; /* AVOID! */ +``` + +**Examples of Function Declaration Indentation:** + +```c +/* CONVENTIONAL INDENTATION */ +Function(int AnArg, double AnotherArg, char *YetAnotherArg, + int *AndStillAnother) +{ + // ... +} + +/* INDENT 8 SPACES TO AVOID VERY DEEP INDENTS */ +static ReallyLongFunctionName(int AnArg, + double anotherArg, char *YetAnotherArg, + int *AndStillAnother) +{ + // ... +} + +/* OR, PUT EACH ARG ON OWN LINE */ +static ReallyLongFunctionName(int AnArg, + double anotherArg, + char *YetAnotherArg, + int *AndStillAnother) +{ + // ... +} +``` + +**Ternary Expressions:** Here are three acceptable ways to format ternary expressions: + +```c +alpha = (aLongBooleanExpression) ? beta : gamma; + +alpha = (aLongBooleanExpression) ? beta + : gamma; + +alpha = (aLongBooleanExpression) + ? beta + : gamma; +``` + +## Comments + +Programs can have two kinds of comments: **implementation comments** and **documentation comments**. + +- Implementation comments are delimited by `/* ... */` and are for commenting out code or providing details about the implementation. +- Documentation comments are delimited by `/** ... */` and are meant to describe the specification of the code, from an implementation-free perspective (e.g., for Doxygen). + +Comments should provide overviews and additional, non-obvious information. Avoid duplicating information already clear from the code. Generally, avoid comments that are likely to become outdated as the code evolves. + +**Note:** Frequent implementation comments can sometimes indicate unclear code. Consider refactoring the code to make it self-explanatory before adding comments. + +### Implementation Comment Formats + +Programs can use four styles of implementation comments: block, single-line, trailing, and end-of-line. + +#### Block Comments + +Used for descriptions of files, methods, data structures, and algorithms. They can appear at the beginning of files, before methods, or within functions. Block comments inside a function should be indented to the same level as the code they describe. + +A block comment should be preceded by a blank line. + +```c +/* + * Here is a block comment. + */ +``` + +#### Single-Line Comments + +Short comments that appear on a single line, indented to the level of the code that follows. If a comment requires more than one line, use the block comment format. A single-line comment should be preceded by a blank line. + +```c +if (condition) +{ + /* Handle the condition. */ + // ... +} +``` + +#### Trailing Comments + +Very short comments that appear on the same line as the code they describe. They should be shifted far enough to separate them from the statements. If multiple trailing comments appear in a code block, align them to the same tab setting. + +```c +if (a == 2) +{ + a = TRUE; /* special case */ +} +else +{ + a = isPrime(a); /* works only for odd a */ +} +``` + +### Documentation Comments + +We use **Doxygen style headers** for documenting C functions, structures, enums, unions, etc. Each doc comment is placed immediately before the declaration it describes, enclosed within `/** ... */` delimiters. For detailed information on Doxygen comment tags (`@return`, `@param`, `@see`), refer to [The Doxygen Manual](http://www.doxygen.org "null"). + +```c +/** + * @brief The function bar. + * @details This function performs a placeholder operation. This text is + * senseless and not intended for serious reading. + * @note This optional text demonstrates a "note" section. There is nothing + * critical here, so you don't need to read this section. + * @param[in] a Description of parameter a. + * @param[out] b Description of the parameter b. + * @param[in,out] c Description of the parameter c. + * @return The error return code of the function. + * @retval ERR_SUCCESS The function is successfully executed. + * @retval ERR_FAILURE An error occurred. + */ +errcode_t bar(int a, int b, int c) +{ + // ... +} +``` + +Some implementation detail does not belong in the documentation. Put it in a block or single-line comment immediately *after* the Doxygen comment and before the declaration. + +### Issue Tracker Integration for Comments + +When using Doxygen tags such as `@bug`, `@todo` or`@warning` within code comments, these **MUST** include a link to a corresponding item in the project's issue tracker (e.g., Jira, GitHub Issues). This ensures traceability and facilitates management of identified issues or future tasks. + +**Example:** + +```c +/** + * @brief The function bar. + * ... + */ +errcode_t bar(int a, int b, int c) +{ + /** More detailed description inside the code */ + /** + * @todo Add contents to this function - See TASK-123 + */ + + /// @warning Do not modify this!! - Refer to WARN-789 + /** + * @bug This is a bug, has been raised as + * BUG-440 + */ + return ERR_SUCCESS; +} +``` + +## Declarations + +### Number Per Line + +One declaration per line is recommended to encourage commenting. + +```c +int level; /* indentation level */ +int size; /* size of table */ +``` + +**Avoid:** + +```c +int level, size; /* AVOID! */ +``` + +Do not put different types on the same line: + +```c +int foo, fooarray[SIZE]; /* WRONG! */ +``` + +An alternative is to align a group of declarations with spaces: + +```c +int level; /* indentation level */ +int size; /* size of table */ +Object currentEntry; /* currently selected table entry */ +``` + +### Initialisation + +Initialise variables in a separate statement immediately after their declaration, unless the initial value depends on a prior computation. + +### Placement + +Place declarations at the beginning of blocks or immediately before first use. + +Mixing declarations and executable statements mid-block should be avoided, except where doing so improves clarity or limits variable scope. + +```c +void MyFunction(...) +{ + int int1; + int1 = 0; /* beginning of block */ + + if (condition) + { + int int2; + int2 = 0; /* beginning of "if" block */ + // ... + } +} +``` + +The only exception is for loop indices in C++, which can be declared within the `for` statement: + +```c +for (int i = 0; i < MaxLoops; i++) +{ + // ... +} +``` + +Avoid local declarations that hide declarations at higher levels (e.g., declaring the same variable name in an inner block). + +```c +int count; +// ... +int MyFunction(...) +{ + if (condition) + { + int count; /* AVOID! This hides the outer 'count' */ + // ... + } + // ... +} +``` + +### Function Declarations + +When coding C functions, use the following formatting rules: + +- All functions must be preceded by a documentation comment describing the function, its arguments, and return code(s). +- The opening brace `{` should appear at the beginning of the line following the declaration statement. +- The closing brace `}` should start a line by itself, aligned with the function's starting line. + +```c +/** + * @brief The function bar. + * @details This function does something which is doing nothing. So this + * text is totally senseless and you really do not need to read this, + * because this text is basically saying nothing. + * @note This optional text shall only show you, how such a "note" + * section is looking. There is nothing which really needs your notice, + * so you do not really need to read this section. + * @param[in] a Description of parameter a. + * @param[out] b Description of the parameter b. + * @param[in,out] c Description of the parameter c. + * @return The error return code of the function. + * @retval ERR_SUCCESS The function is successfully executed + * @retval ERR_FAILURE An error occurred + */ +errcode_t bar(int a, int b, int c) +{ + /** More detailed description inside the code */ + return ERR_SUCCESS; +} +``` + +## Statements ⚙️ + +### Simple Statements + +Each line should contain at most one statement. + +```c +argv++; /* Correct */ +argc--; /* Correct */ + +argv++; argc--; /* AVOID! */ +``` + +### Compound Statements + +Compound statements are lists of statements enclosed in braces `{ statements }`. + +- The enclosed statements should be indented one more level than the compound statement. +- The opening brace `{` should be on a line by itself, following the line that begins the compound statement, indented to the same level as that line. +- The closing brace `}` should begin a line by itself and be indented to the beginning of the compound statement. +- Always use braces around all statements, even single statements, when they are part of a control structure (e.g., `if-else`, `for`). This prevents accidental bugs when adding statements later. + +### `return` Statements + +A `return` statement with a value should not use parentheses unless they significantly improve clarity. + +```c +return; +return MyDisk_size(); +return (size ? size : defaultSize); +``` + +### `if`, `if-else`, `if else-if else` Statements + +The `if-else` class of statements should have the following form: + +```c +if (condition) +{ + statements; +} +else +{ + /* Do Nothing */ +} + +if (condition) +{ + statements; +} +else +{ + statements; +} + +if (condition) +{ + statements; +} +else if (condition) +{ + statements; +} +else +{ + statements; +} +``` + +**Always use braces `{}` with `if` statements.** Avoid the error-prone form that omits braces: + +```c +if (condition) /* AVOID! THIS OMITS THE BRACES {}! */ + statement; +``` + +### `for` Statements + +A `for` statement should have the following form: + +```c +for (initialization; condition; update) +{ + statements; +} +``` + +An empty `for` statement (where all work is done in the initialisation, condition, and update clauses) should have the following form: + +```c +for (initialization; condition; update); +``` + +When using the comma operator in the initialisation or update clause of a `for` statement, avoid the complexity of using more than three variables. If necessary, use separate statements before the `for` loop (for initialisation) or at the end of the loop (for updates). + +### `while` Statements + +A `while` statement should have the following form: + +```c +while (condition) +{ + statements; +} +``` + +An empty `while` statement should have the following form: + +```c +while (condition); +``` + +### `do-while` Statements + +A `do-while` statement should have the following form: + +```c +do +{ + statements; +} +while (condition); +``` + +### `switch` Statements + +A `switch` statement should have the following form: + +```c +switch (condition) +{ + case ABC: + statements; + /* falls through */ + case DEF: + statements; + break; + case XYZ: + statements; + break; + default: + statements; + break; +} +``` + +- Every time a `case` falls through (does not include a `break` statement), add a comment where the `break` statement would normally be (e.g., `/* falls through */`). +- Every `switch` statement should include a `default` case. The `break` in the `default` case is redundant but prevents fall-through errors if the `default` is later changed to a specific case and a new `default` is introduced. + +## White Space + +### Blank Lines + +Blank lines improve readability by logically separating sections of code. + +**Two blank lines should always be used:** + +- Between sections of a source file. + +**One blank line should always be used:** + +- Between functions. +- Between local variable declarations in a block and its first statement. +- Before a block or single-line comment. +- Between logical sections inside a function to improve readability. + +### Blank Spaces + +Blank spaces should be used in the following circumstances: + +- After commas in argument lists. +- All binary operators should be separated from their operands by spaces. +- Blank spaces should never separate unary operators (e.g., unary minus, increment `++`, decrement `--`) from their operands. + +**Examples:** + +```c +a += c + d; +a = (a + b) / (c * d); +while (d++ == s++) // Note: original had assignment, changed to equality for likely intent +{ + n++; +} +``` + +- Expressions in a `for` statement should be separated by blank spaces. + +**Example:** + +```c +for (expr1; expr2; expr3) +``` + +- Casts should be followed by a blank space. + +**Examples:** + +```c +MyFunction1((char) aNum, (double) x); +MyFunction2((int) (cp + 5), ((int) (i + 3) + 1)); +``` + +## Keywords to Frequently Use + +- The **`static`** keyword should be used to declare all functions and variables that do not need to be visible outside of the module in which they are declared. + - All non-API functions and file-local data shall be declared `static` by default. +- The **`const`** keyword should be used whenever appropriate, including: + - To declare variables that should not be changed after initialisation. + - To define call-by-reference function parameters that should not be modified (e.g., `char const * param`). + - To define fields in a `struct` or `union` that should not be modified (e.g., in a `struct` overlay for memory-mapped I/O peripheral registers). + - As a strongly typed alternative to `#define` for numerical constants. +- The **`volatile`** keyword should be used whenever appropriate, including: + - To declare a global variable accessible by any interrupt service routine. + - To declare a global variable accessible by two or more threads. + - To declare a pointer to a memory-mapped I/O peripheral register set (e.g., `timer_t volatile * const p_timer`). + - To declare a delay loop counter. + +## Naming Conventions + +Naming conventions enhance programme readability. + +### Externally Visible Functions + +Should have a short prefix uniquely identifying the module, followed by an underscore, followed by the rest of the identifier. The rest of the identifier should consist of words, each beginning with a capital letter, with no underscores. + +**Example:** `GAT_FindResource` + +### Static Functions + +Should follow the same convention as externally visible functions but do not require the module prefix. + +### Variable Names + +Should be short yet meaningful and mnemonic. Avoid one-character variable names except for temporary "throw-away" variables (e.g., `i, j, k, m, n` for integers; `c, d, e` for characters). + +### `#defines` + +Should be all uppercase. + +### `typedef`s and Structures + +Should have a short prefix, followed by an underscore, followed by the rest of the identifier. The rest of the identifier should consist of words, each beginning with a capital letter, with no underscores. + +**Example:** `GAT_PropList` + +### General Naming Practices + +- Variables: All variables should be declared in a consistent case (`lowerCamelCase`) + - Variables representing truth values (e.g., `TRUE`/`FALSE`) should be of `bool` type. + - If a variable stores a unit, append the unit type using SI notation (e.g., `FooMs` for milliseconds, `CountS` for seconds). + - Macros: all uppercase with underscores + - Types: `Prefix_TypeName` with `_t` suffix +- Functions: All functions should be declared in a consistent case (`PascalCase` for public APIs, `camelCase` for static functions). + - Example: `ThisIsAFunctionName()` + - All functions should return an appropriate success/error value where applicable. + + ```c + int ThisFunction(); /* Returns a signed 8-bit value */ + err_t ThisFunction(); /* Returns an enumerated error code or positive value */ + ``` + + - In the unlikely event that it's impossible to know if a function call succeeded, it may be `void`. + + ```c + void ThisFunction(); + ``` + +## Programming Practices 🧑‍💻 + +### Adherence to Standards + +- All code should adhere to the ISO C standards, aiming for the latest stable version (currently C17/C18). Where appropriate, consider adopting features from C23. +- We should aim for **MISRA C compliant code**, specifically **MISRA C:2012**, to ensure safety, security, and reliability, especially in embedded and real-time systems. + +### Unions + +Unions should generally be avoided. Strong justification must be provided for their use. + +### `goto` + +`goto` statements should generally be avoided. Strong justification must be provided for their use. + +### `do-while` + +`do-while` loops should generally be avoided. Strong justification must be provided for their use. + +### Use `typedef` (and `enum`s) in preference to `#defines` + +New types should be introduced via `typedef`s rather than `#define`s. This makes types visible in debuggers and allows for stronger type-checking by the compiler. Use `enum`s as appropriate. + +### Use of `const` qualifier + +Where possible, pointers should be passed using the `const` qualifier, especially for strings. + +### Global and Static variables + +Do not make any variable global or static without good reason. Access to module-level statics in other files can often be granted via a function call rather than making the variable global. + +### Constants + +Numerical constants (literals) should not be coded directly, except for -1, 0, and 1, or other numbers used in loop counters. Use named constants instead. + +### Variable Assignments + +- Avoid assigning several variables to the same value in a single statement. It reduces readability. + + ```c + fchar = lchar = 'c'; /* AVOID! */ + ``` + +- Do not use the assignment operator in a place where it can be easily confused with the equality operator. + + ```c + if (file = fopen(...)) /* AVOID! */ + { + // ... + } + ``` + + Should be written as: + + ```c + if ((file = fopen(...)) != NULL) + { + // ... + } + ``` + +- Do not use embedded assignments to improve run-time performance; this is the compiler's job. + + ```c + d = (a = b + c) + r; /* AVOID! */ + ``` + + Should be written as: + + ```c + a = b + c; + d = a + r; + ``` + +### Strong Typing + +C is a weakly typed language. Be mindful of this and ensure variables are used with appropriate types to prevent issues where a variable of type A is passed to a function expecting type B. + +### Miscellaneous Practices + +#### Parentheses + +Use parentheses liberally in expressions involving mixed operators to avoid operator precedence problems. Even if precedence seems clear to you, it might not be to others. + +```c +if (a == b && c == d) /* AVOID! */ +if ((a == b) && (c == d)) /* RIGHT */ +``` + +#### Variable Types + +All variables must take appropriate values for their type. For example, if a variable is a `bool` type, it must be set to `TRUE`/`FALSE`, not `0` or `1`. + +```c +bool LinkUp = FALSE; +``` + +#### Returning Values + +Try to make the structure of your programme match the intent. + +```c +if (booleanExpression) +{ + return true; +} +else +{ + return false; +} +``` + +Should instead be written as: + +```c +return booleanExpression; +``` + +Similarly: + +```c +if (condition) +{ + return x; +} +return y; +``` + +Should be written as: + +```c +return (condition ? x : y); +``` + +**Single Return Statement:** There should ideally be only one return statement in a function. Multiple returns can lead to confusion and are a frequent source of errors. + +```c +if (foo) +{ + return a; +} +// ... lots of code ... +return b; +``` + +Should be written as: + +```c +errcode_t retval; // Assuming errcode_t is the return type +if (foo) +{ + retval = a; +} +else +{ + // ... lots of code ... + retval = b; +} +return retval; +``` + +#### Expressions before `?` in the Conditional Operator + +If an expression containing a binary operator appears before the `?` in the ternary `?:` operator, it should be parenthesized. + +```c +(x >= 0) ? x : -x; +``` + +#### Special Comments + +- Use `@warning` in a comment to flag something that is "bogus but works." +- Use `@bug` in a comment to flag something that is "bogus and broken." + +#### Pointers + +All pointers passed to functions should be checked for validity. Static analysis tools can assist in identifying potential issues with pointer usage and validity. + +## Static Analysis 🔍 + +Static analysis should be performed on all code. This should be done using dedicated static analysis tools that support **MISRA C:2012** compliance checking. These tools should be **integrated into the CI/CD pipeline** to automate the enforcement of coding standards and identify issues early in the development cycle. + +## File Templates + +### `.h` Template + +```c +/** + * @file template.h + * @brief Brief description of contents of file. + * @details Long description of the header file's purpose and contents. + * @date date of last edit @todo to be auto added + * @version CVS $Header$ field + * @copyright + * COPYRIGHT NOTICE: ® 20?? All rights reserved.
+ * Unauthorized copying of this file, via any medium is strictly prohibited + * Proprietary and confidential + * @addtogroup name-of-library-or-application-or-module + * @{ + */ + +#ifndef NAMEOFHEADERFILEINCAPITALS_H +#define NAMEOFHEADERFILEINCAPITALS_H 1 + +/******************************************************************************* + * INCLUDES + ******************************************************************************/ + +/******************************************************************************* + * MACROS AND DEFINES + ******************************************************************************/ + +/******************************************************************************* + * TYPEDEFS + ******************************************************************************/ +typedef struct Books { + char title[50]; + char author[50]; + char subject[100]; + int book_id; +} book; + +/******************************************************************************* + * EXPORTED VARIABLES + ******************************************************************************/ + +/******************************************************************************** + * GLOBAL FUNCTION PROTOTYPES + ********************************************************************************/ +errcode_t bar(int a, int b, int c); + +#endif /* NAMEOFHEADERFILEINCAPITALS_H */ + +/** @} end of Doxygen group */ +``` + +### `.c` Template + +```c +/** + * @file template.c + * @brief Brief description of contents of file. + * @details Long description of the file's purpose and functionality. + * @date date of last edit @todo to be auto added + * @version CVS $Header$ field + * @copyright + * COPYRIGHT NOTICE: ® 20?? All rights reserved.
+ * Unauthorized copying of this file, via any medium is strictly prohibited + * Proprietary and confidential + * @addtogroup name-of-library-or-application-or-module + * @{ + */ + +/******************************************************************************* + * INCLUDES + ******************************************************************************/ + +/******************************************************************************* + * DEFINES + ******************************************************************************/ + +/******************************************************************************* + * LOCAL DATA TYPES + ******************************************************************************/ + +/******************************************************************************* + * LOCAL FUNCTION PROTOTYPES + ******************************************************************************/ + +/******************************************************************************* + * LOCAL DATA + ******************************************************************************/ + +/******************************************************************************* + * GLOBAL FUNCTIONS + ******************************************************************************/ +/** + * @brief The function bar. + * @details This function does something which is doing nothing. So this text + * is totally senseless and you really do not need to read this, + * because this text is basically saying nothing. + * @note This optional text shall only show you, how such a "note" section + * is looking. There is nothing which really needs your notice, so you + * do not really need to read this section. + * @param[in] a Description of parameter a. + * @param[out] b Description of the parameter b. + * @param[in,out] c Description of the parameter c. + * @return The error return code of the function. + * @retval ERR_SUCCESS The function is successfully executed + * @retval ERR_FAILURE An error occurred + */ +errcode_t bar(int a, int b, int c) +{ + /** More detailed description inside the code */ + + return ERR_SUCCESS; +} + +/******************************************************************************* + * LOCAL FUNCTIONS + ******************************************************************************/ + +/** @} end of Doxygen group */ +``` diff --git a/content/standards/constrained-binary-payloads.md b/content/standards/constrained-binary-payloads.md new file mode 100644 index 0000000..dd91724 --- /dev/null +++ b/content/standards/constrained-binary-payloads.md @@ -0,0 +1,170 @@ +--- +title: Constrained binary payloads +--- + +# Constrained binary payloads + +## 1. Executive Summary & Policy Scope + +Edge devices, industrial sensors and remote gateways often communicate over networks that are slow, metered or high-latency. Examples are cellular LPWAN (NB-IoT, LTE-M), satellite (Iridium, NTN) and wireless mesh (802.15.4, Wirepas). On these networks, sending uncompressed text such as JSON, XML or plain text costs a lot. Radio time goes up, batteries drain faster, carrier data bills rise and the microcontroller needs more RAM. + +This standard establishes mandatory requirements for payload serialization across all connected devices in the enterprise portfolio. + +```mermaid +flowchart LR + Sensor["Edge Sensor Data"] --> Encoder{"Serialization Engine"} + Encoder -->|RFC 8949 Canonical| CBOR["CBOR (Integer-Keyed)"] + Encoder -->|nanopb Static Struct| Proto["Protocol Buffers (v3)"] + CBOR --> Radio["LPWAN / Satellite Transceiver"] + Proto --> Radio + Radio --> Cloud["Cloud Telemetry Ingestion"] +``` + +[[CAPTION:Figure]] Approved binary serialization paths for constrained connected devices. + +### 1.1 Policy Mandate + +> [!IMPORTANT] +> The transmission of uncompressed textual JSON or XML over cellular LPWAN (NB-IoT, Cat-M1) or satellite links is strictly prohibited for routine periodic telemetry. All routine uplink and downlink messages must use approved compact binary serialization formats adhering to the standards defined herein. + +## 2. Supported Binary Serialization Formats + +Engineering teams must select between two approved serialization technologies based on system architecture requirements: + +| Criterion | CBOR (RFC 8949) | Protocol Buffers (nanopb) | +| :--- | :--- | :--- | +| **Primary Standard** | IETF RFC 8949 / RFC 8610 (CDDL) | Google Protocol Buffers v3 | +| **Schema Requirement** | Optional; verified via CDDL | Mandatory `.proto` file compiled at build time | +| **Self-Describing Nature**| Yes; embedded type tags permit dynamic parsing | No; strictly positional tag-value stream | +| **MCU Memory Model** | Streaming buffer or dynamic parser (< 2 KB RAM) | Zero-allocation static C structures (< 1.5 KB RAM) | +| **Protocol Affinity** | Native standard for CoAP and LwM2M | Ideal for MQTT, gRPC, and cloud streaming | +| **Schema Evolution** | Flexible; keys can be added dynamically | Strict; field numbers must remain immutable | + +[[CAPTION:Table]] Technical selection criteria for binary serialization standards. + +## 3. CBOR Implementation Standards (RFC 8949) + +When CBOR is selected, implementations must comply with the following architectural rules: + +### 3.1 Canonical Integer-Key Mapping + +To minimise payload overhead, CBOR maps must never use string literals as dictionary keys. All keys must be assigned deterministic, sequential unsigned integers: + +```cddl +; Concise Data Definition Language (CDDL) Schema Specification +telemetry_message = { + 0 => uint, ; Sequence Counter (0 to 4294967295) + 1 => uint, ; Unix Epoch Timestamp in seconds + 2 => int, ; Temperature in hundredths of a degree Celsius (for example, 2150 = 21.50 C) + 3 => uint, ; Relative Humidity in tenths of a percent (for example, 552 = 55.2%) + 4 => uint, ; Battery Terminal Voltage in millivolts (for example, 3620 mV) + ? 5 => bstr .size 8 ; Optional Device Hardware Status Flag Mask +} +``` + +[[CAPTION:Figure]] Concise Data Definition Language (CDDL) schema specification. + +By substituting string keys (such as `"battery_terminal_voltage"`) with single-byte integer indices (`4`), the payload size is reduced by up to 75%. + +### 3.2 Canonical Deterministic Encoding + +To support cryptographic hashing, signatures, and reproducible testing, CBOR encoders must follow the **Deterministic Core Encoding Rules of RFC 8949**: + +1. **Integer Sizing**: Integers must be encoded using the shortest possible byte sequence. Encoders must not encode the integer `15` as a 2-byte or 4-byte sequence; it must reside in the single initial header byte. +1. **Deterministic Map Sorting**: Keys within CBOR maps must be sorted in ascending numerical order (`0`, `1`, `2`, ...). +1. **Definite Lengths Only**: Indefinite-length arrays and maps (using break codes `0xFF`) are prohibited. All collections must declare their definite length upfront. + +## 4. Protocol Buffers Implementation Standards (nanopb) + +When Protocol Buffers are deployed on microcontroller firmware, the implementation must use **nanopb** or an approved static memory generator. + +### 4.1 Schema Definition & Tag Assignment + +All message definitions must adhere to Protocol Buffers version 3 syntax (`proto3`): + +```protobuf +syntax = "proto3"; + +package iot.telemetry.v1; + +message EnvironmentalTelemetry { + uint32 sequence_number = 1; + uint32 timestamp_epoch = 2; + sint32 temperature_centi_celsius = 3; // Uses sint32 for efficient ZigZag encoding + uint32 humidity_deci_percent = 4; + uint32 battery_millivolts = 5; + bytes status_flags = 6; +} +``` + +[[CAPTION:Figure]] Protocol Buffers telemetry schema definition. + +Critical Protobuf design rules: + +1. **Reserve Tags 1 to 15 for High-Frequency Fields**: Field numbers 1 through 15 require only a single byte on the wire to encode both the field number and wire type. Field numbers 16 through 2047 require 2 bytes. The most frequently transmitted measurements must use tags 1 to 15. +1. **Use `sint32` / `sint64` for Signed Negative Values**: Standard `int32` encodes negative numbers as full 10-byte varints. `sint32` uses ZigZag encoding, mapping small negative numbers (such as -1 °C) to compact 1-byte or 2-byte representations. +1. **Zero Dynamic Memory Allocation**: In embedded C firmware, `nanopb` options files (`.options`) must define fixed maximum sizes for all strings, byte arrays, and repeated fields: + +```text +// telemetry.options +iot.telemetry.v1.EnvironmentalTelemetry.status_flags max_size:8 +``` + +[[CAPTION:Figure]] Static memory options file for nanopb compilation. + +This guarantees that `pb_decode()` and `pb_encode()` operate purely on static stack structures without calling `malloc()` or risking heap fragmentation. + +## 5. Endianness, Struct Alignment & Padding Hazards + +Directly transmitting raw C structures over radio links (`memcpy` of `struct`) is strictly prohibited due to portability and compiler alignment pitfalls: + +```c +// PROHIBITED: Compilers insert silent padding bytes between fields +struct BadTelemetry { + uint8_t flag; // 1 byte + // 3 bytes of compiler padding inserted here on 32-bit MCUs! + uint32_t timestamp; // 4 bytes + uint16_t voltage; // 2 bytes + // 2 bytes of compiler padding inserted here! +}; +``` + +[[CAPTION:Figure]] Example of improper C structure prone to compiler padding hazards. + +```mermaid +flowchart TD + subgraph MemoryPadding["Compiler Structure Padding (Unpacked)"] + F1["flag (1 Byte)"] + Pad1["PAD (3 Bytes)"] + F2["timestamp (4 Bytes)"] + F3["voltage (2 Bytes)"] + Pad2["PAD (2 Bytes)"] + end +``` + +[[CAPTION:Figure]] Hidden compiler alignment padding wasting memory and network bandwidth. + +Architectural guidelines for binary structures: + +1. **Explicit Serialization**: Never cast raw pointers to structs. Use dedicated serialization helper functions or code generators. +1. **Network Byte Order**: All multi-byte numeric fields in custom binary headers must use Big-Endian (Network Byte Order) per IETF conventions, using explicit conversion macros (`htons()`, `htonl()`, `ntohs()`, `ntohl()`). + +## 6. Schema Evolution and Lifecycle Management + +As connected products evolve over decade-long deployments, message schemas inevitably change. Firmware and cloud platforms must maintain seamless interoperability across heterogeneous firmware revisions: + +1. **Immutable Field Numbers**: In Protocol Buffers and integer-keyed CBOR, a field number once deployed must never be changed, reassigned, or repurposed for a different measurement. +1. **Field Retirement Procedure**: If a measurement is discontinued, mark the field tag as `reserved` in the schema. This prevents future firmware developers from reusing the obsolete tag: + + ```protobuf + message EnvironmentalTelemetry { + reserved 3, 7; + reserved "legacy_optical_reading"; + // New fields start here + } + ``` + + [[CAPTION:Figure]] Protocol Buffers reserved field specification. + +1. **Unknown Field Resilience**: Cloud and edge decoders must ignore and discard unrecognised fields rather than aborting the decoding pipeline. Both CBOR and Protobuf inherently support skipping unknown tags. +1. **Mandatory Schema Versioning in Continuous Integration**: All `.proto` and `.cddl` files must reside in a centralised repository. Automated CI pipelines must run `buf lint` or `buf breaking` checks to fail any pull request that introduces backward-incompatible wire format alterations. diff --git a/content/standards/device-serialisation-and-factory-test.md b/content/standards/device-serialisation-and-factory-test.md new file mode 100644 index 0000000..1057756 --- /dev/null +++ b/content/standards/device-serialisation-and-factory-test.md @@ -0,0 +1,128 @@ +--- +title: Serialisation, provisioning and factory test +--- + +# Serialisation, provisioning and factory test + +## 1. Executive Summary & Production Policy + +Mass-manufacturing connected IoT devices needs rigorous automated checks before units leave the contract manufacturer. Each printed circuit board assembly (PCBA) is verified, given a deterministic identity, injected with its keys and locked. Inadequate factory testing results in high return rates and field dead-on-arrivals (DOA), while compromised provisioning processes introduce forged identities and supply chain vulnerabilities. + +This standard governs hardware serialization numbering schemes, hardware root-of-trust provisioning, automated test fixtures (ATE / bed-of-nails), RF calibration, and physical debug port locking. + +```mermaid +flowchart LR + SMT["Surface Mount Assembly (PCBA)"] --> Fixture["Bed-of-Nails Automated Test Station"] + Fixture --> Identity["Identity & Cert Injection (HSM)"] + Identity --> RFTest["Conducted RF & Current Profiling"] + RFTest --> Fuse["Blow JTAG / SWD Debug Fuses"] + Fuse --> Pack["Laser Marking & Box Packaging"] +``` + +[[CAPTION:Figure]] Manufacturing end-of-line testing, provisioning, and locking sequence. + +## 2. Universal Hardware Serialization Scheme + +Every manufactured device must possess globally unique, tamper-evident identifiers inscribed in both digital non-volatile memory and physical external housing. + +### 2.1 Identifier Taxonomy + +| Identifier Type | Standard / Format | Storage Location | Visibility | Purpose | +| :--- | :--- | :--- | :--- | :--- | +| **Hardware Serial Number** | GS1 AI(21) alphanumeric string (16-32 chars) | Internal OTP Flash / Secure Element | Laser-etched 2D DataMatrix barcode | Inventory tracking, warranty, and field logistics | +| **IEEE MAC Address** | IEEE EUI-48 / EUI-64 hexadecimal | Secure Element / Radio OTP registers | Network frames | Local wireless addressing (BLE, Wi-Fi, 802.15.4) | +| **Cellular Identity (IMEI)** | 15-digit decimal string (3GPP TS 23.003) | Cellular Baseband NVRAM | Carrier network registration | Cellular operator whitelisting and SIM pairing | +| **Cryptographic Device ID**| SHA-256 fingerprint of Device Public Key | Secure Element X.509 Certificate | Cloud authentication handshake | Zero-touch cloud onboarding and mutual TLS (mTLS) | + +[[CAPTION:Table]] Standard identifiers assigned during factory provisioning. + +### 2.2 External Laser Marking & Optical Verification + +1. All outer enclosures and internal PCB shields must be marked with a high-contrast **2D DataMatrix** (ISO/IEC 16022) or **QR Code** containing the serial number and device public key hash. +1. The automated test fixture must verify readability using an industrial vision camera before releasing the board clamp. + +## 3. Cryptographic Provisioning & Hardware Root of Trust + +Hardcoding shared symmetrical keys or compiling private keys directly into firmware binary images is strictly prohibited. Every connected device must incorporate a dedicated **Secure Element** (such as Microchip ATECC608, NXP SE050, or Infineon OPTIGA Trust M) or an on-die Hardware Security Module (HSM). + +```mermaid +sequenceDiagram + participant Station as Factory Provisioning Station + participant HSM as Factory Local HSM + participant SE as Device Secure Element (DUT) + participant Cloud as Cloud Device Registry (CA) + + Station->>SE: Issue GenerateKeyPair(Slot 0) command via I2C + SE->>SE: Generate private key inside tamper-proof silicon + SE-->>Station: Return Device Public Key (ECC P-256) + Station->>HSM: Sign Certificate Request with Factory Intermediate CA + HSM-->>Station: Return signed Device Certificate (X.509) + Station->>SE: Write Certificate into Data Slot 1, then Lock Slot + Station->>Cloud: Register Device Public Key & Serial Number via mTLS API + Cloud-->>Station: Registration Confirmed +``` + +[[CAPTION:Figure]] Secure on-die key generation and certificate signing workflow. + +### 3.1 Provisioning Rules + +1. **On-Die Private Key Generation**: Private keys must be generated inside the Secure Element itself using a hardware True Random Number Generator (TRNG). Private keys must never leave the silicon boundary under any circumstance. +1. **Air-Gapped Factory HSM**: Device certificates must be signed locally at the contract manufacturer by a hardware security module (HSM) holding a restricted intermediate factory CA certificate. The root CA remains offline. +1. **Permanent Slot Locking**: Once the public certificate, device configuration flags, and I2C address are programmed into the Secure Element, the manufacturing script must blow the configuration fuses (`lock_config` and `lock_data`). + +## 4. Automated Test Equipment (ATE) Fixture Design + +To achieve deterministic test throughput (< 45 seconds per unit), production lines must use custom **Bed-of-Nails (Pogo Pin)** automated test fixtures: + +```mermaid +flowchart TD + subgraph Fixture["Pogo Pin Test Fixture"] + PogoVCC["Pogo Pins: Switched Power Rails"] + PogoSWD["Pogo Pins: SWD / JTAG Flash Bus"] + PogoUART["Pogo Pins: Factory Test Console UART"] + PogoSens["Pogo Pins: Analog/Digital Sensor Injection"] + RFConn["Spring-Loaded Coaxial RF Probe"] + end + + subgraph Station["Automated Test Host"] + Tester["PC Test Orchestrator (Python / LabVIEW)"] + Analyzer["Dynamic Current Profiler"] + Programmer["High-Speed SWD Flasher (J-Link / Cyclone)"] + GoldenRadio["Golden RF Transceiver & Attenuator"] + end + + Station --> Fixture + Programmer --> PogoSWD + Analyzer --> PogoVCC + GoldenRadio --> RFConn +``` + +[[CAPTION:Figure]] Bed-of-nails fixture subsystem layout for automated production line testing. + +### 4.1 Mandatory Test Phases + +Every production board must pass four sequential test gates: + +1. **Power-On Short-Circuit & Quiescent Check**: Before full voltage is applied, the fixture checks for short circuits on all power rails. The power analyzer measures initial quiescent current to confirm it matches nominal baseline ($\pm 10\%$). +1. **Firmware Flashing & Integrity**: The production bootloader, primary application image, and factory diagnostic firmware are flashed via high-speed SWD. Readback CRC32/SHA-256 checksums are verified against the master build manifest. +1. **Conducted RF Verification**: To avoid antenna interference from neighboring factory equipment, RF performance is verified through a spring-loaded coaxial switch probe directly contacting the PCB RF test pad. The system verifies transmission power (Tx Power $\ge +20\,\mathrm{dBm}$) and receiver sensitivity across operating bands. +1. **Sensor Calibration**: Onboard sensors (accelerometers, temperature sensors, pressure transducers) are calibrated against precision reference standards in the fixture, and offset calibration factors are written to flash. + +## 5. Security Hardening: Debug Interface Locking + +Leaving physical debug interfaces (JTAG, SWD, UART bootloaders) enabled on field-deployed products allows adversaries to dump firmware binaries, reverse-engineer proprietary algorithms, and extract cryptographic credentials. + +> [!CAUTION] +> Every device undergoing end-of-line packaging must have its hardware debug interfaces permanently disabled via One-Time Programmable (OTP) hardware fuses before leaving the manufacturing station. + +### 5.1 Debug Port Disabling Procedure + +1. After passing all functional tests and certificate verification, the test orchestrator executes the hardware lock command (for example, programming STM32 Readout Protection to RDP Level 2, or blowing NXP / Nordic access port protection fuses - APPROTECT). +1. Once RDP Level 2 is active, the SWD/JTAG debug port is permanently disabled at the silicon level. Debug probes can no longer connect, halt execution, or inspect memory. +1. The test station verifies that the debug probe can no longer read flash or registers. If the port remains open, the board is rejected. + +## 6. Manufacturing Database Integration & Traceability + +1. **Centralized Manufacturing Execution System (MES)**: Every test step, voltage measurement, RF power reading, and certificate hash must be logged in real time to the enterprise MES database indexed by the unique hardware serial number. +1. **Anti-Cloning Rules**: The MES must reject duplicate serial numbers or certificate requests immediately. +1. **Defect Quarantining**: Any board failing a test gate must have its serial number blacklisted in the cloud device registry, preventing rejected factory scrap from ever connecting to production cloud environments. diff --git a/content/standards/index.md b/content/standards/index.md index 95ac330..79042e7 100644 --- a/content/standards/index.md +++ b/content/standards/index.md @@ -5,6 +5,39 @@ author: Mark Bacon # Standards -Coding and platform standards for C, Python, MISRA, LwM2M and constrained IoT devices. +Technical standards for the code and the devices: how C and Python are written, how interfaces and identifiers are designed, and what a field-deployed connected device must do. -This section is moving in from the specialised GU- guidelines in technical-documentation. +The [Guidelines](../guidelines/index.md) say how the engineering is run, and require a coding standard for each language ([EG-013](../guidelines/coding-standard.md)). These pages are those standards, and the device and interface standards that sit alongside them. The [Guides](../guides/index.md) go into depth, and the [Primers](../primers/index.md) introduce the topics. + +## Coding + +[[CAPTION:Table]] Coding standards. + +| Standard | Covers | +| --- | --- | +| [C coding standard](c-coding-standard.md) | Layout, naming, declarations, statements and comments for C, with a focus on embedded firmware | +| [MISRA C](misra-c.md) | What MISRA C is, how to approach violations and deviations, and how it fits the workflow | +| [Python for supporting tools](python.md) | Style, type hints, docstrings, errors, structure and tooling for build scripts and other supporting tools | + +## Design and interfaces + +[[CAPTION:Table]] Design and interfaces standards. + +| Standard | Covers | +| --- | --- | +| [Software architecture principles](software-architecture-principles.md) | Component, SOLID and package principles, and the simplicity principles behind them | +| [API design and error handling](api-design.md) | Resource design, status codes, problem details (RFC 9457), versioning and pagination | +| [Unique identifiers](unique-identifiers.md) | Choosing and storing UUIDs and their time-ordered alternatives, and the effect on database indexes | +| [Designing LwM2M objects and resources](lwm2m-objects.md) | Grouping data into objects, choosing resource types and access, and when to split an object | + +## Connected devices + +[[CAPTION:Table]] Connected devices standards. + +| Standard | Covers | +| --- | --- | +| [IoT device architecture principles](iot-device-architecture-principles.md) | Principles for field-deployed devices in seven pillars, with device tiers and review checks | +| [Battery budgeting and power profiling](iot-battery-budgeting.md) | Energy budgets, low-power state machines, battery chemistry and current profiling | +| [Constrained binary payloads](constrained-binary-payloads.md) | When and how to use CBOR and Protocol Buffers on constrained networks | +| [Offline storage and synchronisation](iot-edge-offline-storage.md) | Flash storage, wear levelling, buffering and synchronising data when a device has been offline | +| [Serialisation, provisioning and factory test](device-serialisation-and-factory-test.md) | Device identities, key injection, hardware locking and automated factory test | diff --git a/content/standards/iot-battery-budgeting.md b/content/standards/iot-battery-budgeting.md new file mode 100644 index 0000000..9365a77 --- /dev/null +++ b/content/standards/iot-battery-budgeting.md @@ -0,0 +1,155 @@ +--- +title: Battery budgeting and power profiling +--- + +# Battery budgeting and power profiling + +## 1. Executive Summary & Purpose + +Connected devices in remote, industrial, commercial and outdoor settings often run on non-rechargeable primary batteries or energy-harvesting hybrids. They are expected to last from 5 to 15 years. Premature battery exhaustion in the field causes severe economic loss, necessitates emergency physical truck rolls, and degrades service-level agreements. + +This standard sets the engineering practices for battery energy budgets, deterministic low-power state machines and the choice of cell chemistry. It also covers passivation, and measuring current consumption in every phase of firmware development. + +```mermaid +flowchart LR + A["1. Chemistry & Cell Sizing"] --> B["2. Firmware State Machine"] + B --> C["3. Dynamic Duty-Cycling"] + C --> D["4. Empirical Power Profiling"] + D --> E["5. Lifetime Verification"] +``` + +[[CAPTION:Figure]] End-to-end battery budgeting and power verification workflow. + +## 2. Mathematical Battery Budget Modelling + +Every battery-powered IoT hardware and firmware development programme must maintain an audited, version-controlled mathematical energy budget model prior to hardware schematic sign-off. + +### 2.1 The Discrete Operational Energy Formula + +The device's total energy use is modelled as the sum of its operating states over an evaluation period $T_{cycle}$. It is then scaled for operating temperature and self-discharge: + +$$Q_{cycle} = \sum_{k=1}^{n} \left( I_k \times t_k \right)$$ + +Where: + +- $I_k$ is the average current draw during operational state $k$ (in milliamperes or microamperes). +- $t_k$ is the duration spent in operational state $k$ per cycle (in seconds or hours). +- $Q_{cycle}$ is the total electrical charge consumed during one complete cycle (in milliampere-hours, $\mathrm{mAh}$). + +The projected device operating lifespan in years, $L_{years}$, is determined by: + +$$L_{years} = \frac{C_{nom} \times \eta_{temp} \times \eta_{voltage} \times (1 - S_{annual})^{L_{years}}}{I_{avg\_annual} \times 8760}$$ + +Where: + +- $C_{nom}$ is the manufacturer nominal battery capacity at +20°C (in $\mathrm{mAh}$). +- $\eta_{temp}$ is the environmental temperature de-rating efficiency factor (ranging from $0.60$ at -20°C to $1.0$ at +20°C). +- $\eta_{voltage}$ is the usable capacity factor dictated by the minimum operating cutoff voltage of the microcontroller and radio. +- $S_{annual}$ is the annual self-discharge rate of the cell chemistry (typically 1% to 2% per year for lithium chemistries). +- $I_{avg\_annual}$ is the time-weighted average continuous current consumption across all operational modes. + +### 2.2 Peukert's Effect and High-Pulse Current Derating + +Batteries exhibit non-linear capacity reduction when subjected to high-current discharge pulses. While an MCU in sleep consumes $3\,\mu\mathrm{A}$, a cellular radio transmitter burst draws between $150\,\mathrm{mA}$ and $400\,\mathrm{mA}$. Drawing high peak currents increases internal $I^2R$ resistive heating inside the cell and causes instantaneous voltage drop: + +$$V_{terminal} = V_{OCV} - I_{pulse} \times R_{internal}$$ + +If $V_{terminal}$ drops below the brownout reset (BOR) threshold of the microcontroller or radio module (typically 2.2 V to 2.8 V), the system resets uncontrollably, resulting in a reboot loop and catastrophic battery depletion. + +## 3. Battery Chemistry Selection Matrix + +Selecting the appropriate primary battery chemistry requires balancing energy density, operating temperature ranges, peak pulse capability, and chemical shelf-life: + +| Chemistry | Nominal Voltage | Energy Density | Pulse Capability | Temperature Range | Self-Discharge / yr | Typical Applications | +| :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| **$Li-SOCl_2$ (Lithium Thionyl Chloride)** | 3.6 V | Very High (up to 650 Wh/kg) | Low (Needs HLC / Supercap) | -60°C to +85°C | < 1.0% | 10+ year outdoor environmental sensors | +| **$Li-MnO_2$ (Lithium Manganese Dioxide)** | 3.0 V | High (up to 300 Wh/kg) | High (Continuous pulses) | -40°C to +70°C | < 1.5% | Asset trackers, smart home devices | +| **$LiFePO_4$ (Lithium Iron Phosphate)** | 3.2 V | Medium (Rechargeable) | Very High (10C discharge) | -20°C to +60°C | ~2.0% | Solar-harvested edge gateways | +| **Alkaline ($Zn-MnO_2$)** | 1.5 V | Low (up to 140 Wh/kg) | Moderate (Drops rapidly) | -10°C to +50°C | ~2.0% to 5.0% | Low-cost consumer indoor sensors | + +[[CAPTION:Table]] Primary battery chemistry comparison for autonomous IoT devices. + +### 3.1 Mitigating Passivation in Lithium Thionyl Chloride ($Li-SOCl_2$) Cells + +$Li-SOCl_2$ cells achieve decade-long shelf lives because an internal insulating passivating film of lithium chloride ($LiCl$) forms across the lithium anode, halting chemical self-discharge. However, this passivation layer dramatically increases internal cell impedance. When the radio attempts an RF transmission burst after months of sleep, the instantaneous voltage drops sharply until the layer is mechanically and electrochemically broken down. + +```mermaid +flowchart LR + subgraph CellSolution["Hybrid Pulse Architecture"] + Cell["Li-SOCl2 Primary Cell
(High Capacity, High Impedance)"] + Cap["Hybrid Layer Capacitor (HLC) / Supercapacitor
(Low Impedance, High Pulse Burst)"] + end + Cell -->|"Trickle Recharges (1-5 mA)"| Cap + Cap -->|Delivers 250-500 mA Peak Bursts| Radio["Cellular / Satellite RF Radio"] +``` + +[[CAPTION:Figure]] Hybrid layer capacitor parallel topology buffering high-current radio pulses. + +Engineering requirements for $Li-SOCl_2$ systems: + +1. **Parallel Pulse Buffer**: Hardware designs must incorporate a Hybrid Layer Capacitor (HLC) or pulse capacitor in parallel with the primary cell to supply peak transmission bursts without stressing the chemical cell. +1. **Firmware Depassivation Routine**: Devices waking from prolonged storage must execute an active depassivation cycle by turning on a controlled resistive load (or MCU core at full clock frequency) for 10 to 30 seconds before asserting radio power rails. + +## 4. Firmware Low-Power State Machine Architecture + +Firmware must be architected as an event-driven state machine that enforces aggressive duty-cycling. The system must spend greater than 99.5% of its operating life in the lowest achievable sleep state. + +```mermaid +stateDiagram-v2 + [*] --> DeepSleep: Power Applied / Reset + DeepSleep --> RTCWake: RTC Alarm / GPIO Interrupt + RTCWake --> SensorSample: Enable Peripherals + SensorSample --> ProcessData: Read ADC / I2C Bus + ProcessData --> RadioBurst: Threshold Exceeded / Interval Met + ProcessData --> DeepSleep: Value Normal & Buffer Unfilled + RadioBurst --> NetworkAck: Transmit Payload (Uplink) + NetworkAck --> DeepSleep: Acknowledged / Timeout +``` + +[[CAPTION:Figure]] Firmware power-state lifecycle transitioning between sleep, acquisition, and transmission. + +### 4.1 Power State Characteristics + +| State | Nominal Current | Typical Duration | Peripherals Enabled | Actions Executed | +| :--- | :--- | :--- | :--- | :--- | +| **1. Deep Sleep (PSM)** | 2.5 $\mu\mathrm{A}$ to 5 $\mu\mathrm{A}$ | 300 s to 86400 s | Low-frequency RTC (32.768 kHz), Backup RAM | Retain RAM state; core logic unpowered | +| **2. Sensor Sample** | 1.5 mA to 4.0 mA | 15 ms to 50 ms | High-speed internal RC clock, I2C/SPI bus, ADC | Read environmental transducers; power down sensors | +| **3. Processing & Flash** | 8.0 mA to 15.0 mA | 5 ms to 20 ms | Flash memory controller, crypto engine | Filter readings; write to wear-leveled storage | +| **4. Radio Sync / TX** | 120 mA to 350 mA | 1.5 s to 8.0 s | Cellular / Satellite / Sub-GHz PA transceiver | Modulate RF carrier; transmit encrypted telemetry | +| **5. Network RX Window** | 15 mA to 45 mA | 500 ms to 2000 ms | Baseband receiver, LNA | Listen for cloud downlinks and acknowledgments | + +[[CAPTION:Table]] Electrical parameters across operational firmware states. + +### 4.2 Critical Low-Power Firmware Design Rules + +1. **Eliminate Floating GPIO Lines**: Unconnected or floating microcontroller pins oscillate randomly due to ambient electrical noise, consuming tens of microamperes per pin through CMOS input buffers. All unused pins must be configured in firmware as Analog Input mode or pulled to a deterministic rail (Pull-up or Pull-down). +1. **Disable Internal Brownout Detectors in Deep Sleep**: While brownout protection is mandatory during active execution, standard BOR circuits consume continuous quiescent current. Use ultra-low-power pulsed BOR or switch BOR off during deep sleep where hardware sequencing is safe. +1. **Employ Tickless RTOS Scheduling**: In standard RTOS configurations, a system timer interrupt fires every millisecond (1 kHz sys-tick), preventing the processor from remaining in deep sleep. The firmware must implement tickless idle, calculating the exact sleep interval until the next scheduled software timer and reprogramming the low-power hardware timer accordingly. +1. **Cut Off Peripheral Power Rails via P-Channel MOSFETs**: External sensors, operational amplifiers, and flash chips frequently leak current in standby. The hardware must place high-side load switches on peripheral power rails, allowing firmware to isolate external circuits completely during sleep. + +## 5. Empirical Power Profiling Methodology + +Calculated energy models must be validated against real hardware silicon across all operating temperatures (-20°C, +25°C, +60°C) using precision laboratory instrumentation. + +```mermaid +flowchart LR + Host["Analysis PC
(Keysight / Keithley Software)"] --> Analyzer["DC Power Analyzer / Joulescope
(Dynamic Range: 10 nA to 2 A)"] + Analyzer -->|"Controlled Voltage (3.6 V)"| DUT["Device Under Test (DUT)
(Production PCB in RF Box)"] + DUT -->|Debug Serial Log| Host +``` + +[[CAPTION:Figure]] Instrumentation topology for empirical dynamic power profiling. + +### 5.1 Required Instrumentation + +1. **Dynamic Current Measurement**: Instruments must feature seamless auto-ranging across 6 decades of dynamic current range (from 10 nA to 2 A) with sampling rates of at least 100 kSa/s (such as the Keysight N6705C DC Power Analyzer or Joulescope JS220). Standard multimeters cannot capture microsecond transmission current spikes and will corrupt measurements through burden voltage drops. +1. **RF Isolation**: Radio profiling must take place inside a sealed RF shielding enclosure to ensure reproducible cellular link conditions and prevent the radio from ramping up transmission power unpredictably. + +### 5.2 Verification Checklist & Acceptance Criteria + +Every firmware release candidate must pass the following empirical power gates: + +1. **Quiescent Deep Sleep Gate**: The physical current draw in deep sleep must not exceed 110% of the calculated component quiescent limit ($I_{sleep} \le 5.5\,\mu\mathrm{A}$). +1. **Transient Peak Gate**: Maximum peak current during RF transmission must not exceed the maximum rating of the parallel capacitor / HLC ($I_{peak} < 450\,\mathrm{mA}$). +1. **Brownout Margin Gate**: Under worst-case cold temperature (-20°C) and maximum transmission pulse, the terminal voltage at the microcontroller VCC rail must remain at least $300\,\mathrm{mV}$ above the hardware BOR reset threshold. +1. **Runaway Watchdog Gate**: If an external sensor or radio module fails to respond over I2C/SPI/UART, a hardware timer must abort the transaction within 500 ms and force the device back to sleep, preventing permanent battery drain. diff --git a/content/standards/iot-device-architecture-principles.md b/content/standards/iot-device-architecture-principles.md new file mode 100644 index 0000000..98227ec --- /dev/null +++ b/content/standards/iot-device-architecture-principles.md @@ -0,0 +1,698 @@ +--- +title: IoT device architecture principles +--- + +# IoT device architecture principles + +**Target Product Domains:** Field-deployed IoT devices operating autonomously in uncontrolled physical environments over multi-year lifecycles +**Author:** Solution Architecture +**Status:** Approved / Baseline +**Document Reference:** [IoT device architecture principles](iot-device-architecture-principles.md) + +## Version History + +| Version | Date | Summary of Change | +| :-- | :-- | :-- | +| 1.0 | Baseline | Original 25-principle, six-pillar architecture baseline for smart meters. | +| 1.1 | 2026-09-14 | Restructured to 33 principles across seven pillars; added a Verification field to every principle; resolved the Rust/MISRA C language boundary; style and terminology consistency fixes. | +| 1.2 | 2026-09-14 | Broadened scope from smart metering specifically to IoT devices generally, using smart meters as the worked example throughout. | +| 1.3 | 2026-09-14 | Removed smart-metre-specific examples and terminology; principles are now fully domain-generic. | +| 1.4 | 2026-09-14 | Added the Quick-Reference Summary. | +| 1.5 | 2026-09-14 | Added the Glossary and this version history table. | +| 1.6 | 2026-09-14 | Added the Applicability Tiers framework; scoped Pillar 1 and the resource-intensive Principles 5, 9, 23, and 25 to device criticality (Tier 1-3). | +| 1.7 | 2026-09-14 | Added 11 new principles covering hardware manufacturability and EMC/RF coexistence, fail-safe hardware defaults, verified boot chain, configuration management, crash diagnostics, cross-version interoperability, firmware layering, real-time timing budgets, systematic fault handling, and reproducible builds. Grew to 44 principles; all cross-references and the Quick-Reference Summary renumbered. | +| 1.8 | 2026-09-14 | Added 3 new principles to Pillar 5: staged release pipeline/rollout governance, feature flag and hardware/firmware variant management, and a unified versioning and compatibility scheme. Grew to 47 principles; all cross-references and the Quick-Reference Summary renumbered accordingly. | + +> **Note:** Dates above reflect the working session in which each revision was made; adjust to reflect actual sign-off dates before circulating as a controlled document. + +## Executive Summary + +> [!NOTE] Relationship to Software Architecture Principles ([Software architecture principles](software-architecture-principles.md)) +> While [IoT device architecture principles](iot-device-architecture-principles.md) addresses physical, firmware, and whole-device architecture for constrained and safety-critical edge hardware, all firmware and edge software implementations MUST simultaneously adhere to the foundational software engineering principles documented in [Software architecture principles](software-architecture-principles.md) (such as SOLID, Dependency Inversion, Modular Cohesion, and Defensive Programming). + +Field-grade IoT devices are a distinct engineering challenge. Many must run continuously for years or decades in hostile, uncontrolled environments, with little or no local maintenance. Battery-powered devices must account for every milliamp-second against a fixed, non-rechargeable, or hard-to-replace power source. Mains- or vehicle-powered devices must withstand electrical transients and continuous environmental stress. Many devices in this category handle regulated functions, financial transactions, or safety-critical control. Their unattended field deployment makes them high-value targets for local tampering and remote cyber exploitation. + +These principles apply to the IoT device as a whole, not only to its firmware and software. Mechanical enclosure, regulatory certification, installation ergonomics, end-user or operator-facing accessibility, and end-of-life handling are first-order architectural concerns alongside core function, security, and code quality. + +This document establishes 47 foundational architecture principles across seven core pillars: + +1. **Core Function Isolation, Functional Safety & Edge Autonomy** +2. **Device-Centric Security & Cryptography** +3. **Power Budgeting & Environmental Endurance** +4. **Product Design, Hardware Modularity & Total Lifecycle Cost** +5. **Firmware Architecture & Storage Integrity** +6. **Code Quality, Memory Safety & Verification** +7. **Whole-Device Compliance, Human Factors & Sustainability** + +Each principle follows a structured format: **Statement**, **Rationale**, **Engineering Implications**, and **Verification**. The verification says how conformance is checked, so each principle can serve as an architecture review gate, not only as a reference note. + +## Applicability Tiers + +Not every principle applies with equal force to every device in scope. Three tiers describe how obligations scale with a device's criticality. The tiers are cumulative, unless a principle says otherwise. A Tier 3 device also meets the Tier 2 and Tier 1 obligations, and a Tier 2 device meets the Tier 1 obligations. + +* **Tier 1 — Constrained / General-Purpose IoT:** no legally regulated or formally certified core function, and no capability to directly cause physical harm or interrupt an essential service through local or remote actuation (e.g. an environmental sensor, an asset tag, a simple telemetry node). +* **Tier 2 — Regulated-Core IoT:** has a legally regulated or formally certified core function (e.g. a certified measurement, safety, or compliance function), but no direct hazardous actuation capability beyond that core. +* **Tier 3 — Safety-Critical Actuation IoT:** can, through local or remote action, directly cause physical harm, interrupt an essential service, or otherwise affect human safety (e.g. a controllable shutoff, lock, contactor, or valve). + +Pillar 1 is written for Tier 2 and Tier 3 devices. Each of its five principles says which tier it applies to and what a Tier 1 device should do instead. Pillars 2-7 apply to all tiers as a baseline. Some principles imply resources that are out of proportion for a Tier 1 device, such as dedicated secure hardware, dual cryptographic trust anchors or a formally gated language boundary. Their Engineering Implications then include a lighter-weight option. Principles 6, 11, 35, and 37 are the ones most affected by this scaling. + +## Quick-Reference Summary + +A one-line statement of each principle, grouped by pillar, for readers who need the shape of the document before the detail. + +### 1. Core Function Isolation, Functional Safety & Edge Autonomy + +| # | Principle | Tier | Summary | +| :-- | :-- | :-- | :-- | +| 1 | Regulated or Safety-Critical Core Isolation | 2-3 | Partition any legally regulated or safety-critical function from general application logic. | +| 2 | Edge Autonomy and Graceful Degradation | All | The device must operate safely with complete, extended loss of all communications. | +| 3 | Sovereign Timekeeping and Absolute Temporal Integrity | All | Clock monotonicity and auditability must not depend on network-provided time. | +| 4 | Governed, Auditable Remote Control of Safety-Critical State | 3 | Remote commands affecting safety or service must be authenticated, audited, and rate-limited. | +| 5 | Fail-Safe Default Hardware State for Safety-Critical Actuation | 3 | Safety-critical actuators must default to a safe state via hardware alone, independent of firmware. | + +### 2. Device-Centric Security & Cryptography + +| # | Principle | Summary | +| :-- | :-- | :-- | +| 6 | Zero-Trust Hardware & Silicon-Rooted Security | Security is anchored in dedicated hardware silicon; every device is treated as physically exposed. | +| 7 | Standardised, Interoperable Comms & Object Models | Application and data models are decoupled from transport, using open standards. | +| 8 | Zero-Touch, Deterministic Factory Provisioning | Identity and keys are injected via an automated, air-gapped process with zero operator trust. | +| 9 | Verified/Measured Boot Chain | Each boot stage cryptographically verifies the next, from the Root of Trust upward. | +| 10 | Observability with Minimal Attack Surface | Diagnostics give forensic visibility without introducing backdoors or leaking data. | +| 11 | Cryptographic Agility & Field-Rotatable Trust Anchors | Algorithms and trust anchors must be field-updatable without hardware replacement. | +| 12 | Software & Hardware Supply Chain Transparency | A verifiable SBOM/HBOM is maintained for every firmware image and hardware revision. | +| 13 | Privacy by Design and Data Minimisation | Data collection and retention are the minimum necessary for the declared purpose. | + +### 3. Power Budgeting & Environmental Endurance + +| # | Principle | Summary | +| :-- | :-- | :-- | +| 14 | Deterministic Ultra-Low Power by Design | Energy consumption is budgeted to the microampere-second across the operating lifecycle. | +| 15 | Physical and Chemical Environmental Robustness | The enclosure maintains integrity across the device's full temperature, ingress, and chemical envelope. | +| 16 | EMC/RF Coexistence by Design | The radio and any sensitive measurement/control circuitry are designed together so neither degrades the other. | + +### 4. Product Design, Hardware Modularity & Total Lifecycle Cost + +| # | Principle | Summary | +| :-- | :-- | :-- | +| 17 | Total Cost of Ownership (TCO) Over BOM Minimisation | Component choices prioritise lifecycle cost over upfront BOM savings. | +| 18 | Long-Term Component Longevity & Obsolescence De-risking | Designs guard against silicon obsolescence across multi-decade horizons. | +| 19 | Modular Communications Architecture | The WAN comms module is mechanically and logically decoupled from the core device. | +| 20 | Design for Manufacturability & Testability (DFM/DFT) | Structured, automatable production test coverage precedes permanent debug lockout. | + +### 5. Firmware Architecture & Storage Integrity + +| # | Principle | Summary | +| :-- | :-- | :-- | +| 21 | Bulletproof, Failsafe Over-the-Air Updates (OTA) | Updates are dual-bank, atomic, signed, and self-recovering on failure. | +| 22 | Release Pipeline & Staged Fleet Rollout Governance | Releases roll out through canary/ring stages with automated, telemetry-gated halt and rollback. | +| 23 | Versioned, Validated Configuration Management | Device configuration is its own versioned, validated, rollback-capable artefact, separate from firmware. | +| 24 | Feature Flags & Hardware/Firmware Variant Management | One codebase supports multiple hardware variants and feature flags via a tested, documented registry. | +| 25 | Delta and Resumable Firmware Delivery | Updates support binary diffs, chunking, and resumable transfer over constrained links. | +| 26 | Wear-Conscious Storage and Flash Longevity | Flash write amplification and erase cycles are modelled to outlast the warranty. | +| 27 | Atomic Non-Volatile State Persistence | Critical state transitions are power-cut safe. | +| 28 | Crash-Safe Diagnostic Capture | A compact diagnostic record survives any hard fault or unexpected reboot, retrievable without a truck roll. | +| 29 | Static Memory Allocation and Deterministic Concurrency | No dynamic allocation after initialisation; all task execution is bounded. | +| 30 | Deterministic Timing & Worst-Case Execution Time (WCET) Budgeting | Every safety-relevant task and interrupt has a documented, demonstrated timing budget. | +| 31 | Layered Firmware Architecture & Internal API Contracts | Firmware is organised into explicit layers with versioned, enforced internal API contracts. | +| 32 | Decoupled HAL & Host-Based Simulation | Core logic is decoupled from hardware to allow automated host-based testing. | +| 33 | Cross-Version Interoperability Contract | The device interoperates correctly with the back-end across mixed firmware versions during rollout. | +| 34 | Unified Versioning & Compatibility Scheme | Firmware, hardware, config, and protocol versions form one coherent, queryable, policy-governed scheme. | + +### 6. Code Quality, Memory Safety & Verification + +| # | Principle | Summary | +| :-- | :-- | :-- | +| 35 | Memory Safety by Default | Network-facing code enforces spatial and temporal memory safety at compile time. | +| 36 | Zero-Warning Compile-Time Hygiene & Static Analysis | Builds enforce zero warnings under strict flags plus automated SAST. | +| 37 | Conformance to Safety-Critical Coding Standards | Code in the regulated/safety-critical core complies with MISRA C or equivalent. | +| 38 | Defensive Programming and Bounded Input Execution | External interfaces validate inputs defensively and fail into safe states. | +| 39 | Systematic Fault Classification and Graceful Degradation | A defined fault-severity taxonomy drives consistent escalation from retry to safe shutdown. | +| 40 | Continuous Fuzzing and Sanitiser-Driven Testing | Parsers and payload handlers undergo continuous fuzzing and sanitiser testing. | +| 41 | Low Cyclomatic Complexity and Modular Cohesion | Software units are small, focused, and limited in branching complexity. | +| 42 | Least Privilege and Domain Separation in Firmware | Firmware components operate under least privilege with hardware-enforced isolation. | +| 43 | Reproducible, Deterministic Builds | Any released binary is exactly reproducible, bit-for-bit, from source and a pinned toolchain. | + +### 7. Whole-Device Compliance, Human Factors & Sustainability + +| # | Principle | Summary | +| :-- | :-- | :-- | +| 44 | Regulatory Certification & Type-Approval Traceability | Every hardware/firmware combination holds current, traceable certification per market. | +| 45 | Installation, Commissioning & Field Ergonomics | The device and its commissioning workflow are designed for safe, low-error field installation. | +| 46 | End-User and Operator Accessibility and Safety Communication | Human-facing interfaces are accessible and don't rely solely on an app or WAN. | +| 47 | Circularity, Sustainability & End-of-Life Decommissioning | The device supports secure decommissioning and meets circularity regulation. | + +## 1. Core Function Isolation, Functional Safety & Edge Autonomy + +> **Applicability:** Principles 1 and 4 in this pillar apply to Tier 2 and Tier 3 devices as defined in Applicability Tiers above. Principles 2 and 3 apply to all tiers, with the specific sub-requirements that only make sense for a certified or actuating core marked accordingly. Principle 5 applies to Tier 3 only. A Tier 1 device (e.g. a simple sensor with no certified function and no actuation capability) should still isolate whichever function it treats as highest-integrity — commonly its own calibration or reporting accuracy — behind an MPU region or equivalent software boundary, even without formal certification; it does not need the certification-grade signature and versioning separation Principle 1 requires of Tier 2 and Tier 3 devices. + +### Principle 1: Regulated or Safety-Critical Core Isolation + +* **Statement:** Any legally regulated or safety-critical core function must be physically and logically partitioned from non-regulated features (telemetry, display, HAN/WAN communications). +* **Applies to:** Tier 2 and Tier 3. A Tier 1 device should apply the same isolation pattern to whichever function it treats as highest-integrity, without the certification-grade signature and versioning separation described below. +* **Rationale:** Certification of a regulated or safety-critical function is costly and time-consuming, and typically governed by a domain-specific standard (e.g. a functional safety standard, a measurement or instrumentation standard, a medical device standard). Isolating the certified function behind strict hardware or memory domains ensures that application-layer updates, radio firmware changes, and network stack refactors can proceed without voiding certification. +* **Engineering Implications:** + * Separate the regulated-function MCU/ASIC from the application processor, or enforce strict MPU/TrustZone hardware boundaries on single-chip solutions. + * Regulated-function data must flow outward via deterministic, read-only internal interfaces (e.g. unidirectional SPI/UART or isolated shared-memory mailboxes). + * Firmware images for the regulated function must have distinct cryptographic signatures and separate versioning registers from application firmware. +* **Verification:** The isolation boundary is documented in the hardware architecture and confirmed by a certification body assessment where applicable; application-side firmware updates are demonstrated to leave the regulated-function signature and version register unchanged. + +### Principle 2: Edge Autonomy and Graceful Degradation + +* **Statement:** The device must operate autonomously as a safe measurement and/or control device during complete, extended loss of all communications networks. +* **Applies to:** All tiers for local data buffering and autonomous operation; the local protective-action requirement (third Engineering Implication below) applies specifically to Tier 3. +* **Rationale:** Wide-area and local wireless networks can suffer outages lasting days or weeks due to infrastructure maintenance, physical shadowing, or back-end system outages. Core function, whether data capture, state tracking, or safety-critical control, cannot depend on real-time network interaction. +* **Engineering Implications:** + * Maintain local, non-volatile circular buffers sized to the device's expected maximum outage duration, so no data the device is responsible for capturing is lost. + * Implement local, autonomous safety routines for any device with a physical actuator: a hazard condition (over-threshold reading, mechanical fault, environmental anomaly) must trigger a local protective action within a defined, tested time budget, without back-end intervention. + * Any local protective action (shutoff, load-shedding, lock, alarm) must trigger deterministically on raw sensor threshold violations rather than on a processed or network-dependent signal. +* **Verification:** Network-outage soak testing, run for the device's specified maximum buffer duration, confirms data integrity and safety-routine response times are unaffected; local protective-action latency is measured and logged against its target on every hardware revision. + +### Principle 3: Sovereign Timekeeping and Absolute Temporal Integrity + +* **Statement:** The device's internal clock must maintain monotonicity, auditability, and deterministic drift correction independently of network-provided time. +* **Applies to:** All tiers wherever the device produces timestamped data; the signed-override and audit-log requirements matter most for Tier 2 and Tier 3, where timestamps back a regulated or safety record. +* **Rationale:** Any timestamped regulated, financial, or audit record relies on a defensible timestamp. Network Time Protocol (NTP) or cellular network time can be spoofed or delayed. Abrupt backward clock steps can corrupt a record or trigger a dispute investigation. +* **Engineering Implications:** + * Equip the design with a temperature-compensated real-time clock (TCXO) with autonomous drift correction across the device's full operating temperature envelope. + * Disallow hard step-changes to system time during normal operation. Back-end time updates must be slew-adjusted gradually (e.g. max 1-2 seconds per hour) unless an authenticated, cryptographically signed administrative override event is executed. + * Every time adjustment must write an immutable, tamper-evident audit log recording previous timestamp, updated timestamp, and the signing entity identity. +* **Verification:** Drift is measured against a reference clock across the full temperature range during EVT/DVT; time-adjustment audit logs are reviewed to confirm no unsigned step-change has occurred in field trial units. + +### Principle 4: Governed, Auditable Remote Control of Safety-Critical State + +* **Statement:** Remote commands capable of altering safety-critical or service-affecting state must be strongly authenticated, individually audited, and rate-limited at both the device and the controlling back-end, with local safeguards that prevent mass or erroneous action. +* **Applies to:** Tier 3, and any Tier 1 or Tier 2 device that happens to accept a remote command capable of altering safety-critical or service-affecting state. Not applicable to a purely passive sensing device with no actuation or control interface. +* **Rationale:** Any device capable of remote actuation, service interruption, or physical lock/unlock is a high-value attack target by virtue of that capability. A compromised back-end credential, or a scripting error that fans out a command to a device population, is a direct life-safety or operational-continuity risk, particularly where the device controls physical access, a hazardous process, or continuity of an essential service. This is a distinct failure mode from the local safety autonomy covered in Principle 2. +* **Engineering Implications:** + * Require a distinct, non-reusable cryptographic authorisation per command; commands must not be replayable. + * Implement device-side rate limiting and anomaly detection (e.g. refuse a command arriving outside an expected pattern without secondary confirmation). + * Maintain an immutable local log of all safety-critical remote commands with origin, timestamp, and authorising identity, retrievable during dispute investigation. + * Define and document safeguards on re-enabling or reversing an action (e.g. a mandatory pre-check or staged re-activation) to prevent an unsafe state on resumption. +* **Verification:** Penetration testing includes simulated back-end credential compromise scenarios targeting mass remote action; audit logs are confirmed retrievable and complete after an action/reversal cycle in field trials. + +### Principle 5: Fail-Safe Default Hardware State for Safety-Critical Actuation + +* **Statement:** Any safety-critical actuator (contactor, valve, lock, or equivalent) must default to its defined safe state through hardware and electrical design alone on power loss, fault, or firmware failure, independent of firmware ever executing. +* **Applies to:** Tier 3. +* **Rationale:** Firmware can hang, crash, or fail to boot. Principle 27 (Atomic Non-Volatile State Persistence) protects the integrity of stored state once firmware is running; this principle protects the physical world when firmware is not running at all, for example during a brownout, a corrupted boot image, or a watchdog loop that never completes initialisation. +* **Engineering Implications:** + * Select actuator drive circuits and mechanical designs (e.g. normally-closed valves, fail-open or fail-closed relay topologies) so that the de-energised or unpowered state is always the safe state for the application. + * Ensure the safe-state behaviour is achieved by the drive circuit's electrical characteristics or a discrete hardware watchdog cutting power to the actuator driver, not by a firmware instruction that assumes the processor is still running correctly. + * Document the safe state and the hardware mechanism that guarantees it as part of the safety case, independent of the software safety case. +* **Verification:** The safe-state behaviour is demonstrated with the microcontroller held in reset, with firmware deliberately corrupted, and with primary power removed, confirming the actuator reaches its safe state in each case without firmware involvement. + +## 2. Device-Centric Security & Cryptography + +### Principle 6: Zero-Trust Hardware & Silicon-Rooted Security + +* **Statement:** Security must be anchored in dedicated hardware silicon; every device is treated as exposed to physical and network-level adversaries. +* **Rationale:** Field-deployed devices are installed in public, semi-public, or unattended locations without physical perimeter defence. Attackers have indefinite access for side-channel analysis, voltage glitching, bus sniffing, and local port fuzzing. +* **Engineering Implications:** + * Enforce a dedicated Secure Element (SE) or hardware Root of Trust (RoT) capable of secure key generation, zero-exposure storage, and hardware acceleration for AES-GCM and elliptic curve cryptography (ECDSA, Ed25519). + * Debug interfaces (JTAG/SWD) must be physically or cryptographically blown/fused at the end of the manufacturing line, once production test coverage (Principle 20) is complete. + * Implement active and passive tamper loops (switch circuits, mesh traces under plastic enclosures, light sensors) that record events and trigger ephemeral key zeroisation upon enclosure breach. + * **Tier scaling:** Tier 2 and Tier 3 devices require a dedicated SE or hardware RoT as described above. A Tier 1 device with a lower risk exposure may substitute a lower-cost secure boot / authenticated-boot IC for a full Secure Element, provided debug lockout and key-exposure protections are still met. +* **Verification:** Independent side-channel and fault-injection lab testing is commissioned pre-production; debug fuse state is confirmed blown on a sample from every production batch. + +### Principle 7: Standardised, Interoperable Comms & Object Models + +* **Statement:** The application layer and telemetry data models must be strictly decoupled from physical transport mechanisms using open industry standards appropriate to the device's domain. +* **Rationale:** Proprietary payload structures lock the operator to specific suppliers and complicate back-end ingestion. Wide-area transport networks evolve independently of the device's application lifecycle. +* **Engineering Implications:** + * Implement a standardised, versioned object or data model appropriate to the domain (e.g. LwM2M, OPC-UA, or an equivalent open specification) rather than a proprietary payload format. + * Isolate application message brokers from the physical and transport protocol stacks (NB-IoT, LTE-M, sub-GHz mesh or proprietary radio, Wi-SUN, Thread, LoRaWAN). + * Payload encodings must be compact and deterministic (e.g. CBOR, Protocol Buffers) to minimise radio airtime. +* **Verification:** Conformance test suites for the chosen companion specification are run against every release; a transport-swap integration test (e.g. NB-IoT to LTE-M) confirms zero change required at the application/object model layer. + +### Principle 8: Zero-Touch, Deterministic Factory Provisioning + +* **Statement:** Device identity, certificates, and cryptographic roots must be injected through an automated, air-gapped process requiring zero human operator trust. +* **Rationale:** Assembly lines are untrusted environments. Storing static global keys or handling private symmetric/asymmetric keys in plain text on production servers introduces catastrophic single-point failure across an entire device fleet. +* **Engineering Implications:** + * Leverage production Hardware Security Modules (HSMs) directly integrated with line programmers. + * Implement IEEE 802.1AR Secure Device Identifier (DevID) standards. Each device generates its operational keypairs on-chip within the Secure Element, exposing only public certificates for factory signing. + * Post-provisioning verification must cryptographically prove identity and lock provisioning registers before the device leaves the factory cell. +* **Verification:** Provisioning process is audited to confirm private key material is never observable outside the SE/HSM boundary; a sample of provisioned units is cryptographically challenged before batch release. + +### Principle 9: Verified/Measured Boot Chain + +* **Statement:** Each stage of the boot process must cryptographically verify the integrity and authenticity of the next stage before executing it, forming an unbroken chain of trust from the hardware Root of Trust to the running application. +* **Rationale:** Principle 6's Root of Trust and Principle 21's dual-bank OTA integrity checking both assume the bootloader itself can be trusted; without a verified boot chain, a compromised or corrupted early boot stage can silently bypass every later integrity check, including the OTA signature verification it is supposed to gate. +* **Engineering Implications:** + * Anchor the first verification step in immutable, factory-programmed boot ROM or fused boot configuration that cannot itself be updated in the field. + * Each stage measures (hashes) and verifies the signature of the next stage before transferring control; a failed verification halts the boot or triggers Principle 21's rollback path rather than executing unverified code. + * Where supported, extend measurement into a hardware-backed log (e.g. a measured-boot register) so a compromised chain is detectable even if it technically completes. +* **Verification:** Boot-chain verification is tested by deliberately corrupting each stage's signature or content in turn on a bench unit, confirming the chain halts or rolls back rather than proceeding; the immutability of the first-stage boot ROM is confirmed during hardware bring-up. + +### Principle 10: Observability with Minimal Attack Surface + +* **Statement:** Health metrics, telemetry, and diagnostic logs must offer comprehensive forensic visibility without introducing backdoors or leaking operational data. +* **Rationale:** Remote triage is necessary to identify failing batteries, drifting sensors, or degraded connectivity. However, unauthenticated debug channels or verbose log outputs provide attack vectors for bad actors. +* **Engineering Implications:** + * Encapsulate device health indicators (RF signal quality, internal impedance, battery remaining capacity, local port access counts) into compact, encrypted binary telemetry payloads. + * A local maintenance or service port must require mutual cryptographic authentication before granting access to internal registers or diagnostic dumps. + * Diagnostic interfaces must not output cleartext passwords, cryptographic material, or raw memory pointers. +* **Verification:** A diagnostic-output review is performed each release to confirm no cleartext secrets appear in logs; local port authentication is penetration tested annually. + +### Principle 11: Cryptographic Agility & Field-Rotatable Trust Anchors + +* **Statement:** The device's cryptographic algorithms and trust anchors must be field-updatable without requiring hardware replacement, and the design must anticipate at least one full algorithm transition within the device's operating life. +* **Rationale:** A long field life will outlast the assumed strength of algorithms and key sizes considered adequate at deployment; ECDSA and AES parameters selected today are already the subject of post-quantum migration planning across multiple standards bodies. A Root of Trust that cannot rotate its trusted keys or upgrade its algorithms (Principle 6) becomes a fleet-wide liability rather than a fleet-wide asset once that happens. +* **Engineering Implications:** + * Design the Secure Element's key storage to support multiple concurrent trust anchors and a signed, auditable rotation procedure. + * Reserve firmware update capacity (Principle 21) sufficient to deliver a new cryptographic library or hybrid classical/post-quantum scheme without a hardware refresh. + * Document a crypto-agility roadmap identifying which algorithms are hardware-accelerated versus software-upgradable, and the fallback behaviour if a legacy algorithm is deprecated mid-fleet-life. + * **Tier scaling:** Tier 2 and Tier 3 devices should support multiple concurrent trust anchors as described above. A Tier 1 device may implement a single rotatable trust anchor instead, provided the rotation procedure itself is still supported and field-tested. +* **Verification:** A simulated key-rotation and algorithm-upgrade exercise is run on pre-production units to confirm the process completes without bricking or core-function disruption; the crypto-agility roadmap is reviewed annually against current cryptographic guidance (e.g. NIST, NCSC). + +### Principle 12: Software & Hardware Supply Chain Transparency + +* **Statement:** A verifiable Software Bill of Materials (SBOM) and Hardware Bill of Materials (HBOM) must be maintained for every released firmware image and hardware revision, covering all third-party and vendor-supplied components. +* **Rationale:** Emerging regulation (e.g. the EU Cyber Resilience Act and equivalent regional regimes) is moving toward mandatory SBOM disclosure and vulnerability reporting for connected devices generally. A fleet running third-party stacks (radio module firmware, mesh/RF libraries, RTOS) for a decade or more needs a standing mechanism to know what is running and to respond when a component is found vulnerable, independent of the first-party code quality controls in Pillar 6. +* **Engineering Implications:** + * Generate a machine-readable SBOM (e.g. CycloneDX or SPDX) as part of every build, covering open-source and vendor-licensed components with version and licence data. + * Subscribe released third-party components to CVE monitoring; define a maximum time-to-patch or documented mitigation for critical vulnerabilities in fielded firmware. + * Track hardware component provenance for security-relevant parts (Secure Element, RF module) to detect counterfeit or substituted components during manufacturing. +* **Verification:** SBOM generation is a required, automated CI gate; a quarterly audit confirms all released firmware versions have an associated SBOM and no unmitigated critical CVE beyond the documented time-to-patch threshold. + +### Principle 13: Privacy by Design and Data Minimisation + +* **Statement:** Data collection, retention, and transmission granularity must be the minimum necessary for the declared purpose, with any personally identifying data protected throughout its lifecycle. +* **Rationale:** Fine-grained sensor data can reveal personal patterns, location, or behaviour, and be classed as personal data under GDPR and equivalent regimes wherever it relates to an identifiable individual. This is a design constraint on the data models and telemetry paths covered in Principles 7 and 10, not a downstream compliance task to be handled after the architecture is fixed. +* **Engineering Implications:** + * Define data retention and granularity tiers (e.g. raw interval data retained locally, coarser aggregates transmitted by default) with end-user-, operator-, or regulator-configurable granularity where required. + * Encrypt personal or sensitive data in transit and at rest, and apply access controls at the back-end consistent with the device-side security model in Pillar 2. + * Document a data flow map from sensor to back-end identifying every point personal or sensitive data is stored, transmitted, or could be inferred. +* **Verification:** A data protection impact assessment is completed and reviewed against the documented data flow map; default transmitted granularity is confirmed to match the minimum agreed with the relevant regulatory scheme before release. + +## 3. Power Budgeting & Environmental Endurance + +### Principle 14: Deterministic Ultra-Low Power by Design + +* **Statement:** Energy consumption is a first-order architectural constraint for any battery- or energy-harvesting-powered device, with every microampere-second rigorously budgeted across the operating lifecycle. +* **Rationale:** Mains- or vehicle-powered devices draw continuous operating power, but many field IoT devices are restricted to a non-rechargeable or hard-to-service power source that must reliably last a decade or more under variable ambient conditions. +* **Engineering Implications:** + * The firmware default state must be deep sleep. Waking states must run in rapid bursts (e.g. wake, sample sensor via DMA, evaluate thresholds, sleep). + * Network interactions must prioritise batched uplinks over continuous listening (e.g. 3GPP PSM/eDRX modes for NB-IoT). + * Characterise power consumption across the device's full temperature operating range to account for passivation, self-discharge, and internal cell resistance increases. +* **Verification:** A full power budget model is maintained and reconciled against measured current draw at each firmware release; projected battery life is recalculated whenever a new feature changes the wake/sleep duty cycle. + +### Principle 15: Physical and Chemical Environmental Robustness + +* **Statement:** The mechanical, thermal, and electronic enclosure design must maintain integrity across the device's specified temperature, ingress, and chemical exposure extremes. +* **Rationale:** Field-deployed devices are placed in exposed, uncontrolled locations, and some are deployed in hazardous or explosive atmospheres requiring intrinsic safety compliance. +* **Engineering Implications:** + * Meet an ingress protection rating (e.g. IP67/IP68) appropriate to the deployment environment; devices deployed in explosive atmospheres must comply with ATEX/IECEx Zone 0/1 intrinsic safety requirements. + * Apply industrial conformal coating (or potting for regulated measurement circuitry) to eliminate dendritic growth, tin whiskers, and corrosion from airborne contaminants. + * Implement transient suppression (MOVs, TVS diodes, gas discharge tubes) appropriate to the device's power source, capable of surviving surges and, where mains-connected, lightning-induced transients. +* **Verification:** IP and, where applicable, ATEX/IECEx ratings are confirmed by accredited third-party test laboratories, with certificates retained against each hardware revision; surge immunity is retested whenever the front-end component selection changes. + +### Principle 16: EMC/RF Coexistence by Design + +* **Statement:** The device's RF transmitter/receiver and any sensitive measurement or control circuitry must be designed together so that neither degrades the other's performance, across the device's full operating envelope. +* **Rationale:** Combining a radio and a sensitive measurement front end on one compact board is now the default architecture for field IoT devices, and self-generated RF interference is a common, hard-to-diagnose source of measurement error or comms unreliability that is invisible in a bench test of either subsystem in isolation. +* **Engineering Implications:** + * Define RF and analogue/measurement domains on the PCB with dedicated grounding, shielding, and component placement (e.g. an RF keep-out zone around sensitive analogue front ends). + * Characterise the measurement or control function's behaviour during active transmission at full radio power, not only at idle, as part of design verification. + * Route high-speed digital and RF traces to minimise coupling into any wired local interface (e.g. a diagnostic or metering bus) that shares the enclosure. +* **Verification:** EMC pre-compliance testing includes a specific test with the radio actively transmitting at full power while the measurement or control function is exercised; any measurable degradation beyond the specified accuracy budget is treated as a design defect, not a tolerance to be documented around. + +## 4. Product Design, Hardware Modularity & Total Lifecycle Cost + +### Principle 17: Total Cost of Ownership (TCO) Over BOM Minimisation + +* **Statement:** Component selection and architectural trade-offs must prioritise total lifecycle operating costs over initial board-level Bill of Materials savings. +* **Rationale:** Saving a small amount on marginal capacitor grades, inferior battery chemistries, or low-end flash chips results in premature field failures. A single technician truck roll to replace a faulty device costs significantly more than the initial manufacturing cost saving. +* **Engineering Implications:** + * Model all decisions using a TCO framework appropriate to the device's expected field life, factoring in failure rates, warranty risk, and dispatch costs. + * Select components rated for extended industrial temperature and operational life (e.g. hybrid layer capacitors for pulse buffering, solid-state switches, and high-endurance memory). + * Establish accelerated life testing (ALT) and highly accelerated life testing (HALT) pipelines during the proto/EVT phase. +* **Verification:** A TCO model accompanies every significant BOM change proposal and is signed off before the change is accepted; ALT/HALT results are reviewed against the projected field failure rate before design freeze. + +### Principle 18: Long-Term Component Longevity & Obsolescence De-risking + +* **Statement:** Hardware and firmware designs must actively guard against silicon obsolescence across multi-year to multi-decade production and maintenance horizons. +* **Rationale:** Consumer electronic lifecycles rarely exceed 3 years. Infrastructure and industrial IoT rollouts often span several years of deployment followed by a decade or more of field support. Silicon discontinuation mid-rollout causes expensive redesigns and re-certifications. +* **Engineering Implications:** + * Mandate minimum availability commitments from core silicon suppliers (MCU, AFE, SE, Transceiver) appropriate to the device's expected support horizon. + * Multi-source passive and discrete components via dual-footprint PCB designs where possible. + * Maintain clean, abstract Hardware Abstraction Layers (HAL) to allow pin-compatible microcontroller variants or alternate memory chips to be integrated without rewriting upper-layer application logic. +* **Verification:** Supplier longevity commitments are held on file and reviewed annually; a HAL portability exercise (porting to a pin-compatible alternate part in a lab build) is run at least once per hardware generation to confirm the abstraction holds. + +### Principle 19: Modular Communications Architecture + +* **Statement:** The wide-area communications module must be mechanically and logically decoupled from the device's core function and base chassis. +* **Rationale:** WAN technologies turn over rapidly (e.g. 2G/3G sunsets, transition from 4G LTE-M/NB-IoT to satellite NTN or 5G RedCap) compared to a device's long-life core mechanics. +* **Engineering Implications:** + * Isolate the communications engine as a swappable communications hub, plug-in daughterboard, or field-replaceable module. + * Standardise internal physical and electrical interconnects (e.g. isolated UART/SPI with well-defined command/data protocols). + * Ensure that swapping or power-cycling a comms module has zero impact on continuous core-function logging. + * The comms module's own firmware and OTA image must carry no elevated read/write access to core-function registers, flash write controllers, or Secure Element keys, consistent with the least-privilege domain separation in Principle 42. +* **Verification:** A physical comms-module swap is demonstrated on a live unit with core-function logging continuing uninterrupted; the comms module's privilege boundary is confirmed by static review against Principle 42 at each release. + +### Principle 20: Design for Manufacturability & Testability (DFM/DFT) + +* **Statement:** The hardware design must include structured, automatable test coverage sufficient to verify correct assembly and functional operation at production volume, before any production-line debug interface is permanently disabled. +* **Rationale:** Principle 6's requirement to fuse off debug interfaces at the end of the line only works if a comprehensive test regime has already run while those interfaces were still live; without it, manufacturing defects and early-life failures are pushed into the field, where they are far more expensive to find and fix (Principle 17). +* **Engineering Implications:** + * Include test points, boundary-scan (JTAG) chains, and built-in self-test (BIST) routines that exercise every populated component and interconnect during production test, before debug fusing. + * Design a structured functional test sequence (power-on, sensor stimulus/response, RF loopback, actuator drive verification) that can run automatically on a production line within a defined cycle time. + * Capture production test results per serial number and retain them for field failure-mode correlation. +* **Verification:** Production test coverage is measured (e.g. percentage of nets and components exercised by boundary scan and functional test) and reviewed against a target before design release; a sample of "test escapes" (units that passed production test but failed early in the field) is root-caused each quarter. + +## 5. Firmware Architecture & Storage Integrity + +### Principle 21: Bulletproof, Failsafe Over-the-Air Updates (OTA) + +* **Statement:** The remote update subsystem must be dual-bank, atomic, cryptographically verified, and guaranteed to be self-recovering under all failure conditions. +* **Rationale:** A bricked device during a fleet-wide OTA campaign requires manual technician dispatches that can collapse the economics of the deployment. +* **Engineering Implications:** + * Implement an immutable, primary-stage bootloader residing in protected flash sectors. + * Adopt an A/B dual-bank memory partitioning scheme. New firmware images write strictly to the inactive bank and undergo cryptographic signature checks (ECDSA) prior to activation. + * Configure an independent hardware watchdog timer that runs on boot. If the newly deployed firmware fails to boot, validate configuration, or establish network connectivity, the system automatically rolls back to the known-good image in the secondary bank. +* **Verification:** Power-loss-during-update fault injection is run on every release candidate; automatic rollback is confirmed to succeed under simulated boot failure, configuration failure, and network failure conditions independently. + +### Principle 22: Release Pipeline & Staged Fleet Rollout Governance + +* **Statement:** Firmware and configuration releases must be deployed to the fleet through a staged rollout (e.g. canary, then a representative ring, then general availability) with automated, telemetry-based halt and rollback criteria, rather than as a single fleet-wide push. +* **Rationale:** Principle 21's dual-bank update mechanism and Principle 23's configuration rollback both protect an individual device from a bad update; neither stops a subtly defective release from reaching the whole fleet before enough devices have reported a problem for anyone to notice. A staged rollout limits the blast radius and produces real field telemetry before wider exposure. +* **Engineering Implications:** + * Define rollout rings (e.g. internal/lab devices, a small statistically representative canary population, then percentage-based waves) with automated progression gated on defined health metrics — crash rate, connectivity, error rate — rather than a fixed calendar schedule alone. + * Implement an automated fleet-level halt-and-rollback trigger, distinct from the per-device rollback in Principle 21, that pauses or reverses a rollout if canary or ring telemetry crosses a defined threshold. + * Maintain a documented release approval and rollback decision process, including who is authorised to pause, resume, or abort a rollout in progress. +* **Verification:** A rollout rehearsal is run against a deliberately defective canary build to confirm the halt criteria actually stop progression before it reaches general availability; the ring structure and gating thresholds are reviewed at least annually against actual fleet size and growth. + +### Principle 23: Versioned, Validated Configuration Management + +* **Statement:** Device configuration (operational parameters, feature flags, credentials, calibration data) must be managed as its own versioned, validated, and rollback-capable artefact, independent of the firmware binary. +* **Rationale:** A firmware image can be correct while a bad configuration push still causes a fleet-wide incident; Principle 21's dual-bank OTA protects the firmware image, but a configuration change delivered outside that path (a live parameter write, a remote calibration update) has no equivalent protection unless this is addressed separately. +* **Engineering Implications:** + * Apply the same atomic, verified-before-activation pattern used for firmware (Principle 21) to configuration changes: stage the new configuration, validate it against a schema and sane-value bounds, then activate atomically. + * Version every configuration change with a rollback path to the last known-good configuration, and log the change with its origin and authorising identity, consistent with Principle 4's command auditing. + * Reject configuration values outside a defined safe envelope at the device, rather than trusting the back-end to have validated them. +* **Verification:** A malformed or out-of-envelope configuration push is tested against a bench unit to confirm it is rejected rather than applied; configuration rollback is exercised as part of the same fault-injection regime used for firmware rollback in Principle 21. + +### Principle 24: Feature Flags & Hardware/Firmware Variant Management + +* **Statement:** A single firmware codebase must support multiple hardware variants and feature-gated capabilities through documented build-time and runtime configuration, with an explicit, tested registry of which flags and variants are valid on which hardware revision, rather than through divergent forked codebases or untracked conditional compilation. +* **Rationale:** Regional radio variants, optional sensors, and incrementally rolled-out capabilities create constant pressure to fork the codebase or scatter untracked conditional logic through it; either erodes the layering discipline in Principle 31 and multiplies the test matrix that Principles 32 and 40 depend on being tractable. +* **Engineering Implications:** + * Select hardware variants through build-time configuration gated by documented hardware capability flags, not ad hoc preprocessor conditionals scattered through application code. + * Maintain a variant/feature-flag compatibility matrix mapping each hardware revision to its supported flags and the specific combinations that have been validated; an unvalidated combination must not be deployable to the field. + * Assign every feature flag an owner and a removal plan; treat a flag left in place after its feature has fully rolled out as technical debt to be retired, not a permanent branch point. +* **Verification:** CI builds and tests the full support matrix of hardware variants and flag combinations, or a documented representative subset agreed with the architecture team; a periodic flag audit identifies and schedules removal of stale flags. + +### Principle 25: Delta and Resumable Firmware Delivery + +* **Statement:** Remote update delivery mechanisms must support binary differential updates, chunked delivery, and resumable transfers over constrained links. +* **Rationale:** Delivering 1-2 MB monolithic binaries over low-speed LPWAN (NB-IoT) or sub-GHz mesh networks consumes excessive radio airtime, drains batteries, and strains network bandwidth. +* **Engineering Implications:** + * Bootloader and staging engines must support binary diff application, reconstructing target images on-chip using the existing image plus verified patch fragments. + * Transport protocols must break payloads into small, idempotently addressable blocks (e.g. CoAP Block-Wise Transfer). + * Interruptions caused by fading coverage or power management must resume transfers from the last acknowledged block without restarting the download sequence. +* **Verification:** A simulated intermittent-coverage transfer (repeated mid-transfer interruption) is confirmed to resume and complete without restarting; delta patch size versus full image size is tracked per release to confirm airtime savings. + +### Principle 26: Wear-Conscious Storage and Flash Longevity + +* **Statement:** Non-volatile memory write amplification and sector erase cycles must be formally modelled to ensure storage media outlasts the device warranty. +* **Rationale:** Logging interval data, event history, and configuration updates over a decade or more can easily exhaust NOR/NAND flash sector endurance limits (typically 10,000 to 100,000 erase cycles), resulting in unrecoverable memory degradation. +* **Engineering Implications:** + * Integrate wear-levelling, fail-safe filesystems (e.g. littlefs) or structured circular log rings with pointer rotation. + * Buffer high-frequency transient telemetry in ultra-low-power internal SRAM/FRAM, committing batch updates to linear flash only at scheduled intervals or upon power-loss detection. + * Formally calculate and document flash wear-out headroom: projected write cycles across the device's field-life target must not exceed 20% of rated flash endurance under worst-case operation. +* **Verification:** The wear-out headroom calculation is reviewed and re-run whenever logging frequency or retention policy changes; accelerated write-cycle testing on production flash parts confirms the modelled endurance margin. + +### Principle 27: Atomic Non-Volatile State Persistence + +* **Statement:** Critical operational state transitions, including usage or event counters, actuator or relay positions, and tamper records, must be power-cut safe. +* **Rationale:** Mains-powered devices can suffer blackouts mid-write; battery-powered devices can experience sudden voltage collapse if a high-current radio pulse coincides with an aged battery. Write operations must never leave registers in a corrupted or ambiguous state. +* **Engineering Implications:** + * Prohibit in-place updates of critical state records. + * Use copy-on-write, double-buffering, or journaled transaction logs with CRC32/SHA verification. + * Include brownout-detect (BOD) interrupts paired with onboard energy reservoir capacitors to gracefully complete in-flight transactions or abort cleanly before core logic shuts down. +* **Verification:** Power-cut injection testing at randomised points in the write cycle confirms no corrupted or ambiguous state is ever observed across a statistically significant number of trials. + +### Principle 28: Crash-Safe Diagnostic Capture + +* **Statement:** The device must persist a compact diagnostic record (reset reason, fault address, stack context, and relevant system state) across any hard fault, watchdog reset, or unexpected reboot, retrievable without a truck roll. +* **Rationale:** Field failures that only manifest as an unexplained reboot are otherwise undiagnosable without physically retrieving the device; a small, always-on diagnostic capture path turns an opaque field failure into an actionable bug report, without conflicting with Principle 10's restriction on leaking sensitive diagnostic detail. +* **Engineering Implications:** + * Reserve a small region of non-volatile memory (outside the wear-levelled main log, per Principle 26) specifically for the most recent crash record, written directly from the fault or watchdog handler with minimal dependencies. + * Capture reset reason, programme counter/fault address, a bounded stack trace, and relevant task/scheduler state at the point of failure. + * Surface the crash record over the device's normal telemetry or diagnostic channel on the next successful boot, consistent with the sanitisation requirements of Principle 10. +* **Verification:** Deliberate fault injection (forced hard fault, forced watchdog timeout) on a bench unit confirms a usable diagnostic record survives the reset and is retrievable on next boot. + +### Principle 29: Static Memory Allocation and Deterministic Concurrency + +* **Statement:** Dynamic runtime memory allocation is prohibited following initialisation, and all task execution must be strictly bounded. +* **Rationale:** Dynamic heap allocation (`malloc`/`free`) leads to heap fragmentation, memory leaks, and unpredictable runtime failures in embedded software intended to run continuously for years without a reboot. +* **Engineering Implications:** + * Allocate all task stacks, operational buffers, network queues, and driver contexts statically at compile time or during boot setup. + * Deploy a deterministic Real-Time Operating System (RTOS) or static event-driven scheduler. + * Assign hardware or software watchdogs to individual execution tasks to prevent priority inversion, deadlocks, or task starvation. +* **Verification:** Static analysis confirms no post-initialisation heap allocation calls are present in shipped firmware; long-duration soak testing (multi-week) monitors for stack high-water-mark drift or watchdog resets. + +### Principle 30: Deterministic Timing & Worst-Case Execution Time (WCET) Budgeting + +* **Statement:** Every task, interrupt handler, and safety-relevant control loop must have a documented worst-case execution time and a demonstrated timing budget that leaves adequate margin under the device's real-time deadlines. +* **Rationale:** Principle 2's local protective-action latency and Principle 29's deterministic scheduling both depend on knowing, not assuming, how long each piece of code takes to run in its worst case; an RTOS with static allocation is necessary but not sufficient for real-time correctness without an explicit timing analysis behind it. +* **Engineering Implications:** + * Establish WCET for interrupt handlers and any task on the path to a safety-critical response (Principle 2), using static analysis, measurement under worst-case cache/memory conditions, or both. + * Design interrupt and task priorities explicitly to prevent priority inversion (e.g. priority inheritance or priority ceiling protocols) rather than relying on empirical tuning. + * Maintain a timing budget document showing each safety-relevant deadline and the margin achieved, reviewed whenever a new feature adds code to a timing-critical path. +* **Verification:** WCET figures are re-measured or re-analysed whenever timing-critical code changes; the timing budget document is reviewed as a release gate for any change touching a safety-relevant control path. + +### Principle 31: Layered Firmware Architecture & Internal API Contracts + +* **Statement:** Firmware must be organised into explicit layers (driver, middleware/platform, application) with versioned, documented internal API contracts between them, and no layer may depend on the internal implementation detail of another. +* **Rationale:** The HAL boundary in Principle 32 isolates hardware specifically; this principle addresses the layering above that, which without an explicit contract tends to erode over a multi-year codebase into circular dependencies and "just this once" shortcuts that make later refactoring or a language migration (Principle 35/37) far more expensive than it needs to be. +* **Engineering Implications:** + * Define and document the API contract at each layer boundary (driver-to-middleware, middleware-to-application) including versioning for backward compatibility as the codebase evolves. + * Enforce the layering with build-level dependency checks (e.g. disallowing application code from including driver-internal headers) rather than relying on convention alone. + * Review and update the layer diagram as part of any architecturally significant change, keeping it a living document rather than a one-time diagram. +* **Verification:** An automated dependency-direction check runs in CI and fails the build on a layering violation; the layer/API documentation is reviewed for currency at each major release. + +### Principle 32: Decoupled HAL & Host-Based Simulation + +* **Statement:** Core business logic, domain state machines, and protocol engines must be decoupled from bare-metal hardware peripherals to allow automated test execution on host infrastructure. +* **Rationale:** Relying exclusively on physical hardware for integration and verification slows release cadences, hinders CI/CD automation, and delays functional testing. +* **Engineering Implications:** + * Isolate hardware-dependent calls behind clean, abstract Hardware Abstraction Layer (HAL) interfaces. + * Implement dual-target toolchains: cross-compilation for target microcontrollers (ARM Cortex-M, RISC-V) and native compilation (x86_64/ARM64) for automated test suites. + * Execute 100% of domain logic, protocol state machines, and cryptographic flows in host-based unit and integration test suites on every pull request. +* **Verification:** CI pipeline reports show host-based test execution on every pull request with no hardware dependency; coverage of domain logic and protocol state machines is tracked and reported per release. + +### Principle 33: Cross-Version Interoperability Contract + +* **Statement:** The device's protocol and data-model implementation must interoperate correctly with the back-end across at least N-1 (and ideally N-2) firmware versions, so a fleet running mixed versions mid-rollout continues to function correctly. +* **Rationale:** An OTA campaign (Principle 21) is never instantaneous across a large fleet; if a protocol or object-model change (Principle 7) is not backward-compatible, the rollout window itself becomes an outage window, and a partially-rolled-back campaign can leave the fleet in an unsupported mixed state indefinitely. This version skew is exposed to the back-end as part of the unified versioning scheme in Principle 34. +* **Engineering Implications:** + * Version the object/data model (Principle 7) explicitly and design changes to be additive where possible, with a defined deprecation window for breaking changes. + * Test new firmware against the currently-deployed back-end version and, where feasible, against N-1 back-end versions, not only against the latest. + * Define and document the supported version skew (e.g. "back-end supports device firmware N and N-1") as an explicit, tested contract rather than an assumption. +* **Verification:** A mixed-version interoperability test (current back-end against the current and immediately prior firmware version) is run as part of every release candidate's test suite before fleet-wide rollout begins. + +### Principle 34: Unified Versioning & Compatibility Scheme + +* **Statement:** The device must expose a single, coherent set of version identifiers, covering firmware, hardware revision, configuration schema, and protocol/object-model version, governed by one documented compatibility policy (e.g. semantic versioning) that the back-end can query and reason about programmatically. +* **Rationale:** Principles 7, 21, 23, and 33 each introduce their own versioned artefact — object model, firmware image, configuration schema, and the supported version skew between them. Without a single scheme relating these to each other, the back-end cannot reliably answer whether a given combination of firmware, config schema, and protocol version is a supported one, which undermines the interoperability contract Principle 33 depends on. +* **Engineering Implications:** + * Adopt one versioning policy (e.g. semantic versioning: major.minor.patch) applied consistently across firmware, configuration schema, and protocol/object model, with documented rules for what counts as a breaking versus additive change. + * Expose all relevant version identifiers — firmware, hardware revision, bootloader, configuration schema, protocol — together in a single queryable device record, rather than scattered across separate, undocumented fields. + * Validate at commissioning and at every update that the resulting combination of versions on the device is a documented, supported combination, rejecting the update otherwise. +* **Verification:** A compatibility-matrix check runs as part of every release, confirming the new version combination is added to the documented supported-combinations list before rollout; a periodic audit confirms fielded devices report only documented, supported version combinations. + +## 6. Code Quality, Memory Safety & Verification + +### Principle 35: Memory Safety by Default + +* **Statement:** Network-facing stacks, parsers, and application logic must enforce spatial and temporal memory safety at compile time. +* **Rationale:** Pointer arithmetic errors, buffer overflows, use-after-free conditions, and uninitialised reads represent the vast majority of exploitable embedded vulnerabilities. +* **Engineering Implications:** + * Implement modern memory-safe languages such as **Rust** for communications stacks, protocol parsing, and application-layer state machines. + * Where C/C++ is retained for legacy integration or the regulated-function core, mandate strict bounded buffer constructs, bounds-checked APIs, and compiler-level stack protection flags (`-fstack-protector-strong`, `-D_FORTIFY_SOURCE=2`). + * The Rust/C language boundary follows the core isolation boundary defined in Principle 1: code within the regulated or safety-critical core remains MISRA C (Principle 37) for certification continuity, while application, comms, and protocol code above it is the target for Rust adoption. + * **Tier scaling:** This formally gated boundary applies to Tier 2 and Tier 3 devices, which have a core to gate it against. A Tier 1 device with no regulated or safety-critical core may apply memory-safe practice pragmatically instead, for example Rust throughout, or C with mandatory sanitiser and static-analysis coverage, without a formal MISRA-gated boundary. +* **Verification:** New network-facing or parsing code is reviewed against the language boundary policy before merge; any exception (C code above the boundary) requires an ADR with a documented justification and mitigation. + +### Principle 36: Zero-Warning Compile-Time Hygiene & Static Analysis + +* **Statement:** Build pipelines must enforce zero-warning compilation under strict compiler configurations and automated static application security testing (SAST). +* **Rationale:** Compiler warnings are early indicators of undefined behaviour, type mismatches, or dead code. Over time, unaddressed warnings accumulate and obscure critical issues. +* **Engineering Implications:** + * Configure build flags with `-Wall -Wextra -Werror -pedantic` (or language equivalents such as Rust Clippy with `#![deny(warnings)]`). + * Integrate automated SAST tools (e.g. PC-lint Plus, Clang-Tidy, CodeQL) directly into the pull request verification pipeline. + * Automated checks must pass with zero unresolved issues before any code can be merged into production branches. +* **Verification:** CI configuration is periodically audited to confirm the strict flag set is still enforced and has not been silently relaxed; SAST findings are tracked to zero open critical/high issues per release. + +### Principle 37: Conformance to Safety-Critical Coding Standards + +* **Statement:** C codebases within the regulated or safety-critical core (Principle 1) and in other safety-critical domains must comply with industry safety standards (e.g. MISRA C:2012/2023, SEI CERT C), with any deviations formally documented and justified. +* **Applies to:** Tier 2 and Tier 3, scoped to the regulated or safety-critical core only. A Tier 1 device with no such core is not required to gate any of its code against MISRA, though the general defensive-programming and static-analysis principles in this pillar (38-42) still apply. +* **Rationale:** Standard C allows broad implementation-defined behaviours that can lead to undefined states across differing compiler versions, optimisation levels, and target microcontrollers. +* **Engineering Implications:** + * Enforce automated MISRA compliance checks as an automated stage in continuous integration. + * Prohibit dangerous constructs: ban dynamic memory allocation, restrict casting between incompatible pointers, eliminate recursion, and enforce bounded loops. + * Any necessary deviation (e.g. vendor HAL code) must be logged via an Architecture Decision Record (ADR) with an accompanying mitigation assessment. +* **Verification:** MISRA compliance reports are generated and reviewed every release; outstanding deviations are tracked against their ADRs and reassessed at each major firmware version. + +### Principle 38: Defensive Programming and Bounded Input Execution + +* **Statement:** All external and inter-module interfaces must validate inputs defensively and fail into safe, deterministic states. +* **Rationale:** Unvalidated inputs from local ports, wireless packets, or misbehaving peripherals can trigger unhandled panics, hard faults, or infinite loops that drain battery power. +* **Engineering Implications:** + * Enforce pre-condition and post-condition parameter validation across all public API boundaries. + * Avoid raw sentinel error codes; utilise explicit `Result` or `Option` types. + * Every loop parsing variable-length incoming frames must have an absolute, bounded iteration limit to prevent denial-of-service through CPU starvation. +* **Verification:** Boundary and negative-path unit tests are required for every public API and are tracked in coverage reporting; code review checklists explicitly confirm bounded-loop and fail-safe-state handling before merge. + +### Principle 39: Systematic Fault Classification and Graceful Degradation + +* **Statement:** The firmware must implement a defined fault-severity taxonomy (e.g. transient, degraded, critical) with a consistent escalation path from local retry through degraded operation to safe shutdown and alerting, applied uniformly across modules rather than decided ad hoc at each call site. +* **Rationale:** Principle 38 ensures individual interfaces fail safely at the point of detection; without an architecture-level fault taxonomy sitting above that, each module tends to invent its own notion of "safe" independently, producing inconsistent behaviour (one module retries forever, another resets the whole device) for faults of similar severity. +* **Engineering Implications:** + * Define a small, fixed set of fault severities and the escalation behaviour for each, applied consistently: for example, a transient sensor read failure retries locally; a repeated failure degrades the affected function while continuing others; a critical failure moves the device to a documented safe state and raises an alert. + * Define explicit, bounded behaviour for resource exhaustion (a full queue, an exhausted buffer pool): a documented drop policy (oldest vs newest) or backpressure signal, never an unbounded wait or undefined behaviour. + * Route all fault escalations through a single, auditable fault-handling module rather than allowing each subsystem to implement its own shutdown or alerting logic. +* **Verification:** A fault-injection test suite exercises each defined severity level across representative modules and confirms the observed escalation behaviour matches the documented taxonomy; resource-exhaustion behaviour is tested by deliberately saturating each bounded queue/buffer. + +### Principle 40: Continuous Fuzzing and Sanitiser-Driven Testing + +* **Statement:** All external protocol parsers, serial decoders, and payload handlers must undergo continuous coverage-guided fuzzing and sanitiser testing. +* **Rationale:** Handcrafted test vectors validate expected behaviours but often miss malformed, edge-case frame structures that cause memory corruption or buffer overflows. +* **Engineering Implications:** + * Build protocol parsers for execution under AddressSanitizer (ASan), UndefinedBehaviorSanitizer (UBSan), and MemorySanitizer (MSan). + * Incorporate coverage-guided fuzzing (e.g. LibFuzzer, AFL++) into CI regression testing, running millions of mutated frames against the device's application protocols and local interfaces. + * Fix all parser crashes and undefined behaviour warnings discovered during fuzz runs prior to release tagging. +* **Verification:** Fuzzing corpus coverage and crash counts are reported per release; no release ships with an open, unresolved sanitiser-detected crash. + +### Principle 41: Low Cyclomatic Complexity and Modular Cohesion + +* **Statement:** Software units must remain small, focused, and maintainable, enforcing strict limits on structural and branching complexity. +* **Rationale:** Sprawling switch statements and heavily nested condition trees are difficult to test, prone to regressions, and challenging to verify during safety reviews. +* **Engineering Implications:** + * Limit function cyclomatic complexity to a maximum score (typically ≤ 10 to 15) and restrict nesting to no more than 3-4 levels deep. + * Functions must perform a single logical responsibility. + * Maintain high branch coverage (≥ 90%) across the regulated-function drivers, state machines, and domain logic engines. +* **Verification:** Complexity metrics are calculated automatically per build and gate the pipeline on threshold breach; branch coverage reports are reviewed each release against the 90% target for the named domains. + +### Principle 42: Least Privilege and Domain Separation in Firmware + +* **Statement:** Firmware components must operate under least privilege, backed by hardware-enforced memory protection. +* **Rationale:** A vulnerability in a complex third-party radio stack (e.g. a cellular AT command parser or a Bluetooth/Zigbee stack) must not grant access to the regulated-function core, actuator shutoff logic, or cryptographic keys. +* **Engineering Implications:** + * Utilise hardware Memory Protection Units (MPU) or ARM TrustZone-M partitions to isolate tasks into unprivileged execution domains. + * The network and communications task domain must have zero read/write access to regulated-function registers, flash writing controllers, or Secure Element keys. + * Inter-task communication between domains must use structured, authenticated inter-process communication (IPC) queues rather than open shared memory. +* **Verification:** Domain boundary enforcement is confirmed by fault injection (attempting cross-domain access from the comms task in a lab build) at each hardware or MPU configuration change. + +### Principle 43: Reproducible, Deterministic Builds + +* **Statement:** Any released firmware binary must be exactly reproducible, bit-for-bit, from its source and a documented, pinned toolchain. +* **Rationale:** Principle 12's SBOM is only trustworthy if the binary it describes can be independently regenerated and compared; without reproducible builds, an SBOM is an unverifiable claim about the binary rather than a checkable fact, and a compromised build server could silently insert unauthorised code between source and shipped image. +* **Engineering Implications:** + * Pin exact compiler, linker, and toolchain versions (not floating "latest" references) for every release, and record them alongside the SBOM. + * Eliminate non-deterministic build inputs (embedded build timestamps, absolute file paths, unordered file iteration) that would otherwise make two builds from identical source diverge. + * Publish or retain the build hash for each released image so it can be independently verified against a from-source rebuild. +* **Verification:** A from-source rebuild is performed independently of the primary build pipeline for each release and its hash compared against the released image; any mismatch blocks release until resolved. + +## 7. Whole-Device Compliance, Human Factors & Sustainability + +### Principle 44: Regulatory Certification & Type-Approval Traceability + +* **Statement:** Every hardware and firmware combination released to the field must hold current, traceable regulatory certification for all applicable regimes (radio, safety, EMC, and any regulated function certification) in each target market. +* **Rationale:** A field IoT device is typically a radio device and a mains- or battery-powered electrical product simultaneously, and in some domains a legally regulated instrument as well. Certification is market-specific and revision-specific; shipping an uncertified hardware/firmware combination is a compliance failure independent of engineering quality. +* **Engineering Implications:** + * Maintain a certification matrix mapping each shipped hardware revision and firmware version to its valid certificates and target markets. + * Treat any change affecting RF performance, safety-relevant circuitry, or a regulated function as a trigger for re-certification assessment before release. + * Retain certification test reports and traceability records for the full device warranty period plus any regulatory retention requirement. +* **Verification:** The certification matrix is reviewed and signed off before any release is approved for a given market; re-certification triggers are checked as a mandatory step in the change approval process for RF, safety, or regulated-function changes. + +### Principle 45: Installation, Commissioning & Field Ergonomics + +* **Statement:** The physical device, its connectors, and its commissioning workflow must be designed for safe, low-error installation by a field technician under real-world site conditions. +* **Rationale:** Installation happens once, at scale, often by third-party or subcontracted technicians working to a time budget, in poor lighting or awkward physical access. Errors made here (miswiring, incomplete commissioning, incorrect configuration) generate costly return visits and, in domains involving hazardous processes, direct safety risk. +* **Engineering Implications:** + * Design mechanical keying and connector polarisation to make incorrect physical connection difficult or impossible. + * Provide clear, deterministic local commissioning feedback (LED sequence, display state, or audible confirmation) that unambiguously confirms successful join to the network and correct configuration. + * Support commissioning via a local port or short-range interface (Principle 10) without requiring a live WAN connection at install time. +* **Verification:** A structured usability trial with representative field technicians is run before rollout, measuring installation time and error rate; commissioning failure modes observed in the trial are fed back into connector or workflow design before mass deployment. + +### Principle 46: End-User and Operator Accessibility and Safety Communication + +* **Statement:** Any human-facing interface must communicate status and safety-critical information in a way that is accessible to users with visual, auditory, or cognitive impairments, and must not rely solely on a smartphone app or WAN connectivity. +* **Rationale:** Devices with a direct human interface reach a wide range of users, including elderly and disabled users, and communicate safety-relevant events that carry real consequences if missed. Accessibility here is a safety property, not only a usability preference. +* **Engineering Implications:** + * Provide critical alerts (an alarm condition, a service interruption, a safety shutoff) through at least one channel that does not depend on a smartphone app or active WAN session. + * Follow established accessibility guidance (e.g. WCAG-equivalent contrast and text-size principles adapted to embedded displays) for any on-device screen. + * Support audible or tactile alerting as an alternative to visual-only indication where the device or a companion display is the primary interface. +* **Verification:** Human-facing interfaces are reviewed against accessibility guidance before release, with a documented sign-off; critical alert delivery is tested with WAN connectivity deliberately disabled to confirm local-channel fallback works. + +### Principle 47: Circularity, Sustainability & End-of-Life Decommissioning + +* **Statement:** The device must support secure, complete decommissioning at end of life, and its materials and packaging must meet applicable environmental and circularity regulation. +* **Rationale:** At fleet scale, end-of-life handling is both a security event (a removed device still holds cryptographic keys and any locally retained personal or operational history) and an environmental and regulatory obligation (WEEE, RoHS, and equivalent regimes), distinct from the component-level obsolescence concerns in Principle 18. +* **Engineering Implications:** + * Define a secure decommissioning procedure covering key zeroisation, data erasure, and back-end de-registration, executable by a technician at removal without requiring the device to be network-connected. + * Select materials and construction consistent with RoHS restrictions and design for practical disassembly and material recovery at end of life. + * Specify packaging and logistics materials to meet applicable recyclability and transport-durability requirements. +* **Verification:** The decommissioning procedure is demonstrated and timed during field trials; material compliance declarations (RoHS, WEEE) are held on file and reviewed at each significant BOM revision. + +## Summary Matrix: Pillars vs. Lifecycle Impact + +| Principle Group | Primary Lifecycle Impact | Key Design Goal | +| :--- | :--- | :--- | +| **1. Core Function Isolation & Autonomy** | Compliance, Reliability, Social Safety | Certified/safety-critical core isolation, uninterrupted core function, zero network dependency, governed remote control, hardware-level fail-safe defaults | +| **2. Security & Crypto** | Threat Mitigation, Field Survivability, Privacy | Silicon-based trust, verified boot chain, physical tamper resilience, secure factory provisioning, crypto agility, supply chain transparency, data minimisation | +| **3. Power & Environment** | Field Lifespan (10-20 Yrs) | Microamp-second budgeting, environmental sealing, surge and intrinsic safety compliance, EMC/RF coexistence | +| **4. Product & Cost** | Total Cost of Ownership | Multi-decade TCO optimisation, swappable communications, obsolescence protection, manufacturability and test coverage | +| **5. Firmware Architecture** | Remote Maintainability, Real-Time Correctness, Release Safety | Failsafe dual-bank OTA and staged rollout governance, configuration and variant management, wear-levelled flash, atomic state persistence, layered architecture, timing determinism, unified versioning and cross-version interoperability | +| **6. Code Quality & Safety** | Software Robustness, Security | Compile-time memory safety, MISRA/CERT adherence, systematic fault handling, fuzzing, reproducible builds | +| **7. Whole-Device Compliance & Sustainability** | Regulatory Standing, Human Safety, Environmental Impact | Type-approval traceability, technician and end-user usability, secure and compliant end-of-life | + +## Glossary + +* **AFL++ / LibFuzzer:** Coverage-guided fuzzing tools that generate mutated inputs to find crashes and undefined behaviour in parsers. +* **ALT / HALT:** Accelerated Life Testing / Highly Accelerated Life Testing — compressed-time methods for estimating long-term component reliability. +* **ASan / UBSan / MSan:** AddressSanitizer, UndefinedBehaviorSanitizer, MemorySanitizer — compiler-based tools that detect memory and undefined-behaviour bugs at runtime. +* **ATEX / IECEx:** European and international certification schemes for equipment used in potentially explosive atmospheres. +* **BIST:** Built-In Self-Test — hardware or firmware routines that exercise and verify a device's own circuitry, typically during manufacturing test or boot. +* **BOD:** Brownout Detect — a hardware interrupt triggered when supply voltage drops below a safe operating threshold. +* **Canary / Ring Deployment:** A staged rollout pattern that releases a change to a small, representative subset of the fleet before progressively wider deployment, so problems surface before general availability. +* **CBOR:** Concise Binary Object Representation — a compact, binary data serialisation format suited to constrained networks. +* **CoAP:** Constrained Application Protocol — a lightweight web-transfer protocol designed for constrained IoT devices and networks. +* **CVE:** Common Vulnerabilities and Exposures — a public identifier for a known software or hardware vulnerability. +* **DevID (IEEE 802.1AR):** A standard for a globally unique, cryptographically verifiable device identity provisioned at manufacture. +* **DFM / DFT:** Design for Manufacturability / Design for Testability — hardware design practice that builds in structured, automatable production test coverage. +* **DMA:** Direct Memory Access — a hardware mechanism that moves data (e.g. from a sensor) without CPU intervention, saving power. +* **ECDSA / Ed25519:** Elliptic-curve digital signature algorithms used to verify authenticity and integrity of firmware and messages. +* **eDRX / PSM:** Extended Discontinuous Reception / Power Saving Mode — 3GPP power-saving modes for cellular IoT devices (e.g. NB-IoT). +* **EMC:** Electromagnetic Compatibility — a device's ability to operate correctly in its intended electromagnetic environment without causing or suffering interference. +* **GDPR:** General Data Protection Regulation — the EU's data protection and privacy law; equivalent regimes exist in other jurisdictions. +* **HAL:** Hardware Abstraction Layer — a software layer that isolates application logic from hardware-specific implementation detail. +* **HBOM:** Hardware Bill of Materials — a structured, verifiable list of a device's physical components and their provenance. +* **HSM:** Hardware Security Module — a dedicated, tamper-resistant device for generating, storing, and using cryptographic keys. +* **IP67 / IP68:** Ingress Protection ratings under IEC 60529 describing resistance to dust and water ingress. +* **IPC:** Inter-Process Communication — structured mechanisms (e.g. queues, mailboxes) by which isolated tasks or processes exchange data. +* **JTAG / SWD:** Joint Test Action Group / Serial Wire Debug — standard hardware debug interfaces that must be disabled or fused off in production. +* **LoRaWAN / Wi-SUN / Thread:** Low-power wide-area and mesh networking protocols used for IoT connectivity. +* **LTE-M / NB-IoT:** Cellular IoT connectivity standards (LTE for Machines; Narrowband IoT) optimised for low power and wide coverage. +* **LwM2M:** Lightweight M2M — an open, standardised device management and data-model protocol for constrained IoT devices. +* **MISRA C:** A set of software development guidelines for the C language, widely used in safety-related embedded systems. +* **MPU:** Memory Protection Unit — hardware that restricts which memory regions a given task or process can access. +* **NCSC / NIST:** UK National Cyber Security Centre / US National Institute of Standards and Technology — bodies that publish cryptographic and security guidance. +* **OPC-UA:** Open Platform Communications Unified Architecture — an open, platform-independent industrial interoperability standard. +* **OTA:** Over-the-Air — remote delivery of firmware or configuration updates to a deployed device. +* **RoT:** Root of Trust — the foundational, inherently trusted hardware or firmware component from which a device's security is anchored. +* **RoHS / WEEE:** Restriction of Hazardous Substances / Waste Electrical and Electronic Equipment — EU directives governing materials and end-of-life handling of electronic products. +* **RTOS:** Real-Time Operating System — an operating system designed to guarantee deterministic response times for scheduled tasks. +* **SAST:** Static Application Security Testing — automated analysis of source code to find security weaknesses without executing it. +* **SBOM:** Software Bill of Materials — a structured, machine-readable list of the software components in a build, including third-party and open-source dependencies. +* **SE:** Secure Element — a dedicated, tamper-resistant hardware chip for secure key storage and cryptographic operations. +* **SemVer:** Semantic Versioning — a versioning scheme (major.minor.patch) that encodes the compatibility impact of a change directly in the version number. +* **TCO:** Total Cost of Ownership — the full lifecycle cost of a device or component, including failure, warranty, and dispatch costs, not just its purchase price. +* **TCXO:** Temperature-Compensated Crystal Oscillator — a clock source that corrects for temperature-induced drift. +* **TrustZone-M:** An ARM hardware security extension for Cortex-M microcontrollers that creates isolated secure and non-secure execution domains. +* **WCAG:** Web Content Accessibility Guidelines — a widely referenced accessibility standard, applied here by analogy to embedded displays and interfaces. +* **WCET:** Worst-Case Execution Time — the longest possible time a piece of code can take to run, used to budget real-time deadlines. diff --git a/content/standards/iot-edge-offline-storage.md b/content/standards/iot-edge-offline-storage.md new file mode 100644 index 0000000..fe20fa5 --- /dev/null +++ b/content/standards/iot-edge-offline-storage.md @@ -0,0 +1,140 @@ +--- +title: Offline storage and synchronisation +--- + +# Offline storage and synchronisation + +## 1. Executive Summary & Problem Statement + +Connected devices deployed in the field experience unpredictable network connectivity due to severe weather, cellular coverage dead zones, gateway reboots, and planned sleep cycles. Despite continuous or extended connectivity outages, connected devices must capture mission-critical sensor telemetry, system audit events, and tamper alarms without data loss. + +However, non-volatile solid-state storage such as NOR and NAND flash has physical limits. Flash blocks wear out with repeated write-erase cycles, and writing needs a block erase first. A sudden power loss during a write can corrupt the storage structures. + +This guideline establishes mandatory architectures for embedded storage hierarchy selection, wear-leveling filesystem implementation, offline FIFO priority queuing, and reliable cloud synchronization. + +```mermaid +flowchart LR + Sensor["Sensors & Events"] --> RingBuffer["Priority Spooling Engine
(LittleFS / Flash Ring)"] + RingBuffer --> Memory["Non-Volatile Flash
(Wear-Leveled Sectors)"] + Memory --> SyncEngine{"Cloud Sync Engine"} + SyncEngine -->|Online & Connected| Transmit["Idempotent Chunk Upload"] + SyncEngine -->|Offline Outage| Buffer["Accumulate in FIFO Buffer"] + Buffer --> Memory +``` + +[[CAPTION:Figure]] Edge offline storage and synchronization lifecycle. + +## 2. Embedded Non-Volatile Storage Hierarchy + +Selecting the appropriate storage technology depends on write frequency, endurance requirements, and system cost: + +| Memory Technology | Erase Granularity | Write Endurance | Write Speed | Standby Power | Typical Use Case | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **SPI NOR Flash** | Sector (4 KB / 64 KB) | 100,000 cycles / block | Moderate (~1 MB/s) | Ultra-low (< 1 $\mu\mathrm{A}$) | Firmware images, telemetry queues, LittleFS | +| **SPI NAND Flash** | Block (128 KB - 256 KB) | 10,000 - 100,000 cycles | High (~10-50 MB/s) | Low (~10 $\mu\mathrm{A}$) | Edge gateways, multi-megabyte logging | +| **I2C/SPI EEPROM** | Byte or Page (32-64 B) | 1,000,000+ cycles | Slow (~100 KB/s) | Ultra-low (< 1 $\mu\mathrm{A}$) | Device calibration, serial numbers, boot counters | +| **FRAM (Ferroelectric)**| Byte | $10^{14}$ (Virtually unlimited) | High (Bus speed) | Ultra-low (< 1 $\mu\mathrm{A}$) | High-frequency telemetry write buffers | +| **eMMC / SD Card** | Internal controller | Variable (Managed) | Fast (50+ MB/s) | Medium (~100 $\mu\mathrm{A}$) | Linux edge gateways, video/audio caching | + +[[CAPTION:Table]] Non-volatile memory technology comparison for IoT devices. + +## 3. Flash Memory Physics and Wear Leveling + +Unlike volatile RAM, solid-state flash memory cannot overwrite existing data in place. Storage must be erased in large blocks (typically 4 KB sectors or 64 KB blocks) before individual pages (256 bytes) can be programmed from `1`s to `0`s. + +```mermaid +flowchart TD + subgraph EraseEndurance["Flash Block Degradation"] + Block0["Block 0 (100,000 Erase Limit)"] + Block1["Block 1 (100,000 Erase Limit)"] + BlockN["Block N (100,000 Erase Limit)"] + end + Controller["Wear Leveling Engine (LittleFS)"] -->|Distributes Erases Evenly| EraseEndurance +``` + +[[CAPTION:Figure]] Wear leveling engine distributing write-erase cycles across physical flash blocks. + +### 3.1 Flash Wear-Out Calculations + +Repeated oxide breakdown during high-voltage erase cycles eventually causes flash cells to leak charge, leading to unrecoverable bit-flips. + +Consider a device recording a 64-byte telemetry record every 60 seconds directly to a fixed 4 KB flash sector: +- Writes per day: $1,440$ records. +- Erases required per day: $\frac{1440 \times 64\,\mathrm{B}}{4096\,\mathrm{B}} = 22.5\mathrm{erases/day}$. +- Total sector lifespan without wear leveling: $\frac{100,000\mathrm{cycles}}{22.5\mathrm{erases/day}} \approx 4,444\mathrm{days} \approx 12.1\mathrm{years}$. + +However, if firmware updates a single 4-byte counter in place by erasing the entire 4 KB sector each minute: +- Lifespan: $\frac{100,000}{1440} \approx 69.4\mathrm{days}$. The flash chip is physically destroyed in under ten weeks. + +### 3.2 Filesystem Standard: LittleFS Mandate + +For microcontroller firmware with external or internal SPI NOR flash, **LittleFS** is the mandatory embedded filesystem: + +1. **Power-Loss Resilience**: LittleFS uses a copy-on-write architecture with atomic pointer updates. If power is interrupted mid-write, the filesystem rolls back cleanly to the last consistent state without sector corruption. +1. **Dynamic and Static Wear Leveling**: LittleFS tracks erase counts per block and redistributes active and static files to ensure all physical flash blocks degrade uniformly. +1. **Bounded RAM Footprint**: LittleFS does not cache full filesystem trees in RAM, operating with fixed buffer pools (< 4 KB RAM). + +> [!NOTE] +> Legacy filesystems such as FAT12/FAT16/FAT32 or raw SPIFFS are strictly prohibited on raw SPI NOR flash: FAT relies on fixed-location File Allocation Tables that wear out rapidly, and SPIFFS suffers from non-deterministic garbage collection latency. + +## 4. Edge Buffer Architecture & Backpressure Management + +Devices must separate volatile memory logging from persistent flash spooling through a multi-tiered priority queue: + +```mermaid +flowchart TD + In["Incoming Events & Telemetry"] --> Classify{"Priority Classifier"} + Classify -->|Critical: Tamper / Brownout| P0["Queue P0: High Priority Flash Spool"] + Classify -->|Standard: Periodic Telemetry| P1["Queue P1: Standard Flash FIFO"] + Classify -->|Diagnostic: Verbose Trace| P2["Queue P2: Ephemeral RAM Buffer"] + + P0 --> Flash["Persistent Flash Partition"] + P1 --> Flash + P2 --> RAM["Volatile RAM Ring Buffer"] + + Flash --> SyncEngine["Cloud Synchronizer"] +``` + +[[CAPTION:Figure]] Multi-tier priority queue segregating critical alarms from periodic telemetry. + +### 4.1 Backpressure Policies During Extended Outages + +When connectivity is lost for days or weeks, the persistent flash storage buffer will eventually approach capacity. The firmware must execute deterministic backpressure strategies: + +1. **Priority Eviction**: Diagnostic traces (Queue P2) are dropped immediately. +1. **Telemetry Downsampling**: Standard periodic telemetry (Queue P1) is downsampled or aggregated (for example, converting 1-minute samples into hourly minimum, maximum, and average records). +1. **Oldest-Record Dropping**: If flash utilization exceeds 95%, the oldest unacknowledged routine telemetry records must be discarded first (FIFO drop-head). High-priority alarms (Queue P0) must never be evicted. + +## 5. Reliable Cloud Synchronization Protocol + +Reconnecting after a multi-day network outage can cause a flood of buffered data that overwhelms cellular bandwidth or cloud ingestion endpoints. The synchronization subsystem must adhere to the following protocol rules: + +### 5.1 Cursor-Based Checkpointing + +Data must be retrieved and uploaded using monotonic cursor offsets rather than deleting files immediately upon sending: + +```mermaid +sequenceDiagram + participant Edge as Edge Device (Flash) + participant Cloud as Cloud Ingest Service + + Edge->>Edge: Read Chunk from Flash (Cursor: 12000 to 14000) + Edge->>Cloud: POST /telemetry/batch (Records, BatchID: 42, Range: 12000-14000) + alt Upload Successful + Cloud-->>Edge: HTTP 200 OK / MQTT ACK (BatchID: 42 confirmed) + Edge->>Edge: Advance Commit Cursor to 14000, then Release Flash Blocks + else Connection Interrupted + Note over Edge,Cloud: Timeout / Socket Drop + Edge->>Edge: Keep Commit Cursor at 12000, then Retry upon reconnect + end +``` + +[[CAPTION:Figure]] Cursor-based transaction checkpointing ensuring zero data loss during sync failures. + +### 5.2 Idempotency and Deduplication + +Because network connections may drop immediately after the server receives a batch but before the device receives the acknowledgment, duplicate deliveries will occur. + +1. Every telemetry record must contain a unique device-scoped monotonic sequence number and UTC epoch timestamp. +1. The cloud ingestion pipeline must implement an idempotent deduplication window (using Redis or DynamoDB keys) to suppress retransmitted batches without failing the upload pipeline. +1. Uplink batch sizes must be capped to fit within standard transport MTUs (for example, 512 bytes to 1024 bytes per cellular transmission burst) to prevent fragmented packet retransmissions. diff --git a/content/standards/lwm2m-objects.md b/content/standards/lwm2m-objects.md new file mode 100644 index 0000000..23e708f --- /dev/null +++ b/content/standards/lwm2m-objects.md @@ -0,0 +1,374 @@ +--- +title: Designing LwM2M objects and resources +--- + +# Designing LwM2M objects and resources + +This guide covers how to design new LwM2M objects, how to group or split data items, and how to choose resource access types (R, W, RW, or Execute).\ +It works for: sensors, actuators, gateways, industrial controllers, wearables, home devices, vehicles, etc. + +## What an LwM2M Object Is + +An LwM2M *object* is a structured container used to model a feature, capability, or function of an IoT device.\ +It defines a standard way for a server and client to exchange configuration, commands, and data. + +Objects are the building blocks of the LwM2M data model. + +### How an Object Is Structured + +An object contains: + +- Object ID\ + A unique numeric identifier (for example, 3 for Device, 5 for Firmware Update). Standard object IDs are defined by OMA, but manufacturers can also use their own custom object IDs for device-specific functionality. +- Object Instances\ + One object can have **one instance** or **many instances**.\ + Each instance represents one occurrence of the object’s concept. +- Resources\ + Each instance contains a set of **resources**.\ + Resources are the individual data points, settings, or actions the server interacts with. + +A resource can be: + +- a configuration value (write), +- a measurement (read), +- a command (execute), +- or a value that both sides read and write (RW). + +### Basic Hierarchy + +LwM2M addresses data using a simple three-level path: + +```swift +/ObjectID/InstanceID/ResourceID +``` + +A resource is always accessed through this path. + +### Examples + +```pgsql +/1000/0/1 → Read a status value +/1000/0/2 → Update configuration +/1000/0/3 → Execute an action +``` + +### Single-instance vs Multi-instance Objects + +Objects may support **one** instance or **multiple**: + +#### Single-instance objects + +Used when the device only ever has one of something. + +Structure: + +```swift +/ObjectID/0/ResourceID +``` + +There will only ever be instance **0**. + +#### Multi-instance objects + +Used when a device has a list of similar items or entries. + +Structure: + +```bash +/ObjectID/0/... +/ObjectID/1/... +/ObjectID/2/... +``` + +Each instance represents one item, record, channel, or entity. + +### Resource Types + +Each resource has: + +- ID – numeric identifier +- Name – descriptive field name +- Type - data representation used by the resource + - Integer Whole numbers, often used for counters, IDs, thresholds, flags. + - String: human-readable text or structured text fields. + - Boolean True/false states. + - Opaque Raw binary data, uninterpreted by LwM2M (keys, payloads, custom formats). + - Time Timestamp in seconds since Unix epoch (signed integer). + - ObjLnk A reference to another object instance in the form `:`. +- Access Mode – how the server interacts with the resource + - R (Read) The server can read the value; the device owns and updates it. + - W (Write) The server sets the value; used for configuration or parameters. + - RW (Read/Write) Both sides can access the value; use only when the value is legitimately shared. + - E (Execute) Triggers an action with side effects (commands, operations, one-shot behaviour). +- Mandatory/Optional flag +- Description – what it means and how it behaves + +The object definition lists all resources available for any instance. + +### Why Object Design Matters + +A well-designed object: +- is clear and unambiguous +- cleanly separates configuration, status, and commands +- is secure by design +- is easy for clients and servers to implement +- supports versioning without redesigning the protocol +- avoids coupling unrelated behaviours into a single object + +Good object modelling ensures predictable behaviour across millions of devices and years of lifecycle. + +## Define the purpose of the object + +Every object should have a **single, clear purpose**.\ +A simple method: + +**“This object exists to…”** + +Examples of good-purpose statements (generic): + +- “…configure communication behaviour.” +- “…report sensor measurements.” +- “…control a device function.” +- “…manage a firmware update operation.” + +If the purpose needs multiple “and”s, it’s too broad. + +## Group data by lifecycle and behaviour + +A clean LwM2M model groups resources that naturally belong together.\ +Ask: + +### Do these items change together? + +If yes, keep them in the same object. + +### Do they change independently? + +If yes, they likely belong in different objects. + +You’re aiming to keep: + +- long-lived configuration separate from +- real-time measurements, and separate again from +- operational commands, and separate again from +- transaction-like actions. + +Grouping by lifecycle reduces coupling and keeps versioning straightforward. + +## Identify resource access types (R, W, RW, Execute) + +Use simple rules: + +### Read (R) + +Values the device exposes but the server should not modify.\ +Typical patterns: + +- Status +- Sensor readings +- Identity information +- Diagnostics + +### Write (W) + +Values the server can set, normally configuration or modes.\ +Patterns: + +- Thresholds +- Timers +- Limits +- Operational parameters + +### Execute (E) + +An operation with side effects.\ +Patterns: + +- Trigger a function +- Reset something +- Start/stop an operation +- Load or apply something + +### Read–Write (RW) + +Only used when both sides legitimately control a value. + +RW is valid but should be used sparingly because it blurs ownership. + +## Questions to ask when choosing access types + +A small set of questions helps you classify properly: + +### Does changing this value cause behaviour? + +If yes, it’s usually Execute, not Write. + +### Is the value permanent or semi-permanent? + +If yes, Write (configuration).\ +If no, maybe it’s status (Read). + +### Does the device own the value? + +If the device produces the value and the server only observes it -> Read. + +### Does the server own the value? + +If the server controls the value → Write. + +### Do both need to influence it? + +If genuinely yes-> RW. + +### Is it a one-off operation? + +If so -> Execute. + +## Avoid mixing configuration, state, and commands + +This is the biggest design mistake people make. + +### Keep these separate + +- Configuration +- Current operational status +- Commands / actions +- Historical/event data + +Each has its own cadence and behaviour. + +If you mix them inside one object: + +- Access rights become messy +- Version control becomes awkward +- You risk unintended updates or accidental overwrites +- Servers become more complex + +Split objects whenever these concerns diverge. + +## Single-instance vs multi-instance + +Ask: **“Could there ever be more than one of these?”** + +### Single-instance + +Use when only one set of data makes sense: + +- Device identity +- Core configuration +- Communication parameters + +### Multi-instance + +Use when items form a list or history: + +- Sensors of the same type +- Logs +- Events +- Tasks +- Rules/schedules +- Per-channel configurations + +Watching this early prevents awkward redesigns later. + +## Object and resource naming + +Keep naming: + +- Neutral +- Descriptive +- Domain-independent +- Easy to understand at a glance + +Good names simplify onboarding and maintenance. + +Avoid: + +- Abbreviations that need a dictionary +- Vendor-specific naming inside generic objects +- Jargon that ties the object to one use case + +## Respect security boundaries + +Security is often the reason to split objects. + +Keep separate: + +- Sensitive parameters +- Credentials +- Commands that produce physical or financial effects +- Logs that may contain sensitive data + +If different security levels apply, you need separate objects. + +## Versioning considerations + +Objects tend to live for years.\ +Design to make versioning manageable: + +- Keep objects narrow in scope +- Don’t overload a single object with too many responsibilities +- When behaviour changes meaningfully, version the object rather than hack new semantics into an existing resource +- Use optional resources for new features if backwards compatibility matters + +## Decision checklist (simple “flowchart in words”) + +:::mermaid +flowchart TD + + Start([Start Object Design]) --> CheckPurpose + + CheckPurpose{Is the object's purpose clear?} + CheckPurpose -->|Yes| ClassifyData + CheckPurpose -->|No| SplitForPurpose[Split the object based on purpose] --> ClassifyData + + ClassifyData{Are data items config, status, or operations?} --> CheckLifecycle + + CheckLifecycle{Do any items differ in lifecycle, update rate, or ownership?} + CheckLifecycle -->|Yes| SplitForLifecycle[Split objects based on lifecycle differences] --> CheckSecurity + CheckLifecycle -->|No| CheckSecurity + + CheckSecurity{Do any items require separate security?} + CheckSecurity -->|Yes| SplitForSecurity[Split objects based on security boundaries] --> DetermineOwnership + CheckSecurity -->|No| DetermineOwnership + + DetermineOwnership{Who owns each resource?} + DetermineOwnership --> DeviceRead[Device-Owned → Read] + DetermineOwnership --> ServerWrite[Server-Owned → Write] + DetermineOwnership --> SharedRW["Shared → Read/Write (only when justified)"] + + DeviceRead --> CheckForAction + ServerWrite --> CheckForAction + SharedRW --> CheckForAction + + CheckForAction{Is the resource an action?} + CheckForAction -->|Yes| UseExecute[Use Execute resource] + CheckForAction -->|No| CheckHistory + + CheckHistory{Does this require historical entries?} + CheckHistory -->|Yes| UseMultiInstance[Use a multi-instance object] + CheckHistory -->|No| End([End Object Design]) +::: + +## Recommended generic object structure template + +A clean structure you can reuse across projects: + +1. Object Purpose +1. High-level Description +1. Device Role (producer/consumer) +1. Resource Table: + - ID + - Name + - Type (string, int, boolean, opaque, time, objlink) + - Access (R, W, RW, E) + - Mandatory/optional + - Description +1. Security Notes +1. Instance Model: + - Single-instance or multi-instance +1. Expected Client Behaviour +1. Expected Server Behaviour +1. Versioning Notes + +This ensures consistent modelling across teams. diff --git a/content/standards/misra-c.md b/content/standards/misra-c.md new file mode 100644 index 0000000..c25d580 --- /dev/null +++ b/content/standards/misra-c.md @@ -0,0 +1,392 @@ +--- +title: MISRA C +--- + +# MISRA C + +*A practical introduction to MISRA C and safe embedded software development* + +## What is MISRA? + +**MISRA C** is a set of **coding guidelines for the C language**, created to reduce defects in **safety-critical and high-reliability embedded systems**. + +It originated from the **Motor Industry Software Reliability Association (MISRA)** in the 1990s, when automotive systems started relying heavily on embedded software for safety. + +Since then, MISRA C has become the **de facto standard** for safe C in industries such as: + +- automotive +- industrial control +- aerospace +- medical +- defence +- energy metering +- telecoms +- marine/offshore + +Wherever **failure matters**, MISRA is commonly used. + +## MISRA is Not a Style Guide + +A common misconception is that MISRA is about: + +- formatting +- naming conventions +- code beauty +- personal preferences + +It isn’t. + +> **MISRA is a risk-reduction strategy**, not a style guide. + +Every rule in MISRA exists because a **pattern of real-world failures** caused injuries, field recalls, safety hazards, or extremely expensive field fixes. + +MISRA rules force developers to avoid **dangerous parts of C** — sharp edges in the language that are legal, but unsafe. + +Think of MISRA as a **map of known landmines** in C, based on decades of painful history. + +## Why We Use MISRA + +We use MISRA to: + +### Reduce Defects Early + +MISRA helps prevent bugs **before they reach production** by limiting: + +- undefined behaviour +- implementation-defined behaviour +- poorly-specified type conversions +- dangerous pointer use +- unbounded operations + +C gives you a **loaded gun**.\ +MISRA makes you **check where it’s pointing**. + +### Improve Maintainability + +Embedded code outlives the original authors.\ +MISRA encourages patterns that are **easy to understand**, so others can safely modify the code without triggering regressions. + +### Improve Portability + +C behaves differently on different compilers and architectures.\ +MISRA avoids features that depend on compiler-specific behaviour, so code works reliably on: + +- different MCUs +- different compilers +- different optimisation levels + +### Support Certification and Audit + +For any regulated or quality-critical product, MISRA is accepted as **evidence** that the development team: + +- uses industry best practice +- applies static analysis +- controls risk in software design + +It provides a **defensible engineering position**. + +## What MISRA Covers (High Level) + +MISRA C focusses on **dangerous or ambiguous aspects of C**, for example: + +- implicit type conversions +- signed/unsigned arithmetic +- pointer aliasing +- volatile access +- use of dynamic memory +- unreachable code +- data race risks +- unsafe casts +- recursion +- uninitialised variables +- hidden side-effects +- multiple exit points +- direct hardware access without structure + +It defines rules like: + +- “Do not cast away const” +- “No implicit arithmetic promotion” +- “Use explicit types for width” +- “No hidden side-effects in macros” + +You don’t need to memorise every rule.\ +You need to understand **why the rule exists**. + +## The Most Important Mindset + +### Fix the Cause, Not the Warning + +When a MISRA rule is broken, the question is not: + +> “How do I silence the violation?” + +It is: + +> **“What risk is this warning signalling?”** + +MISRA violations are often **symptoms** of deeper design issues: + +- wrong types +- wrong ownership model +- wrong interface contract +- pointer lifetime issues +- unsafe implicit conversion +- silent truncation / overflow +- race condition between ISR and task + +A **cast** or **suppression** may hide the violation, but it also hides the **actual safety problem**. + +#### Example Anti-Pattern + +**Bad “fix”:** + +```ini +value = (uint16_t)data; // cast added to silence MISRA +``` + +Underlying problem:\ +`data` is a 32-bit value that may overflow `uint16_t`. + +A cast doesn’t fix it — it hides it. + +**Good fix:**\ +Define the right type: + +```cpp +uint16_t value = clamp_value(data, UINT16_MAX); +``` + +Or fix the interface so the API returns the correct width. + +## How to Approach MISRA Violations + +Every MISRA violation fits one of **three categories**: + +### Real Defect + +MISRA highlights an actual bug: + +- uninitialised variable +- out-of-bounds write +- incorrect signedness +- potential overflow + +**Fix the bug.** + +### Design Problem + +MISRA exposes a **design flaw**: + +- wrong type design +- unclear ownership +- unsafe pointer hand-off +- duplicated logic in different modules +- implicit dependency on optimisation level + +**Fix the design**, not the line. + +### Acceptable Deviation + +Sometimes the **pattern is valid** in embedded work: + +- hardware register access +- memory-mapped IO +- controlled interrupt behaviour +- vendor HAL abstractions +- toolchain limitations + +In these cases: + +- use a **local deviation** with explanation +- clearly document the **technical rationale** +- keep it **visible**, not hidden + +## Deviations (When You Must Break a Rule) + +MISRA does **not** expect zero violations.\ +It expects: + +- violations are **known** +- violations are **justified** +- violations are **local** +- violations are **documented** +- violations do **not hide risk** + +Example good deviation note: + +> **Deviation — MISRA Rule 11.3**\ +> `uint8_t*` cast to `volatile uint8_t*` for direct hardware register access.\ +> Access is atomic. No aliasing.\ +> Required due to vendor register model. + +This gives **context for the next developer**. + +Avoid suppression like: + +```cpp +/* MISRA ignore */ +``` + +That helps nobody. + +## Practical Tips for Developers + +### Understand the Tool Output + +Static analysis tools report: + +- the **rule violated** +- the **reason** +- the **code location** + +Always read **the rationale**, not just the rule title.\ +MISRA includes **why the rule exists**. + +### Fix Types, Not Call Sites + +If you see repeated casts, clamps, or masks: + +- the **API is wrong** + +Example: + +Instead of casting 100 times: + +```scss +send((uint8_t)value); +``` + +Fix the interface **once**: + +```cpp +void send(uint8_t value) { … } +``` + +This reduces **both safety risk and static analysis noise**. + +### Respect Hardware + +In embedded software, violations often link to hardware behaviour: + +- `volatile` is required +- memory is not flat +- registers are not normal variables + +Sometimes MISRA **must be deviated** to express hardware behaviour correctly — but document it. + +### Learn the Common Patterns + +A few patterns reoccur constantly: + +- Table-driven state machines\ + (instead of scattered logic) + +- Fixed-width integer types\ + (`uint16_t`, `uint32_t`) + +- Avoid dynamic memory\ + (prefer static allocation) + +- Avoid side-effects in macros +- Single responsibility functions + +These patterns tend to be **MISRA-friendly** naturally. + +## What MISRA Does *Not* Solve + +MISRA reduces risk, but it does not guarantee: + +- correctness of logic +- performance +- that your design is good +- that someone won’t misuse the API +- that a race condition won’t slip through +- that all edge cases are tested + +MISRA + **strong reviews** + **testing discipline** is the correct combination. + +## Responsibilities During Code Review + +When reviewing MISRA findings: + +A reviewer should ask: + +- What risk is MISRA pointing to? +- Is there a better design? +- Does the change introduce future technical debt? +- Is the suppression safe and justified? +- Why does the current interface require a cast? +- Does this break blocking or timing? + +Approach the issue with **curiosity**, not accusation. + +## MISRA in Our Workflow + +In a modern embedded pipeline: + +- MISRA is run **automatically** in CI +- MISRA is run **locally** before PR +- MISRA must be **green** for merge +- Deviations must be **documented** +- Static analysis reports are **stored** +- New warnings are **blocked** + +The CI pipeline is **evidence** for requirement closure. + +## Learning MISRA Quickly + +You do **not** need to memorise every rule.\ +Focus on: + +- the **common patterns** +- why the rules exist +- what risks they address + +Over time: + +- you will naturally write MISRA-clean code +- the mindset becomes automatic + +The best MISRA developers rarely talk about “MISRA rules”, they talk about **safe C**. + +## Short Summary (For Your Desk) + +**MISRA is:** + +- a **safety strategy**, not a style guide +- a **map of how C has failed in practice** +- a way to **prevent bugs before they happen** +- a method to create **evidence for testing** +- a tool to support **maintainability** + +**When MISRA fails:** + +- fix the cause, not the warning +- redesign the **interface**, not the call-site +- deviations must be **justified**, not hidden +- keep violations **local and visible** + +## Recommended Reading (Optional) + +If you want to dive deeper: + +- *MISRA C:2012 Guidelines* (official) +- *Beyond MISRA: Designing Safe C Interfaces* (Barr Group webinars) +- *Effective C* — Robert Seacord +- *Best Kept Secrets of Peer Code Review* — SmartBear +- *Clean Code* — Robert Martin (concepts only) +- *Code Complete* — Steve McConnell + +And for deeper safety context: + +- ISO 26262-6 (Embedded Software) +- IEC 61508-3 +- CERT C Secure Coding Standard + +## Final Thought + +> **Writing MISRA-compliant code is less about following rules and more about thinking clearly.**\ +> If you understand **why** a rule exists, you’ll follow it naturally. + +MISRA is not the goal — **reliable, safe software** is the goal. diff --git a/content/standards/python.md b/content/standards/python.md new file mode 100644 index 0000000..649d7ca --- /dev/null +++ b/content/standards/python.md @@ -0,0 +1,476 @@ +--- +title: Python for supporting tools +--- + +# Python for supporting tools + +This document defines recommended standards for Python used in **supporting tools**, such as: + +- Build scripts +- Documentation generators +- Analysis utilities +- Automation helpers + +These guidelines focus on **clarity, maintainability, and predictability** rather than raw performance. + +They are intended to stand alone as practical engineering guidance. + +## Guiding Principles + +Python code should: + +- Be easy to read +- Be easy to modify +- Fail clearly +- Prefer explicit over implicit + +Design for the reader first, not the author. + +Assume the reader is: + +- Another engineer +- Six months in the future +- Under time pressure + +If the code is obvious to that person, it is good code. + +## Style Guide (PEP 8) + +All Python code should broadly follow PEP 8 formatting rules. + +Key conventions: + +- 4 spaces for indentation +- Line length ≤ 88 characters +- Snake_case for functions and variables +- PascalCase for classes +- UPPER_CASE for constants + +Good: + +```python +max_retries = 3 + +def load_config(path: Path) -> dict: + ... +``` + +Poor: + +```python +MaxRetries=3 +def LoadConfig(Path): + ... +``` + +Avoid tightly packed or clever one-liners. + +Good: + +```python +total = price * quantity +``` + +Poor: + +```python +total=price*quantity +``` + +Consistency matters more than personal preference. + +## Type Hints (PEP 484) + +Type hints document intent and enable static analysis. + +Use type hints for: + +- Function parameters +- Return types +- Public variables +- Class attributes + +Example: + +```python +def parse_file(path: Path) -> list[str]: + ... +``` + +Avoid untyped public functions: + +```python +def parse_file(path): + ... +``` + +Use concrete types where possible: + +```python +def read_lines(path: Path) -> list[str]: + ... +``` + +Instead of: + +```python +def read_lines(path: Path) -> list: + ... +``` + +Benefits: + +- Clear contracts +- Better IDE support +- Earlier bug detection + +Do not overcomplicate types. Prefer readability over perfection. + +## Docstrings (PEP 257) + +Docstrings explain what a module, class, or function **does and why it exists**. + +They are not comments. + +Docstrings should: + +- Use triple double quotes +- Start with a one-line summary +- Be written in plain English + +Example: + +```python +def generate_report(input_dir: Path) -> None: + """Generate summary report from input directory. + + Walks the directory tree, extracts metadata, and writes a JSON report. + """ +``` + +Bad docstring: + +```python +def add(a, b): + """Add two numbers.""" +``` + +Good docstring: + +```python +def add(a: int, b: int) -> int: + """Calculate total retry budget for scheduler.""" +``` + +Docstrings are required for: + +- Modules +- Public functions +- Classes + +Docstrings are optional for very small private helpers. + +## File Headers + +Each script should begin with a short header describing: + +- Purpose +- How to run +- Dependencies + +Example: + +```python +""" +Generate documentation PDF from Markdown sources. + +Usage: + python build_docs.py --input docs/ + +Requires: + pypandoc + pyyaml +""" +``` + +This allows someone to understand the script without opening the rest of the file. + +## Comments + +Comments explain **why**, not **what**. + +Good: + +```python +# Retry once to handle intermittent filesystem latency +``` + +Poor: + +```python +# Increment i by 1 +i += 1 +``` + +If a comment explains what the code does, the code should be rewritten instead. + +Avoid commented-out code. + +## Error Handling + +Errors should: + +- Be explicit +- Be meaningful +- Fail fast + +Good: + +```python +if not path.exists(): + raise FileNotFoundError(f"Config file not found: {path}") +``` + +Bad: + +```python +try: + load_file() +except: + pass +``` + +Catch exceptions only when: + +- You can recover, or +- You can add context + +Example: + +```python +try: + data = json.loads(raw) +except json.JSONDecodeError as exc: + raise ConfigError("Invalid configuration file") from exc +``` + +## Function and Module Structure + +Prefer: + +- Small functions +- One responsibility per function +- Shallow nesting + +Good: + +```python +def load_config(path: Path) -> dict: + raw = read_file(path) + return parse_config(raw) +``` + +Poor: + +```python +def load_config(path): + with open(path) as f: + data = f.read() + try: + obj = json.loads(data) + ... +``` + +Split logic into steps. + +## Imports + +Import order: + +1. Standard library +2. Third-party +3. Local + +Example: + +```python +from pathlib import Path +import json + +import yaml + +from app.config import Config +``` + +Never use wildcard imports. + +## Formatting and Linting Tools + +Use automated tools: + +- Black (formatting) +- Ruff (linting) + +Humans should not format code manually. + +Tools enforce consistency. + +## Virtual Environments + +All tools should run inside a virtual environment. + +Create: + +```bash +python -m venv .venv +``` + +Activate (Linux/macOS): + +```bash +source .venv/bin/activate +``` + +Activate (Windows): + +```powershell +.venv\Scripts\activate +``` + +Never commit `.venv`. + +## pyproject.toml + +Use `pyproject.toml` as the single source of truth for: + +- Dependencies +- Tool configuration + +Example: + +```toml +[project] +name = "doc-tools" +version = "0.1.0" +requires-python = ">=3.10" +dependencies = ["pyyaml", "pypandoc"] + +[tool.black] +line-length = 88 +``` + +## Command-Line Interfaces + +Scripts with arguments must use `argparse`. + +Example: + +```python +parser = argparse.ArgumentParser() +parser.add_argument("--input", required=True) +``` + +Avoid reading `sys.argv` directly. + +## Version Compatibility + +Target Python 3.10+ unless stated otherwise. + +## Appendix A – Suggested Logging Approach (Coloured Logging Framework) + +Supporting tools should prefer logging over print. + +Central module example: + +```python +import logging + +class ColoredFormatter(logging.Formatter): + GREY = "\x1b[38;20m" + CYAN = "\x1b[36;20m" + GREEN = "\x1b[32;20m" + YELLOW = "\x1b[33;20m" + RED = "\x1b[31;20m" + BOLD_RED = "\x1b[31;1m" + RESET = "\x1b[0m" + + FORMAT = "[%(asctime)s] %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s" + + FORMATS = { + logging.DEBUG: CYAN + FORMAT + RESET, + logging.INFO: GREEN + FORMAT + RESET, + logging.WARNING: YELLOW + FORMAT + RESET, + logging.ERROR: RED + FORMAT + RESET, + logging.CRITICAL: BOLD_RED + FORMAT + RESET, + } + + def format(self, record): + log_fmt = self.FORMATS.get(record.levelno) + formatter = logging.Formatter(log_fmt, datefmt="%H:%M:%S") + return formatter.format(record) + + +def setup_logger(verbose: bool) -> None: + logger = logging.getLogger() + logger.handlers.clear() + logger.propagate = False + + level = logging.DEBUG if verbose else logging.INFO + logger.setLevel(level) + + console = logging.StreamHandler() + console.setLevel(level) + console.setFormatter(ColoredFormatter()) + logger.addHandler(console) +``` + + +## Appendix B – Minimal Script Skeleton + +```python +""" +Example tool. + +Usage: + python example.py --input file.txt +""" + +from pathlib import Path +import argparse +import logging + +from app.logging import setup_logger + +logger = logging.getLogger(__name__) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--input", required=True) + parser.add_argument("--verbose", action="store_true") + args = parser.parse_args() + + setup_logger(args.verbose) + + path = Path(args.input) + if not path.exists(): + raise FileNotFoundError(path) + + logger.info("Processing %s", path) + + +if __name__ == "__main__": + main() +``` + +--- + +## Summary + +Good tooling Python should be: + +- Readable +- Typed +- Logged +- Documented +- Predictable diff --git a/content/standards/software-architecture-principles.md b/content/standards/software-architecture-principles.md new file mode 100644 index 0000000..b78b1a9 --- /dev/null +++ b/content/standards/software-architecture-principles.md @@ -0,0 +1,304 @@ +--- +title: Software architecture principles +--- + +# Software architecture principles + +This guideline defines the 23 foundational software architecture principles that govern system design across embedded firmware, edge appliances, and cloud services. Adhering to these principles ensures that software systems remain maintainable, testable, scalable, and resilient over multi-year operational lifecycles. + +## Overview and Categorisation + +> [!NOTE] Relationship to IoT Device Architecture Principles ([IoT device architecture principles](iot-device-architecture-principles.md)) +> [Software architecture principles](software-architecture-principles.md) defines foundational, domain-agnostic software architecture principles (SOLID, package coupling, dependency inversion, defensiveness) applicable to all software components. When designing firmware or physical hardware devices operating in constrained, hostile, or regulated environments, these software principles are specialised and extended by [IoT device architecture principles](iot-device-architecture-principles.md), which governs whole-device concerns including functional safety, hardware isolation, power budgets, secure boot, and cryptographic roots of trust. + +Architectural principles operate at a higher level of abstraction than individual coding idioms. They establish governing boundaries for subsystem decomposition, dependency management, package cohesion, and run-time failure handling. + +The 23 principles are organised into four architectural categories: + +| Category | Principles Included | Focus Area | +| :--- | :--- | :--- | +| **Component & Dependency Design** | DIP, SoC, IoC, DI, LoD, Encapsulation | Decoupling abstractions from concrete implementations and isolating volatile logic. | +| **SOLID & Class Design** | SRP, OCP, LSP, ISP | Ensuring cohesion, extensibility, and substitutability across object and interface boundaries. | +| **Package & Modular Architecture** | ADP, SDP, SAP, CCP, CRP, REP | Governing package composition, versioning stability, and release boundaries. | +| **Pragmatic Engineering & Reliability** | DRY, KISS, YAGNI, POLA, CQS/CQRS, Fail Fast, Defensive Programming | Preventing premature complexity, ensuring predictable behaviour, and trapping faults early. | + +[[CAPTION:Table]] Categorisation of the 23 fundamental software architecture principles. + +```mermaid +flowchart TD + subgraph Reliability["Pragmatic Engineering & Reliability"] + FAIL[Fail Fast] + DEF[Defensive Programming] + KISS[KISS / YAGNI] + DRY[DRY] + end + subgraph Component["Component & Dependency Design"] + DIP[Dependency Inversion] + SOC[Separation of Concerns] + IOC[IoC & DI] + ENC[Encapsulation] + end + subgraph Package["Package Architecture"] + ADP[Acyclic Dependencies] + SDP[Stable Dependencies] + SAP[Stable Abstractions] + REL[Package Release Rules: CCP, CRP, REP] + end + Component --> Package + Package --> Reliability +``` +[[CAPTION:Figure]] Interplay between architectural principle categories. + +## 1. Component and Dependency Design Principles + +### 1.1 Dependency Inversion Principle (DIP) + +High-level policy modules MUST NOT depend upon low-level detail modules; both MUST depend upon abstractions. Abstractions MUST NOT depend upon details; details MUST depend upon abstractions. + +In embedded firmware and IoT architectures, concrete hardware drivers (for example, a specific I2C temperature sensor or SPI flash chip) frequently change across board revisions. The core application logic must depend solely on an abstract sensor or storage interface. + +```c +/* Low-level sensor abstraction */ +typedef struct { + int (*read_temperature)(void *context, float *temp_celsius); + int (*reset)(void *context); +} sensor_interface_t; + +/* High-level telemetry service depends on the abstraction, not concrete hardware */ +typedef struct { + const sensor_interface_t *sensor_api; + void *sensor_context; +} telemetry_service_t; + +int telemetry_service_sample(telemetry_service_t *svc, float *result) { + if (svc == NULL || svc->sensor_api == NULL) { + return -1; + } + return svc->sensor_api->read_temperature(svc->sensor_context, result); +} +``` +[[CAPTION:Figure]] Dependency Inversion in C via function pointer interface structures. + +### 1.2 Separation of Concerns (SoC) + +A software system MUST be separated into distinct sections such that each section addresses a separate concern. Common partitions separate business logic, user interface, network transport protocols, and persistent data storage. + +Separation of Concerns is the foundational driver behind Layered Architecture, Clean Architecture, and Hexagonal Architecture (Ports and Adapters). Modifying the underlying persistence mechanism (for example, swapping SQLite for Flash Data Storage) must never require modifying business calculation logic. + +### 1.3 Inversion of Control (IoC) + +Inversion of Control shifts the responsibility of execution flow, component lifecycle management, and control dispatch from custom application code to an underlying framework or event loop. + +In traditional procedural code, application logic directly creates and calls helper routines. Under IoC, the framework executes the overall lifecycle and invokes registered application callbacks when specific lifecycle events occur (for example, RTOS task schedulers, GUI event dispatchers, or asynchronous event loops). + +### 1.4 Dependency Injection (DI) + +Dependencies MUST be supplied (injected) into a component at instantiation or configuration time rather than instantiated internally by the component itself. + +Dependency injection facilitates unit testing through mock or stub implementations without requiring conditional `#ifdef` compilation or runtime global state manipulation. + +```python +from typing import Protocol + +class TelemetryPublisher(Protocol): + def publish(self, topic: str, payload: bytes) -> bool: ... + +class DeviceMonitor: + def __init__(self, publisher: TelemetryPublisher) -> None: + self._publisher = publisher # Injected dependency + + def report_status(self, device_id: str, battery_mv: int) -> bool: + message = f'{{"device": "{device_id}", "battery": {battery_mv}}}'.encode('utf-8') + return self._publisher.publish(f"devices/{device_id}/status", message) +``` +[[CAPTION:Figure]] Dependency Injection in Python using Protocol structural typing. + +### 1.5 Law of Demeter (LoD / Principle of Least Knowledge) + +A unit or method SHOULD only have knowledge of its immediate collaborators and MUST NOT traverse structural navigation chains (`a.getB().getC().doAction()`). + +Violating the Law of Demeter tightly couples calling code to internal structural layouts. A change in intermediate object hierarchies ripples across distant callers. Components must expose direct intent-revealing operations rather than exposing nested data structures. + +### 1.6 Encapsulation and Information Hiding + +Modules MUST hide their internal data representations and implementation algorithms behind stable, public functional interfaces. + +Exposing raw internal structs or state variables forces client code to manage invariance and concurrency. In C firmware, opaque pointer idioms (`typedef struct sensor_context sensor_context_t;` declared in header, defined only in `.c`) ensure internal state cannot be manipulated directly by external modules. + +## 2. SOLID Class and Interface Design Principles + +### 2.1 Single Responsibility Principle (SRP) + +A class, module, or component MUST have one, and only one, reason to change. + +Every software component must encapsulate a single cohesive aspect of system functionality. If a module formats telemetry payloads, manages flash writing, and establishes LTE connections, it possesses three reasons to change. It must be refactored into three independent modules. + +### 2.2 Open-Closed Principle (OCP) + +Software entities (classes, modules, functions) MUST be open for extension, but closed for modification. + +System capabilities should be extensible by adding new code (such as new plugin implementations, protocol codecs, or strategy objects) without modifying tested, existing source code. + +### 2.3 Liskov Substitution Principle (LSP) + +Subtypes MUST be substitutable for their base types without altering the correctness or behavioural expectations of the programme. + +If an interface promises that `read()` is non-blocking, a derived implementation must not execute a blocking network request. Preconditions must not be strengthened in a subtype, and postconditions must not be weakened. + +### 2.4 Interface Segregation Principle (ISP) + +Clients MUST NOT be forced to depend on methods or interfaces that they do not use. + +Large, monolithic interfaces ("fat interfaces") must be decomposed into smaller, role-specific interfaces. For instance, splitting a combined `DeviceController` into `ReadableSensor`, `ActuatorTarget`, and `DiagnosticProbe` prevents read-only modules from depending on configuration or power-down methods. + +## 3. Package and Modular Architecture Principles + +When grouping classes and functions into deployable packages, libraries, or RTOS static modules, the following package design principles apply: + +### 3.1 Acyclic Dependencies Principle (ADP) + +The dependency graph between packages or architectural subsystems MUST NOT contain cycles. + +Circular dependencies (`Package A -> Package B -> Package C -> Package A`) create tight coupling that forces all components in the cycle to be compiled, tested, and released simultaneously. Cycles must be broken by introducing an intermediate abstraction interface or extracting shared types into a new, lower-level package. + +```mermaid +flowchart LR + subgraph Cycle["Prohibited: Circular Dependency"] + A1[Sensor Manager] --> B1[Network Stack] + B1 --> C1[System Diagnostic] + C1 --> A1 + end + subgraph Acyclic["Compliant: Inverted Dependency"] + A2[Sensor Manager] --> B2[Network Stack] + A2 --> D2[Diagnostic Interface] + C2[System Diagnostic] --> D2 + B2 --> D2 + end +``` +[[CAPTION:Figure]] Breaking circular package dependencies with intermediate interface modules. + +### 3.2 Stable Dependencies Principle (SDP) + +Dependencies MUST point in the direction of stability: a package SHOULD depend only on packages that are more stable than itself. + +Stability in software architecture is defined by the effort required to make a change. A package with many incoming dependencies is highly stable because altering it requires updating all dependants. Highly volatile packages (such as UI or experimental drivers) must depend on stable packages (core protocols and mathematical models). + +### 3.3 Stable Abstractions Principle (SAP) + +A package SHOULD be as abstract as it is stable. + +Stable packages must consist largely of interfaces and abstract definitions so that their behaviour can be extended without modifying their source code. Volatile packages should contain concrete implementations that can be modified rapidly without impacting external consumers. + +### 3.4 Common Closure Principle (CCP) + +Classes or files that change together for the same business or technical reason MUST be packaged together into the same component. + +The Common Closure Principle is the package-level equivalent of the Single Responsibility Principle. When requirements change, the modifications should be confined to a minimal number of packages. + +### 3.5 Common Reuse Principle (CRP) + +Classes or modules that are not used together SHOULD NOT be packaged together. + +Packages must not force dependants to take dependencies on code they do not need. If a library bundles a Bluetooth stack with an unrelated FAT32 file system, clients using only the file system are burdened with Bluetooth code, security exposure, and memory overhead. + +### 3.6 Reuse/Release Equivalence Principle (REP) + +The granule of reuse is the granule of release. Anything that is reused across team boundaries MUST be tracked, versioned, and released through formal version control. + +Shared code cannot be managed by ad-hoc file copies or unversioned subdirectories. Shared libraries and modules must maintain explicit Semantic Versioning (`MAJOR.MINOR.PATCH`) to ensure predictable integration. + +## 4. Pragmatic Engineering and Reliability Principles + +### 4.1 Don't Repeat Yourself (DRY) + +Every piece of knowledge or business logic MUST have a single, unambiguous, authoritative representation within a system. + +Duplication of logic (such as copying packet parsing routines across microservices or firmware modules) inevitably leads to divergence, where bug fixes applied to one copy fail to reach others. DRY applies to operational knowledge, configuration schemas, and protocol definitions, not merely identical lines of syntax. + +### 4.2 Keep It Simple (KISS) + +Systems perform best when kept simple rather than made complex; simplicity MUST be a key design goal and unnecessary complexity avoided. + +Architectures must prioritise direct, understandable solutions over elaborate abstraction layers that provide no current functional value. The simplest viable architecture is the easiest to verify, maintain, and debug. + +### 4.3 You Aren't Gonna Need It (YAGNI) + +Software components and capabilities MUST NOT be implemented until they are actually required by verified requirements. + +Designing for speculative future requirements ("what if we switch to a distributed blockchain datastore later?") wastes development time. It inflates code size and introduces bugs into capabilities that may never be deployed. + +### 4.4 Principle of Least Astonishment (POLA) + +Component interfaces, function signatures, and system behaviours MUST behave in a manner that least surprises the user or consuming developer. + +Functions must not produce unexpected side effects. A method named `calculate_checksum()` must not silently mutate packet headers or write to flash storage. Naming conventions, return codes, and error modes must remain strictly consistent across the entire codebase. + +### 4.5 Command Query Separation (CQS) and CQRS + +Methods that modify state (Commands) SHOULD NOT return data, and methods that return data (Queries) MUST NOT modify observable state. + +At the micro level (CQS), separating mutations from state queries simplifies reasoning and concurrency control. At the system level (Command Query Responsibility Segregation, CQRS), reads and writes go through separate data models. Telemetry ingestion and analytical queries can then scale, and be stored, independently. + +### 4.6 Fail Fast + +Systems and software routines MUST detect and report operational failures as close as possible to the point of origin, terminating or rejecting the transaction immediately. + +Failing fast prevents silent data corruption. When a parameter validation fails or a communication packet fails a CRC check, the routine must return an explicit error immediately rather than continuing execution with corrupted internal state. + +```c +int lwm2m_codec_decode_tlv(const uint8_t *buf, size_t len, lwm2m_tlv_t *out) { + /* Fail Fast: validate parameters immediately */ + if (buf == NULL || out == NULL || len == 0) { + return -EINVAL; + } + if (len < TLV_MIN_HEADER_SIZE) { + return -EBADMSG; + } + /* Proceed with decoding only when inputs are verified */ + return parse_tlv_header(buf, len, out); +} +``` + +[[CAPTION:Figure]] Fail Fast parameter and buffer validation in C. + +### 4.7 Defensive Programming + +Components MUST defend their operational integrity against unexpected inputs, network corruption, hardware faults, and downstream service failures. + +Defensive programming mandates: + +- Strict boundary validation of all external inputs and deserialised payloads. +- Range checking on array indices, pointers, and numeric calculations to prevent buffer overflows and integer wrap-around. +- Robust timeout handling on all inter-process and network socket communications. +- Watchdog timer refresh only after successful execution of critical task loops. + +## Summary Checklist + +| Principle | Core Question for Architecture Reviews | +| :--- | :--- | +| **DIP** | Do high-level business policies depend exclusively on abstract interfaces? | +| **SoC** | Are business logic, persistence, and communications partitioned into separate layers? | +| **IoC / DI** | Are dependencies injected into modules rather than hardcoded internally? | +| **LoD** | Are objects interacting only with their immediate collaborators? | +| **SRP** | Does each module or subsystem have only a single reason to change? | +| **OCP** | Can new capabilities be added by adding code rather than modifying core logic? | +| **LSP** | Can all interface implementations be substituted without breaking contracts? | +| **ISP** | Are client interfaces small, focused, and free of unused operations? | +| **ADP** | Is the package dependency graph strictly acyclic? | +| **SDP / SAP** | Do dependencies point toward stability, and are stable packages abstract? | +| **CCP / CRP** | Do packaged classes change together and are they reused together? | +| **DRY** | Is every business rule and protocol schema defined in exactly one place? | +| **KISS / YAGNI** | Is the design free of speculative complexity and unneeded abstraction? | +| **Fail Fast** | Are invalid arguments and corrupted frames rejected immediately at boundaries? | +| **Defensive** | Are buffers bounded, timeouts enforced, and error conditions handled? | + +[[CAPTION:Table]] Summary verification matrix for the 23 fundamental architecture principles. + +## Related Documents + +- [Design Checklist Guide](../templates/design-review-checklist.md) +- [ADR Authoring Guide](../guides/decision-records.md) +- [Software Detailed Specification Standard](../guides/software-design-specification.md) +- [Architecture Review Board (ARB) Guide](../guides/architecture-review-board.md) +- [Architecture Review Checklist](../templates/architecture-review-checklist.md) +- [API Design and Error Management Standards](api-design.md) diff --git a/content/standards/unique-identifiers.md b/content/standards/unique-identifiers.md new file mode 100644 index 0000000..1cd6f6b --- /dev/null +++ b/content/standards/unique-identifiers.md @@ -0,0 +1,185 @@ +--- +title: Unique identifiers +--- + +# Unique identifiers + +This guideline establishes the engineering standard and architectural decision framework for generating, storing, and indexing unique identifiers across distributed systems, relational databases, event streams, and constrained IoT edge devices. + +## 1. Context and Problem Space + +Selecting an identifier scheme is one of the most critical structural decisions in software and data architecture. Primary keys and entity identifiers impact storage efficiency, database index performance, cache locality, horizontal scalability, and information security. + +```mermaid +flowchart TD + A["Unique Identifier Strategies"] --> B["Opaque Identifiers"] + A --> C["Time-Ordered / Monotonic Identifiers"] + A --> D["Database Sequences"] + B --> E["UUID v4 (Random)"] + B --> F["NanoID (URL-safe)"] + C --> G["UUID v7 (RFC 9562)"] + C --> H["ULID (Crockford Base32)"] + C --> I["Snowflake (64-bit Bitfield)"] + D --> J["Auto-Increment BigInt"] +``` + +[[CAPTION:Figure]] Taxonomy of unique identifier schemes. + +### 1.1 The Pitfalls of Centralised Database Sequences + +Relational database autoincrementing sequences (such as PostgreSQL `BIGSERIAL` or MySQL `AUTO_INCREMENT`) provide sequential integers that are computationally cheap to index. However, they introduce severe architectural liabilities in modern distributed architectures: + +- **Single Point of Failure and Contention**: A centralised database must coordinate every sequence increment, generating latch contention under high write throughput. +- **Inability to Generate Offline or at the Edge**: Constrained edge gateways and mobile clients cannot allocate IDs without an active network roundtrip to the primary database. +- **Predictability and Security Exploitation**: Sequential IDs expose business intelligence (for example, competitors can deduce daily order volume by subtracting consecutive order numbers) and invite Insecure Direct Object Reference (IDOR) traversal attacks. + +## 2. Comparative Evaluation of Identifier Implementations + +Architects must evaluate candidate identifier schemes across six primary architectural vectors: + +- **Bit Length and Storage Footprint**: Memory overhead in RAM caches and on-disk tables. +- **Lexicographical Sortability**: Ability to insert sequentially without random disk I/O. +- **B-Tree Index Fragmentation**: Mitigating leaf page splits and random page rewrites. +- **Collision Resistance**: Mathematical entropy under high concurrent generation rates. +- **URL and Human Friendliness**: Readability in debug logs and browser query parameters. +- **Information Leakage**: Risk of exposing MAC addresses, internal IP addresses, or creation timestamps. + +| Identifier Type | Size (Bits / Bytes) | String Format | Monotonic / Sortable | Collision Entropy | Primary Use Case | +| :--- | :--- | :--- | :--- | :--- | :--- | +| **Database Sequence** | 64 bits (8 B) | Integer string | Fully sequential | Zero (coordinated) | Legacy single-instance OLTP or OLAP data warehouses. | +| **UUID v1** | 128 bits (16 B) | 36-char hex hyphenated | Partially (60-bit timestamp) | Low (MAC + clock seq) | Deprecated; leaks device hardware MAC address. | +| **UUID v4** | 128 bits (16 B) | 36-char hex hyphenated | None (fully random) | High (122 bits entropy) | Cryptographic nonces, ephemeral tokens, low-write tables. | +| **UUID v7 (RFC 9562)** | 128 bits (16 B) | 36-char hex hyphenated | Millisecond sortable | High (74 bits entropy) | **Standard recommended default** for modern RDBMS/NoSQL primary keys. | +| **ULID** | 128 bits (16 B) | 26-char Crockford Base32 | Millisecond sortable | High (80 bits entropy) | Distributed systems desiring compact, case-insensitive string encoding. | +| **Snowflake (Twitter)** | 64 bits (8 B) | Integer or hex string | Millisecond sortable | Moderate (12-bit sequence) | Ultra-high throughput microservices requiring compact 64-bit integer storage. | +| **NanoID** | 168 bits (21 chars) | 21-char base64 URL-safe | None (random) | High (configurable) | Client-facing public resource URLs and frontend application IDs. | + +[[CAPTION:Table]] Comprehensive identifier evaluation matrix across architectural dimensions. + +## 3. Database B-Tree Index Fragmentation and Storage Impact + +The interaction between identifier randomness and database indexing structures dictates write scalability. + +### 3.1 The Random UUID v4 Write Amplification Hazard + +Relational databases (PostgreSQL, MySQL InnoDB, SQLite) store primary keys within B+Tree index structures. When rows are inserted with random keys such as UUID v4: + +- Inserts hit random leaf pages across the tree. +- When an index page fills, the database engine must execute an expensive **page split**, allocating a new disk block and rewriting adjacent nodes. +- Over time, table and index fragmentation climbs above $40\%$, dramatically enlarging the database working set and degrading buffer pool cache hit rates from $99\%$ down to under $60\%$. + +```mermaid +flowchart TD + subgraph Random_UUIDv4["Random UUID v4 Inserts"] + R1["ID: f47ac10b..."] --> P3["Leaf Page 3 (Full -> Page Split)"] + R2["ID: 1ae24f5c..."] --> P1["Leaf Page 1"] + R3["ID: c9a0f8b1..."] --> P2["Leaf Page 2 (Full -> Page Split)"] + end + subgraph Monotonic_UUIDv7["Monotonic UUID v7 Inserts"] + M1["ID: 018e3f4a-0001..."] --> MP1["Right-most Leaf Page"] + M2["ID: 018e3f4a-0002..."] --> MP1 + M3["ID: 018e3f4a-0003..."] --> MP2["Append New Page (Zero Splits)"] + end +``` + +[[CAPTION:Figure]] Comparison of B+Tree insertion dynamics: random UUID v4 page fragmentation versus monotonic UUID v7 append behaviour. + +### 3.2 Monotonic UUID v7: The Optimal Modern Architecture + +Standardised under IETF RFC 9562, **UUID v7** resolves the tension between distributed uniqueness and index efficiency: + +- **48-bit Unix timestamp**: Encodes milliseconds since epoch in big-endian order at the most significant bits. +- **4-bit Version**: Fixed to `0111` (binary 7). +- **12-bit Sub-millisecond or Counter**: Supports high-resolution ordering. +- **2-bit Variant**: Fixed to `10` (RFC 4122 / 9562 compliant). +- **62-bit Cryptographic Randomness**: Guarantees global collision avoidance across independent nodes. + +The most significant bits increase with time, so database inserts behave like sequential appends at the right-hand edge of the B+Tree. This keeps index density above $90\%$ and prevents premature page splits. + +## 4. Implementation Guidelines by Workload + +Architects and engineering teams must adhere to the following selection rules: + +### 4.1 Workload Decision Rules + +- **Standard Distributed Microservices and Databases**: Implement **UUID v7**. Store in the database natively as binary `UUID` (16 bytes) rather than string `VARCHAR(36)` to save $55\%$ index storage. +- **Ultra-High Throughput Distributed Streams (Kafka / In-Memory Systems)**: Implement **Snowflake / Sonyflake** when storage footprint must remain within 64-bit integer boundaries ($8\mathrm{bytes}$). +- **Constrained Edge IoT Devices**: Constrained microcontrollers (FreeRTOS, Zephyr) should generate UUID v7 using hardware true random number generators (TRNG) paired with RTC millisecond timestamps. If RTC is unavailable, devices must transmit ephemeral local sequence IDs that edge gateways promote to UUID v7 upon ingestion. +- **Public-Facing Web URLs and Customer References**: Use **NanoID** or **UUID v7** with Base62 encoding to generate short, URL-safe, non-enumerable slugs. + +### 4.2 Reference Generation Examples + +#### Python 3 Reference (UUID v7 via standard / fast library) + +```python +import uuid +import time + +def generate_uuid7() -> uuid.UUID: + """Generate RFC 9562 UUIDv7 with millisecond timestamp and random entropy.""" + # Obtain current millisecond timestamp + millis = int(time.time() * 1000) + + # 48 bits of timestamp + time_high = (millis >> 16) & 0xFFFFFFFF + time_mid = millis & 0xFFFF + + # Generate cryptographic random bytes for remaining entropy + import secrets + rand_bytes = secrets.token_bytes(10) + + # Construct 16-byte raw UUID with version 7 (0x70) and variant 1 (0x80) + raw = bytearray(16) + raw[0:4] = time_high.to_bytes(4, byteorder="big") + raw[4:6] = time_mid.to_bytes(2, byteorder="big") + raw[6] = 0x70 | (rand_bytes[0] & 0x0F) + raw[7] = rand_bytes[1] + raw[8] = 0x80 | (rand_bytes[2] & 0x3F) + raw[9:16] = rand_bytes[3:10] + + return uuid.UUID(bytes=bytes(raw)) +``` + +[[CAPTION:Figure]] Python reference implementation for RFC 9562 UUID v7 generation. + +#### C Reference for Constrained Embedded Systems (FreeRTOS / POSIX) + +```c +#include +#include + +typedef struct { + uint8_t bytes[16]; +} uuid7_t; + +/** + * Generates an RFC 9562 UUIDv7 in place on an embedded microcontroller. + * Requires a millisecond tick source and hardware TRNG. + */ +int generate_uuid7_embedded(uuid7_t *out_uuid, uint64_t epoch_ms, int (*get_trng)(uint8_t *buf, size_t len)) { + if (!out_uuid || !get_trng) { + return -1; + } + + /* Step 1: Write 48-bit big-endian millisecond timestamp */ + out_uuid->bytes[0] = (uint8_t)((epoch_ms >> 40) & 0xFF); + out_uuid->bytes[1] = (uint8_t)((epoch_ms >> 32) & 0xFF); + out_uuid->bytes[2] = (uint8_t)((epoch_ms >> 24) & 0xFF); + out_uuid->bytes[3] = (uint8_t)((epoch_ms >> 16) & 0xFF); + out_uuid->bytes[4] = (uint8_t)((epoch_ms >> 8) & 0xFF); + out_uuid->bytes[5] = (uint8_t)(epoch_ms & 0xFF); + + /* Step 2: Fill remaining 10 bytes with hardware randomness */ + if (get_trng(&out_uuid->bytes[6], 10) != 0) { + return -2; /* TRNG fault */ + } + + /* Step 3: Set RFC 9562 Version 7 and Variant 1 bits */ + out_uuid->bytes[6] = 0x70 | (out_uuid->bytes[6] & 0x0F); + out_uuid->bytes[8] = 0x80 | (out_uuid->bytes[8] & 0x3F); + + return 0; +} +``` + +[[CAPTION:Figure]] MISRA C compliant embedded UUID v7 generator for microcontrollers. diff --git a/content/templates/agents-md.md b/content/templates/agents-md.md new file mode 100644 index 0000000..4392280 --- /dev/null +++ b/content/templates/agents-md.md @@ -0,0 +1,64 @@ +--- +title: AGENTS.md for AI agents +--- + +# AGENTS.md for AI agents + +Put this file at the root of a repository that AI coding agents work in. It tells them the commands to run, the conventions to follow and what they must not touch, and points them at the [agent rules](../guidelines/agent-rules.md). For Claude Code, add a `CLAUDE.md` that contains `@AGENTS.md`, so both tools read the same instructions. + +## Template + +````markdown +# AGENTS.md + +Instructions for AI coding agents working in this repository. Read this file first. The full guidelines are in `docs/guidelines/` and a condensed rule list is in `docs/guidelines/agent-rules.md`. + +## Project + +One paragraph: what this repository is, the target hardware or platform, and the main language. + +## Commands + +- Build: `` +- Unit tests: `` +- Lint and format: `` +- Static analysis: `` +- Everything CI runs: `` +- Docs: `` + +## Conventions + +- Commits: Conventional Commits, with `Refs:` for requirement or issue IDs and `Assisted-by:` for AI assistance +- Branches: short-lived `feat/`, `fix/`, `docs/` and `chore/`. Never push to `main` +- Requirements: `docs/requirements/`. Cite IDs as `REQ-...` +- Decisions: `docs/adr/`. Propose a new ADR for significant or hard-to-reverse decisions, and never edit accepted ones +- Principles: CP-02 simplicity, CP-16 surgical changes, CP-20 surface assumptions. When a rule is silent, apply `docs/guidelines/principles-catalogue.md` +- Rules marked *(core)* cannot be tailored. Other rules follow the product conformance profile in `docs/conformance-profile.md` + +## Boundaries + +Do not modify any of the following without explicit instruction in the task: guidelines, `AGENTS.md`, pipeline definitions, CODEOWNERS, secrets and signing material, generated or vendored code, security-relevant code, and shared history. Never disable, skip or weaken a test or check. + +Never put secrets or personal data in code, logs, commits or prompts. + +## Working method + +1. State assumptions and open questions before starting. Ask if a requirement is missing or ambiguous. +1. Make the smallest change that meets the requirement. Report unrelated problems and do not fix them in passing. +1. Add or update tests with the change. Reproduce a bug with a failing test before fixing it. +1. Run the full local check command and include the results in the PR. +1. Update documentation in the same change. +1. Open a PR with a Conventional Commit title and the filled template. Never approve or merge your own work. + +## Definition of done + +Builds cleanly, tests pass, lint and analysis are clean, docs are updated, the PR template is completed, and commits carry `Refs:` and `Assisted-by:`. + +## Where things are + +- Guidelines index: `docs/guidelines/README.md` +- Agent rules (condensed): `docs/guidelines/agent-rules.md` +- Principles: `docs/guidelines/principles-catalogue.md` +- Component register: `docs/components.yaml` (where the product has one) +- Templates: `docs/templates-reference/` +```` diff --git a/content/templates/architecture-principles.md b/content/templates/architecture-principles.md new file mode 100644 index 0000000..f82599a --- /dev/null +++ b/content/templates/architecture-principles.md @@ -0,0 +1,109 @@ +--- +title: Architecture principles +--- + +# Architecture principles + +Use this template to record the architecture principles a product or team has agreed. Each principle needs a rationale and its implications, or it is a slogan rather than a principle. + +## Template + +```markdown +--- +title: Architecture Principles - [System/Product/Organization Name] +status: Draft / Reviewed / Approved / Baselined / Evolving +date: YYYY-MM-DD +author: [Name or Role of Lead Architect] +version: 1.0 +related_documents: + - Architecture Vision - [System/Product Name] +--- + +# Architecture Principles: [System/Product/Organization Name] + +This document defines the core architectural principles that guide the design and evolution of the [System/Product/Organization Name]'s architecture. These principles serve as a shared understanding and decision-making framework for architects, developers, and other stakeholders, ensuring consistency, quality, and alignment with our strategic goals. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this document is to establish a set of clear and actionable architectural principles that: +* Provide a foundation for architectural decisions. +* Promote consistency and coherence across different parts of the system. +* Guide technology selection and design patterns. +* Facilitate communication and alignment among teams. +* Support the long-term evolvability and sustainability of the architecture. + +### 1.2 Scope + +These principles apply to the design and development of all software, firmware, and relevant hardware components within the [System/Product/Organization Name]. They are intended to be high-level guiding statements, not detailed design rules. + +### 1.3 Definitions and Acronyms + +[List any architecture-specific terms, definitions, or acronyms used in this document.] + +## 2. Core Architectural Principles + +For each principle, provide a clear statement, its rationale (why it's important), and its implications (how it affects decisions). + +### 2.1 Principle 1: [Concise Principle Name] + +* **Statement:** [Clear, concise statement of the principle. e.g., "Design for Change and Evolution."] +* **Rationale:** [Why is this principle important? What problem does it solve or what value does it bring? e.g., "Software systems operate in dynamic environments; anticipating and accommodating change reduces future cost and risk."] +* **Implications:** [How does this principle affect architectural and design decisions? e.g., "Favor modular, loosely coupled designs. Use evolutionary architecture patterns. Prioritize clear interfaces over tight coupling."] + +### 2.2 Principle 2: [Concise Principle Name] + +* **Statement:** [e.g., "Prioritize Operational Excellence."] +* **Rationale:** [e.g., "A system's value is limited by its ability to be reliably operated, monitored, and maintained in production."] +* **Implications:** [e.g., "Embed observability (logging, metrics, tracing) from the outset. Design for automated deployment and recovery. Implement robust alerting and incident response mechanisms."] + +### 2.3 Principle 3: [Concise Principle Name] + +* **Statement:** [e.g., "Embrace Security by Design."] +* **Rationale:** [e.g., "Security must be an inherent property of the system, not an afterthought, to protect data, users, and business reputation."] +* **Implications:** [e.g., "Conduct threat modeling early in the design phase. Implement least privilege access. Encrypt sensitive data at rest and in transit. Automate security testing."] + +### 2.4 Principle 4: [Concise Principle Name] + +* **Statement:** [e.g., "Optimize for Cost-Effectiveness."] +* **Rationale:** [e.g., "Architectural decisions have significant long-term financial implications for both development and operations."] +* **Implications:** [e.g., "Regularly review cloud resource utilization. Choose technologies with appropriate licensing models. Balance performance needs with infrastructure costs."] + +### 2.5 Principle 5: [Concise Principle Name] + +* **Statement:** [e.g., "Promote Simplicity and Clarity."] +* **Rationale:** [e.g., "Simple designs are easier to understand, build, test, and maintain, reducing cognitive load and accelerating delivery."] +* **Implications:** [e.g., "Avoid unnecessary complexity or over-engineering. Favor well-understood patterns. Document design decisions clearly and concisely."] + +### 2.6 Principle 6: [Concise Principle Name] + +* **Statement:** [e.g., "Ensure Scalability and Performance."] +* **Rationale:** [e.g., "The system must be able to handle anticipated growth in users, data, and load while maintaining acceptable performance levels."] +* **Implications:** [e.g., "Design for horizontal scaling. Identify and optimize performance bottlenecks. Utilize appropriate caching strategies."] + +### 2.7 Principle 7: [Concise Principle Name] + +* **Statement:** [e.g., "Foster Reusability and Standardization."] +* **Rationale:** [e.g., "Leveraging existing components and adhering to standards reduces duplication, improves consistency, and accelerates development."] +* **Implications:** [e.g., "Identify common services and components for reuse. Define and adhere to coding and architectural standards. Utilize shared libraries and platforms."] + +## 3. Applying and Maintaining Principles + +### 3.1 Application + +* These principles should be considered during all architectural design discussions and decision-making processes. +* They serve as criteria for evaluating architectural alternatives. +* Deviations from principles should be explicitly justified and documented (e.g., in an ADR). + +### 3.2 Review and Evolution + +* These principles are living guidelines and should be reviewed periodically (e.g., annually or as part of major strategic shifts) to ensure their continued relevance and effectiveness. +* Feedback from development teams and stakeholders is crucial for their evolution. + +## 4. References + +[List any other documents (e.g., Architecture Vision, ADRs, relevant standards) referenced in this document.] +``` diff --git a/content/templates/architecture-vision.md b/content/templates/architecture-vision.md new file mode 100644 index 0000000..01e00d2 --- /dev/null +++ b/content/templates/architecture-vision.md @@ -0,0 +1,133 @@ +--- +title: Architecture vision +--- + +# Architecture vision + +Use this template to describe where a system's architecture is heading and why, before the detailed design starts. It links the business goals to the main technical choices, so later decisions can be checked against it. + +## Template + +```markdown +--- +title: Architecture Vision - [System/Product Name] +status: Draft / Reviewed / Approved / Baselined / Evolving +date: YYYY-MM-DD +author: [Name or Role of Lead Architect] +version: 1.0 +related_documents: + - Product Vision - [Product Name] + - ADR-001 - [Key Architectural Decision] +--- + +# Architecture Vision: [System/Product Name] + +This document defines the overarching architectural vision for the [System/Product Name]. It describes the desired future state of the system's architecture, its strategic alignment, and the high-level goals it aims to achieve. This vision serves as a guiding star for all architectural and design decisions, ensuring consistency and alignment with business objectives. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Architecture Vision document is to: +* Articulate a clear and inspiring long-term direction for the system's architecture. +* Align all stakeholders (business, product, development, operations) on the architectural strategy. +* Provide a high-level blueprint for subsequent detailed design and implementation efforts. +* Ensure the architecture supports current and future business needs and product capabilities. + +### 1.2 Scope + +This document covers the architectural vision for the [System/Product Name]. It focuses on the strategic direction, key drivers, and high-level approaches, rather than detailed design specifications or specific architectural principles (which are covered in a separate document). + +### 1.3 Definitions and Acronyms + +[List any architecture-specific terms, definitions, or acronyms used in this document.] + +## 2. Architecture Vision Statement + +A concise, inspiring statement that defines the desired future state of the architecture. It should convey the essence of what the architecture will enable and how it will contribute to the overall success. + +*Example: To build a highly scalable, resilient, and adaptable cloud-native architecture that enables rapid feature delivery, global reach, and continuous innovation for our [Product Name] platform.* + +## 3. Strategic Alignment + +How does this architecture vision directly support and enable the overarching business strategy and product vision? What core business problems does it help solve, or what opportunities does it unlock? + +* [Reference relevant business objectives, OKRs, or the Product Vision.] +* [Explain how the architecture provides a competitive advantage or addresses market needs.] + +## 4. Key Architectural Goals & Objectives + +What are the specific, measurable goals that this architecture aims to achieve? These should be outcomes that the architecture enables. + +* **[Goal 1]:** [e.g., "Achieve 99.99% availability for core services."] +* **[Goal 2]:** [e.g., "Reduce average deployment time for new features by 50%."] +* **[Goal 3]:** [e.g., "Support a 10x increase in user concurrency within 3 years."] +* **[Goal 4]:** [e.g., "Ensure compliance with [specific regulation, e.g., GDPR, ISO 27001] by design."] +* **[Goal 5]:** [e.g., "Minimize operational cost per user by 20% over 2 years."] + +## 5. Key Architectural Drivers & Concerns + +What are the primary factors that influence and shape this architecture? These are the forces that the architecture must address. + +* **Performance:** [e.g., low latency requirements, high transaction throughput.] +* **Scalability:** [e.g., anticipated growth in users/data, elastic demand.] +* **Security:** [e.g., data privacy, threat landscape, compliance mandates.] +* **Reliability/Resilience:** [e.g., fault tolerance, disaster recovery, uptime targets.] +* **Maintainability/Evolvability:** [e.g., ease of change, technical debt management, future-proofing.] +* **Cost:** [e.g., operational expenses, development costs, infrastructure spend.] +* **Time-to-Market:** [e.g., rapid prototyping, continuous delivery needs.] +* **Usability/Developer Experience:** [e.g., ease of use for internal teams, onboarding new developers.] +* **Regulatory/Compliance:** [e.g., industry-specific standards, data residency.] +* **Technology Constraints:** [e.g., existing tech stack, specific vendor requirements.] + +## 6. High-Level Architectural Strategy + +What are the main strategic approaches or patterns that will be adopted to achieve the vision and goals? This is the "how" at a conceptual level. + +* **[Strategy 1]:** [e.g., "Cloud-Native First: Leverage managed cloud services for scalability and reduced operational overhead."] +* **[Strategy 2]:** [e.g., "Microservices Architecture: Decompose functionality into independently deployable services for agility and resilience."] +* **[Strategy 3]:** [e.g., "Event-Driven Architecture: Utilize asynchronous messaging for loose coupling and real-time data flow."] +* **[Strategy 4]:** [e.g., "API-First Design: Expose well-defined APIs for all services to promote interoperability."] +* **[Strategy 5]:** [e.g., "Infrastructure as Code (IaC): Automate infrastructure provisioning and management."] +* **[Strategy 6]:** [e.g., "Security by Design: Embed security controls and practices throughout the development lifecycle."] + +## 7. Key Architectural Decisions (High-Level) + +Summarize or reference any fundamental architectural decisions that have already been made or are critical to this vision. These should ideally link to specific Architecture Decision Records (ADRs). + +* [ADR-001: Choice of Cloud Provider (AWS/Azure/GCP)] +* [ADR-005: Primary Database Technology Selection (e.g., PostgreSQL)] +* [ADR-010: Main Communication Protocol (e.g., gRPC vs. REST)] + +## 8. Non-Goals / Out of Scope + +What is explicitly *not* covered or aimed for by this architecture vision? This helps manage expectations and focus efforts. + +* [e.g., "Detailed component-level design (covered in SDS documents)."] +* [e.g., "Specific implementation details for non-core functionalities."] +* [e.g., "Support for legacy systems beyond defined integration points."] + +## 9. High-Level Roadmap & Evolution + +Provide a conceptual view of how the architecture is expected to evolve over time, aligning with product roadmap phases. + +* **Phase 1 (Foundation):** [e.g., "Establish core cloud infrastructure, deploy foundational services."] +* **Phase 2 (Expansion):** [e.g., "Migrate key business domains, introduce advanced data processing."] +* **Phase 3 (Optimization):** [e.g., "Refine scalability, enhance operational efficiency, explore AI/ML integrations."] + +## 10. Stakeholders + +Who are the key individuals or groups that need to understand, contribute to, or be aligned with this architecture vision? + +* **Executive Leadership:** [For strategic alignment and resource allocation.] +* **Product Management:** [To ensure alignment with product roadmap and features.] +* **Development Leads/Teams:** [For guiding implementation and detailed design.] +* **Operations/DevOps Teams:** [For deployment, monitoring, and operational considerations.] +* **Security Team:** [For security reviews and compliance.] + +## 11. References + +[List any other documents (e.g., Product Vision, Business Strategy, relevant ADRs, C4 Model diagrams) referenced in this Architecture Vision.] +``` diff --git a/content/templates/crosscutting-concerns.md b/content/templates/crosscutting-concerns.md new file mode 100644 index 0000000..ba0360f --- /dev/null +++ b/content/templates/crosscutting-concerns.md @@ -0,0 +1,232 @@ +--- +title: Crosscutting concerns +--- + +# Crosscutting concerns + +Use this template to describe the concerns that cut across every part of a system, such as security, logging, error handling and configuration. Writing them down once stops each component solving them differently. + +## Template + +```markdown +--- +title: Crosscutting Concerns / Architectural Concerns - [System/Product Name] +status: Draft / Reviewed / Approved / Baselined / Evolving +date: YYYY-MM-DD +author: [Name or Role of Lead Architect] +version: 1.0 +related_documents: + - Architecture Vision - [System/Product Name] + - Architecture Principles - [System/Product Name] +--- + +# Crosscutting Concerns / Architectural Concerns: [System/Product Name] + +This document defines the strategies and approaches for addressing key crosscutting concerns within the [System/Product Name]'s architecture. These concerns impact multiple parts of the system and require a consistent, overarching strategy to ensure coherence, quality, and maintainability. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this document is to: +* Identify and define the common architectural concerns that span multiple components or subsystems. +* Document the chosen strategy and approach for addressing each concern. +* Ensure consistency in implementation across the architecture. +* Provide guidance for developers and architects on how to handle these concerns. +* Facilitate understanding and communication of complex architectural aspects. + +### 1.2 Scope + +This document covers the high-level strategy and common implementation patterns for crosscutting concerns across the [System/Product Name]. It does not delve into component-specific implementation details, which will be covered in detailed design specifications (SDS). + +### 1.3 Definitions and Acronyms + +[List any relevant terms, definitions, or acronyms.] + +## 2. Crosscutting Concerns Log + +This log provides a summary of each identified crosscutting concern. + +| Concern ID | Concern Name | Description (Brief) | Strategy (Brief) | Owner | Last Updated | +|------------|--------------|---------------------|------------------|-------|--------------| +| **CC-001** | Security | Protecting system from unauthorized access. | Layered security, least privilege, encryption. | Jane Doe | YYYY-MM-DD | +| **CC-002** | Logging & Monitoring | Capturing and analyzing system events and metrics. | Centralized logging, distributed tracing. | John Smith | YYYY-MM-DD | +| **CC-003** | Error Handling & Resilience | Graceful degradation and recovery from failures. | Circuit breakers, retries, consistent error responses. | Alice Brown | YYYY-MM-DD | +| **CC-004** | Configuration Management | Managing application settings and environment variables. | Externalized, version-controlled configuration. | Bob White | YYYY-MM-DD | +| **CC-005** | Authentication | Verifying user/system identity. | OAuth 2.0, JWT. | Carol Green | YYYY-MM-DD | +| **CC-006** | Authorization | Controlling access to resources. | Role-Based Access Control (RBAC). | Carol Green | YYYY-MM-DD | + +## 3. Detailed Crosscutting Concerns + +For each crosscutting concern, provide a detailed description of the strategy, chosen solution, implications, and implementation guidelines. + +### 3.1 Concern: Security (CC-001) + +#### 3.1.1 Description + +[Briefly describe what "security" means in the context of this system (e.g., protecting data at rest and in transit, preventing unauthorized access, ensuring data integrity).] + +#### 3.1.2 Architectural Strategy / Solution + +[Describe the high-level strategy for addressing security. e.g., "Implement a layered security approach with defense-in-depth principles. Utilize industry-standard encryption protocols. Adopt a 'least privilege' model for all access."] + +* **Key Technologies/Patterns:** [e.g., TLS, OAuth 2.0, JWT, HSMs, Web Application Firewalls (WAFs).] +* **Related ADRs:** [Link to relevant ADRs, e.g., ADR-005: "Choice of Authentication Protocol."] + +#### 3.1.3 Implications + +* **Benefits:** [e.g., "Reduced risk of data breaches," "Compliance with regulatory requirements," "Increased customer trust."] +* **Drawbacks/Trade-offs:** [e.g., "Increased complexity in initial setup," "Potential performance overhead due to encryption."] + +#### 3.1.4 Implementation Guidelines + +[Provide high-level guidelines for how this concern should be implemented across components.] + +* **Data Protection:** All sensitive data must be encrypted at rest (AES-256) and in transit (TLS 1.2+). +* **Access Control:** Implement Role-Based Access Control (RBAC) for all internal and external APIs. +* **Input Validation:** All external inputs must be validated at the service boundary. +* **Vulnerability Management:** Regular security scans and penetration testing are mandatory. + +### 3.2 Concern: Logging & Monitoring (CC-002) + +#### 3.2.1 Description + +[Briefly describe the purpose of logging and monitoring in this system (e.g., for debugging, auditing, performance tracking, operational visibility).] + +#### 3.2.2 Architectural Strategy / Solution + +[Describe the strategy. e.g., "Implement centralized logging and distributed tracing. Standardize log formats and levels. Utilize metrics for real-time operational visibility."] + +* **Key Technologies/Patterns:** [e.g., ELK Stack (Elasticsearch, Logstash, Kibana), Prometheus, Grafana, OpenTelemetry.] +* **Related ADRs:** [e.g., ADR-010: "Centralized Logging Solution Selection."] + +#### 3.2.3 Implications + +* **Benefits:** [e.g., "Faster incident resolution," "Improved system observability," "Better understanding of user behavior."] +* **Drawbacks/Trade-offs:** [e.g., "Storage costs for logs," "Performance overhead if logging is excessive."] + +#### 3.2.4 Implementation Guidelines + +* **Standardized Logging:** Use a common logging library and predefined log levels (DEBUG, INFO, WARN, ERROR, FATAL). +* **Contextual Logging:** Include relevant context (e.g., request ID, user ID, component name) in all logs. +* **Distributed Tracing:** Instrument services to support end-to-end tracing of requests. +* **Key Metrics:** Define and collect essential metrics for each service (e.g., request rates, error rates, latency). + +### 3.3 Concern: Error Handling & Resilience (CC-003) + +#### 3.3.1 Description + +[Describe the approach to handling errors and ensuring the system can withstand and recover from failures.] + +#### 3.3.2 Architectural Strategy / Solution + +[e.g., "Implement consistent error response formats. Utilize resilience patterns like circuit breakers, retries, and bulkheads. Design for graceful degradation."] + +* **Key Technologies/Patterns:** [e.g., Hystrix (or similar), Polly, consistent API error schemas.] +* **Related ADRs:** [e.g., ADR-015: "API Error Response Standard."] + +#### 3.3.3 Implications + +* **Benefits:** [e.g., "Improved system stability and availability," "Better user experience during partial failures," "Faster recovery from outages."] +* **Drawbacks/Trade-offs:** [e.g., "Increased complexity in service interactions," "Requires careful testing of failure scenarios."] + +#### 3.3.4 Implementation Guidelines + +* **Consistent Error Responses:** All APIs/interfaces must return errors in a standardized format. +* **Circuit Breakers:** Implement circuit breakers for calls to external services or unreliable internal components. +* **Retries:** Use exponential backoff and jitter for transient error retries. +* **Idempotency:** Design operations to be idempotent where possible. + +### 3.4 Concern: Configuration Management (CC-004) + +#### 3.4.1 Description + +[How are application and system configurations managed across different environments?] + +#### 3.4.2 Architectural Strategy / Solution + +[e.g., "Externalize all environment-specific and sensitive configurations. Version control configuration files. Use a secure configuration store for secrets."] + +* **Key Technologies/Patterns:** [e.g., Environment variables, Kubernetes ConfigMaps/Secrets, HashiCorp Vault, AWS Secrets Manager, Azure Key Vault.] +* **Related ADRs:** [e.g., ADR-020: "Secure Configuration Management Solution."] + +#### 3.4.3 Implications + +* **Benefits:** [e.g., "Easier environment promotion," "Reduced risk of configuration errors," "Improved security for sensitive data."] +* **Drawbacks/Trade-offs:** [e.g., "Initial setup overhead," "Requires careful access control to configuration stores."] + +#### 3.4.4 Implementation Guidelines + +* **Externalize Config:** No hardcoded configurations in code. +* **Environment Variables:** Use environment variables for non-sensitive, environment-specific settings. +* **Secret Management:** Store all sensitive data (API keys, database credentials) in a dedicated, secure secret management system. +* **Version Control:** Configuration files (excluding secrets) should be version-controlled. + +### 3.5 Concern: Authentication (CC-005) + +#### 3.5.1 Description + +[How users and other systems prove their identity to the system.] + +#### 3.5.2 Architectural Strategy / Solution + +[e.g., "Implement a centralized authentication service. Use industry-standard protocols for user authentication and authorization."] + +* **Key Technologies/Patterns:** [e.g., OAuth 2.0, OpenID Connect, JWT (JSON Web Tokens), Identity Providers (Okta, Auth0, Azure AD).] +* **Related ADRs:** [e.g., ADR-025: "Centralized Identity Management System."] + +#### 3.5.3 Implications + +* **Benefits:** [e.g., "Single Sign-On (SSO) for users," "Consistent security posture," "Reduced development effort for authentication."] +* **Drawbacks/Trade-offs:** [e.g., "Complexity of protocol implementation," "Reliance on external identity provider."] + +#### 3.5.4 Implementation Guidelines + +* **Centralized Auth:** All authentication requests must go through the designated authentication service. +* **Token-Based Auth:** Use JWTs for stateless authentication between services. +* **Secure Storage:** Store user credentials securely (e.g., hashed and salted passwords). + +### 3.6 Concern: Authorization (CC-006) + +#### 3.6.1 Description + +[How the system determines what authenticated users/systems are allowed to do.] + +#### 3.6.2 Architectural Strategy / Solution + +[e.g., "Implement Role-Based Access Control (RBAC) or Attribute-Based Access Control (ABAC)."] + +* **Key Technologies/Patterns:** [e.g., RBAC frameworks, policy engines.] +* **Related ADRs:** [e.g., ADR-030: "Authorization Model Selection."] + +#### 3.6.3 Implications + +* **Benefits:** [e.g., "Granular control over resources," "Easier management of user permissions."] +* **Drawbacks/Trade-offs:** [e.g., "Complexity in defining roles/policies," "Performance overhead for complex evaluations."] + +#### 3.6.4 Implementation Guidelines + +* **Policy Enforcement:** Enforce authorization policies at the service/API boundary. +* **Role/Permission Definition:** Clearly define roles and their associated permissions. +* **Auditing:** Log all authorization attempts and failures. + +## 4. Application and Maintenance + +### 4.1 Application Guidelines + +* These concerns should be considered during all design phases (from high-level architecture to detailed component design). +* All new components and features must adhere to the defined strategies for these concerns. +* Any deviations must be explicitly justified and documented, ideally via an ADR. + +### 4.2 Maintenance and Evolution + +* This document is a living artifact and should be reviewed periodically (e.g., annually or as part of major architectural shifts). +* New crosscutting concerns may emerge and should be added as needed. +* Strategies may evolve based on new technologies, threats, or business needs. + +## 5. References + +[List any other relevant documents (e.g., Architecture Vision, Architecture Principles, specific ADRs, security policies, coding standards) referenced in this document.] +``` diff --git a/content/templates/decision-log.md b/content/templates/decision-log.md new file mode 100644 index 0000000..910d48e --- /dev/null +++ b/content/templates/decision-log.md @@ -0,0 +1,78 @@ +--- +title: Decision log +--- + +# Decision log + +Use this template for a running list of the decisions a project makes. Significant architecture decisions still get their own [architecture decision record](adr.md); the log points to them. + +## Template + +```markdown +--- +title: Decision Log - [Project/System/Topic Name] +status: Active / Archived +date: YYYY-MM-DD (Date of Log Creation) +owner: [Name or Role of Decision Log Owner] +version: 1.0 +related_documents: + - [Link to relevant project plan, design doc, meeting minutes, etc.] +--- + +# Decision Log: [Project/System/Topic Name] + +This log serves as a central record for key decisions made regarding [Project/System/Topic Name]. Its purpose is to ensure that all significant choices, their justifications, and their consequences are transparently documented for current and future reference. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Decision Log is to: +* Document all significant decisions made. +* Capture the rationale and alternatives considered for each decision. +* Record the date, deciders, and any implications of the decision. +* Provide a historical context for future architectural or project evolution. +* Facilitate impact analysis when changes are proposed. +* **Where applicable, ensure decisions are linked to relevant Architecture Decision Records (ADRs).** + +### 1.2 Scope + +This Decision Log covers decisions related to [briefly describe the scope, e.g., "architectural choices for Product X," "project management decisions for Project Y," "design decisions for Module Z"]. + +## 2. Decision Log Entries + +[List each decision made. For each decision, include its rationale, alternatives considered, and any agreed-upon next steps or implications. **Architectural decisions should, where applicable, link to a corresponding Architecture Decision Record (ADR) for more detailed context and justification.**] + +| Decision ID | Decision (Concise) | Rationale | Alternatives Considered | Implications | Deciders | Date Made | Related ADR (if applicable) | Status | Notes | +|-------------|--------------------|-----------|-------------------------|--------------|----------|-----------|-----------------------------|--------|-------| +| **D-001** | Adopt Microservices for new backend. | Need for scalability, independent deployment, team autonomy. | Monolith, Layered Architecture. | Increased operational complexity, initial setup cost. | Lead Architect, CTO | YYYY-MM-DD | ADR-001 | Approved | | +| **D-002** | Use PostgreSQL as primary database. | Strong community, ACID compliance, good tooling. | MongoDB, MySQL. | Need for relational schema design. | Lead Architect, Tech Lead | YYYY-MM-DD | ADR-005 | Approved | | +| **D-003** | Defer Feature X to Phase 2. | High effort for low immediate value; focus on core MVP. | Include in MVP with reduced scope. | Feature X not available in initial release. | Product Owner, Project Manager | YYYY-MM-DD | | Approved | Communicated to stakeholders. | +| **D-004** | Implement OAuth 2.0 for user authentication. | Industry standard, security best practices, future extensibility. | Custom token-based auth. | Requires third-party library integration. | Security Lead, Lead Dev | YYYY-MM-DD | ADR-010 | Approved | | + +## 3. Usage Guidelines + +* **Decision ID:** Unique identifier for the decision (e.g., D-001). +* **Decision (Concise):** A brief, clear statement of the decision made. +* **Rationale:** Explain *why* this decision was made. What factors were considered? What problem does it solve? +* **Alternatives Considered:** Briefly list other options that were evaluated and why they were rejected. +* **Implications:** Describe the direct consequences or impact of this decision (positive and negative). +* **Deciders:** List the individuals or roles who were key in making or approving the decision. +* **Date Made:** The date the decision was finalized. +* **Related ADR (if applicable):** Link to a more detailed Architecture Decision Record if the decision is architectural. +* **Status:** (e.g., Approved, Reversed, Superseded). +* **Notes:** Any additional relevant information or context. + +## 4. Maintenance + +This Decision Log is a living document and should be updated whenever: +* A significant decision is made. +* A previously made decision is revisited or changed. +* New information impacts the rationale or implications of a decision. + +## 5. References + +[List any documents, meeting minutes, or other resources that provide further context or detail for the decisions logged here.] +``` diff --git a/content/templates/index.md b/content/templates/index.md index 964f7bc..de757ec 100644 --- a/content/templates/index.md +++ b/content/templates/index.md @@ -28,20 +28,50 @@ Each checklist is a task list, so it can go straight into a pull request, an iss ## Templates -[[CAPTION:Table]] The templates. +Grouped by when in a project you are likely to need them. + +[[CAPTION:Table]] Templates for vision and architecture. | Template | Use it for | |----------|------------| +| [Project vision](project-vision.md) | Agreeing why a project exists, who it is for and what success looks like | +| [Architecture vision](architecture-vision.md) | Describing where a system's architecture is heading, and why | +| [Architecture principles](architecture-principles.md) | Recording the principles a product or team has agreed | +| [Architecture document](architecture-document.md) | Describing a system's architecture, with C4 diagram starters | +| [Architecture canvases](architecture-canvases.md) | Starting or explaining an architecture on one page | +| [Crosscutting concerns](crosscutting-concerns.md) | Security, logging, errors and configuration across a whole system | | [Architecture decision record](adr.md) | Recording one significant decision | | [Request for comments](rfc.md) | Proposing a change and gathering views before deciding | + +[[CAPTION:Table]] Templates for requirements, design and testing. + +| Template | Use it for | +|----------|------------| | [Requirements and quality scenarios](requirements.md) | Writing requirements that can be traced and tested | +| [NFR scenario](nfr-scenario.md) | Writing one non-functional requirement as a testable scenario | +| [Requirements traceability matrix](requirements-traceability-matrix.md) | Tracing requirements to design, code and tests | | [Interface specification](interface-specification.md) | Describing an interface between components or systems | -| [Architecture document](architecture-document.md) | Describing a system's architecture, with C4 diagram starters | -| [Architecture canvases](architecture-canvases.md) | Starting or explaining an architecture on one page | +| [Software design specification](software-design-specification.md) | The detailed design of a system or module | | [Experiment record](experiment-record.md) | Running a spike, prototype or piece of research | -| [Postmortem](postmortem.md) | Learning from an incident or a defect found in the field | +| [Test plan](test-plan.md) | Planning how a product, system or component is tested | + +[[CAPTION:Table]] Templates for delivery and operations. + +| Template | Use it for | +|----------|------------| +| [Systems engineering management plan](systems-engineering-management-plan.md) | Running the engineering of a larger multi-discipline project | +| [Team working agreement](team-working-agreement.md) | Agreeing how a team works together | +| [Meeting minutes](meeting-minutes.md) | Recording a meeting's decisions and actions | +| [Decision log](decision-log.md) | A running list of a project's decisions | +| [Risk register](risk-register.md) | Tracking risks, their owners and what is being done about them | +| [Technical debt register](technical-debt-register.md) | Recording technical debt and the plan to pay it down | +| [Release plan](release-plan.md) | Planning a release or deployment, including the rollback | +| [Software bill of materials](sbom.md) | Summarising the components a product is built from | | [Pull requests and commits](pull-request-and-commits.md) | Pull request descriptions, commit messages and a commit message hook | +| [Postmortem](postmortem.md) | Learning from an incident or a defect found in the field | +| [Lessons learned](lessons-learned.md) | Looking back at the end of a project or phase | | [Task brief for an AI agent](agent-task-brief.md) | Handing a coding task to an AI agent | +| [AGENTS.md for AI agents](agents-md.md) | Telling AI agents how to work in a repository | | [Conformance profile and component register](product-conformance-profile.md) | Tailoring guidelines to a product, and listing its components | ## Files to download diff --git a/content/templates/lessons-learned.md b/content/templates/lessons-learned.md new file mode 100644 index 0000000..81ddc1b --- /dev/null +++ b/content/templates/lessons-learned.md @@ -0,0 +1,104 @@ +--- +title: Lessons learned +--- + +# Lessons learned + +Use this template at the end of a project or phase to record what went well, what did not, and what the team will do differently. For a single incident, use the [postmortem](postmortem.md) instead. + +## Template + +```markdown +--- +title: Lessons Learned - [Project/Initiative Name or Phase] +status: Draft / Reviewed / Finalised +date: YYYY-MM-DD (Date of Session) +project_start_date: YYYY-MM-DD +project_end_date: YYYY-MM-DD (or Current Phase End Date) +author: [Name or Role of Facilitator] +participants: [List of key participants/stakeholders] +version: 1.0 +--- + +# Lessons Learned: [Project/Initiative Name or Phase] + +This document captures the lessons learned from the [Project/Initiative Name or Phase]. The purpose of this review is to reflect on our experiences, identify what went well and what could be improved, and formulate actionable recommendations to enhance our processes, practices, and outcomes for future projects. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Lessons Learned document is to: +* Systematically collect knowledge gained from our experiences. +* Identify successful practices to be repeated and institutionalized. +* Pinpoint challenges and problems, and understand their root causes. +* Develop actionable recommendations for continuous improvement. +* Share insights across teams and the organization. + +### 1.2 Context of Review + +[Briefly describe the project, phase, or initiative being reviewed. What was its scope, objectives, and timeframe?] + +### 1.3 Participants + +[List the individuals or groups who participated in the Lessons Learned session(s).] + +## 2. What Went Well? (Successes & Strengths) + +[Identify and describe the aspects of the project/initiative that were successful, effective, or positive. Why were they successful?] + +* **[Success 1]:** [Description, e.g., "Our cross-functional team collaboration was excellent."] + * *Why it worked:* [e.g., "Daily syncs fostered strong communication and quick problem-solving."] +* **[Success 2]:** [Description, e.g., "Early stakeholder engagement led to clear requirements."] + * *Why it worked:* [e.g., "Regular review sessions with product owners prevented scope creep."] +* **[Success 3]:** [Description, e.g., "The automated testing pipeline significantly reduced regression bugs."] + * *Why it worked:* [e.g., "Investment in robust CI/CD paid off by catching issues early."] + +## 3. What Could Be Improved? (Challenges, Issues & Weaknesses) + +[Identify and describe the aspects that presented challenges, caused issues, or could have been handled better. What were the root causes of these difficulties?] + +* **[Challenge 1]:** [Description, e.g., "Hardware delays impacted software integration schedule."] + * *Root Cause(s):* [e.g., "Underestimated supplier lead times, lack of early prototyping."] +* **[Challenge 2]:** [Description, e.g., "Requirements changed frequently in the later stages."] + * *Root Cause(s):* [e.g., "Insufficient initial discovery, lack of clear product vision."] +* **[Challenge 3]:** [Description, e.g., "Technical debt accumulated in the authentication module."] + * *Root Cause(s):* [e.g., "Prioritised speed over quality in early sprints, no dedicated tech debt budget."] + +## 4. What Did We Learn? (Key Insights) + +[Based on the successes and challenges, what are the overarching insights or new understandings gained? These are the "lessons" themselves.] + +* **[Lesson 1]:** [e.g., "Proactive engagement with external suppliers is critical for hardware-dependent projects."] +* **[Lesson 2]:** [e.g., "Robust validation of initial assumptions is key to minimizing late-stage requirement changes."] +* **[Lesson 3]:** [e.g., "Allocating regular, small amounts of time for technical debt remediation is more effective than large, infrequent efforts."] +* **[Lesson 4]:** [e.g., "The benefit of 'Docs as Code' for Living Design Documents significantly outweighs the initial setup effort."] + +## 5. Recommendations & Action Items + +[Based on the lessons learned, what specific, actionable recommendations and tasks should be implemented for future projects or processes? Assign owners and target dates.] + +| Action Item ID | Description | Recommendation Type | Owner | Target Date | Status | +|----------------|-------------|---------------------|-------|-------------|--------| +| **LL-AI-001** | Establish a 'supplier risk assessment' checklist for new hardware projects. | Process Improvement | John Smith | YYYY-MM-DD | Open | +| **LL-AI-002** | Incorporate a dedicated 'discovery phase' for new features before committing to development. | Process Improvement | Jane Doe | YYYY-MM-DD | Open | +| **LL-AI-003** | Allocate 15% of sprint capacity to technical debt remediation tasks. | Policy Change | Alice Brown | YYYY-MM-DD | Open | +| **LL-AI-004** | Develop a standard template for PoC reports to capture technical findings. | Documentation | Bob White | YYYY-MM-DD | Open | + +* **Recommendation Types:** (e.g., Process Improvement, Policy Change, Tooling Update, Training/Skill Development, Documentation, Architectural Guidance) + +## 6. Communication & Sharing Plan + +[How will these lessons learned be shared with relevant teams and stakeholders to ensure broader adoption of recommendations?] + +* [e.g., "Present key findings in next All-Hands meeting."] +* [e.g., "Share document link in relevant Slack/Teams channels."] +* [e.g., "Integrate recommendations into updated 'Ways of Working' guidelines."] +* [e.g., "Conduct a focused workshop with other project leads on specific challenges."] + +## 7. References + +[List any relevant project documents, incident reports, or data used during this review.] +``` diff --git a/content/templates/meeting-minutes.md b/content/templates/meeting-minutes.md new file mode 100644 index 0000000..ed8d8ca --- /dev/null +++ b/content/templates/meeting-minutes.md @@ -0,0 +1,130 @@ +--- +title: Meeting minutes +--- + +# Meeting minutes + +Use this template to record a meeting's attendees, decisions and actions. The actions and decisions matter more than a record of who said what. + +## Template + +```markdown +--- +title: "Meeting Minutes - [Meeting Name/Topic]" +status: "[Draft | Reviewed | Finalised]" +date: "YYYY-MM-DD" # Date of Meeting +time: "HH:MM - HH:MM [Timezone]" +location: "[e.g., Virtual (Google Meet/Zoom), Conference Room A]" +facilitator: "[Name or Role of Facilitator]" +scribe: "[Name or Role of Scribe]" +attendees: + - "[Name 1] ([Role/Team])" + - "[Name 2] ([Role/Team])" + - "[Name 3] ([Role/Team])" +absentees: + - "[Name 4] ([Reason for absence])" +version: "1.0" +related_documents: + - "[Link to relevant project plan, design doc, etc.]" +--- + +# Meeting Minutes / Decision Log: [Meeting Name/Topic] + +These are the minutes for the meeting held on [Date of Meeting] regarding [Meeting Name/Topic]. The purpose of this document is to record key discussions, decisions made, and action items assigned, providing a clear reference for all participants and stakeholders. + +--- + +## 1. Meeting Details + +### 1.1 Meeting Name / Topic + +[Clearly state the name or main topic of the meeting.] + +### 1.2 Date and Time + +* **Date:** YYYY-MM-DD +* **Time:** HH:MM - HH:MM [Timezone] + +### 1.3 Location + +[Specify where the meeting took place (e.g., "Virtual (Google Meet)," "Conference Room A").] + +### 1.4 Attendees + +[List all individuals who attended the meeting, including their roles or teams.] + +* [Name 1] ([Role/Team]) +* [Name 2] ([Role/Team]) +* [Name 3] ([Role/Team]) +* ... + +### 1.5 Absentees + +[List any individuals who were expected but did not attend, with a brief reason if known.] + +* [Name 4] ([Reason for absence]) +* ... + +## 2. Agenda + +[List the topics that were planned for discussion.] + +1. [Agenda Item 1] +2. [Agenda Item 2] +3. [Agenda Item 3] +... + +## 3. Key Discussions + +[Summarize the main points discussed for each agenda item. Focus on the essence of the conversation, key arguments, and different perspectives. Avoid verbatim transcripts.] + +### 3.1 [Agenda Item 1 Topic] + +* [Main point of discussion 1] +* [Main point of discussion 2] +* [Any key data or information presented] + +### 3.2 [Agenda Item 2 Topic] + +* [Main point of discussion 1] +* [Main point of discussion 2] + +## 4. Decisions Made (Decision Log) + +[Clearly state each decision made during the meeting. For each decision, include its rationale and any agreed-upon next steps or implications. Link to an ADR if it's an architectural decision.] + +* **Decision 1: [Concise description of the decision]** + * **Rationale:** [Explain *why* this decision was made. What factors were considered? What alternatives were discussed and rejected?] + * **Implications:** [Briefly describe the direct consequences or impact of this decision.] + * **Related ADR (if applicable):** [Link to ADR-XXX] +* **Decision 2: [Concise description of the decision]** + * **Rationale:** + * **Implications:** + +## 5. Action Items + +[List all action items assigned during the meeting. Each action item should be clear, specific, and have an assigned owner and a target completion date.] + +| Action Item # | Description | Owner | Due Date | Status | Notes | +|---------------|-------------|-------|----------|--------|-------| +| **AI-001** | Draft proposal for new API endpoint `GET /users/{id}/profile`. | Jane Doe | YYYY-MM-DD | Open | Will include schema definition. | +| **AI-002** | Investigate feasibility of integrating with external payment gateway X. | John Smith | YYYY-MM-DD | Open | Research existing connectors. | +| **AI-003** | Schedule follow-up meeting with Marketing on feature Y requirements. | Alice Brown | YYYY-MM-DD | Open | Book a slot for next week. | + +## 6. Next Meeting + +* **Date:** [YYYY-MM-DD] (if scheduled) +* **Time:** [HH:MM] (if scheduled) +* **Topic(s):** [Briefly list main topics for next meeting, if known.] + +## 7. Review & Approval + +[Optional: Section for formal review and approval of minutes.] + +* **Reviewed By:** [Name/Role] - [Date] +* **Approved By:** [Name/Role] - [Date] + +## 8. References + +[List any documents, presentations, or other resources that were referenced or distributed during the meeting.] +``` diff --git a/content/templates/nfr-scenario.md b/content/templates/nfr-scenario.md new file mode 100644 index 0000000..2a6c0fc --- /dev/null +++ b/content/templates/nfr-scenario.md @@ -0,0 +1,19 @@ +--- +title: NFR scenario +--- + +# NFR scenario + +Use this template to write a non-functional requirement as a testable quality scenario: the conditions, the event, the response and how it is measured. A requirement such as "the system shall be fast" cannot be verified; a scenario with a threshold can. The [requirements template](requirements.md) covers requirements in general. + +## Template + +```markdown +ID: NFR-XXXX-000 +Attribute: Performance and timing +Environment: Conditions under which the requirement applies +Stimulus: The event that triggers the behaviour +Response: What the system does +Measure: How the response is measured, with a threshold +Verification: Test | Analysis | Inspection | Demonstration +``` diff --git a/content/templates/project-vision.md b/content/templates/project-vision.md new file mode 100644 index 0000000..2715308 --- /dev/null +++ b/content/templates/project-vision.md @@ -0,0 +1,98 @@ +--- +title: Project vision +--- + +# Project vision + +Use this template at the start of a project or initiative to agree why it exists, who it is for and what success looks like. Keep it to a page or two, and revisit it when the scope or the market changes. + +## Template + +```markdown +--- +title: Project Vision - [Project Name/Initiative] +status: Draft / Approved / In Progress / Completed +date: YYYY-MM-DD +owner: [Name or Role of Project Sponsor/Lead] +version: 1.0 +--- + +# Project Vision: [Project Name/Initiative] + +This template outlines the fundamental purpose, scope, and desired outcomes for a specific project or initiative. It serves to align all stakeholders, provide clear direction, and ensure the project contributes effectively to broader organisational goals. + +## Project Name / Initiative Title + +[Clearly state the name of the project or initiative.] + +## Project Vision Statement + +A concise and inspiring statement that articulates the desired future state or impact this project aims to achieve. It answers: "What success looks like at a high level once this project is complete?" + +*Example: To establish a robust, scalable, and secure cloud infrastructure that enables rapid deployment of new services and reduces operational costs by Q4 2025.* + +## Strategic Alignment & Purpose + +How does this project align with and contribute to the overarching business strategy, product vision, or organisational objectives? Why is this project critical now? + +* [Reference relevant strategic goals, company OKRs, or product vision.] +* [Explain the fundamental reason for initiating this project.] + +## Problem / Opportunity Statement + +Clearly articulate the primary problem(s) this project aims to solve, or the significant opportunity it aims to seize. What challenges exist that this project addresses? What new possibilities does it unlock? + +*Example: Our current on-premise infrastructure is nearing capacity, lacks the agility for rapid scaling, and incurs high maintenance overhead, hindering our ability to innovate and respond to market demands.* + +## Key Objectives & Desired Outcomes + +What specific, measurable objectives will this project achieve? These should be outcomes (what changes as a result), not just outputs (what we deliver). How will we know if the project is successful? + +* **Objective 1:** [e.g., Reduce infrastructure operational costs by 20% within 12 months post-migration.] +* **Objective 2:** [e.g., Achieve 99.99% uptime for core services in the new environment.] +* **Objective 3:** [e.g., Enable self-service provisioning of development environments within 1 hour.] +* **Objective 4:** [e.g., Successfully migrate 80% of legacy applications to the new platform by [Date].] + +## Scope & Boundaries + +What is definitively *in scope* for this project? What is *out of scope*? This helps manage expectations, prevent scope creep, and focus efforts. + +* **In Scope:** + * [Specific deliverables, systems, or areas to be addressed.] + * [e.g., Cloud platform setup and configuration (AWS/Azure/GCP).] + * [e.g., Migration of applications A, B, and C.] + * [e.g., Development of automated deployment pipelines for new services.] +* **Out of Scope:** + * [Specific items, features, or phases that will NOT be part of this project.] + * [e.g., Migration of all legacy applications (beyond A, B, C).] + * [e.g., Custom application development (unless directly related to platform features).] + * [e.g., Long-term operational support model (will be defined by a subsequent initiative).] + +## Key Stakeholders + +Who are the primary individuals, teams, or groups who have a vested interest in this project's success or are significantly impacted by it? + +* **Project Sponsor(s):** [Names/Roles] +* **Key Business Leads:** [Names/Roles] +* **Affected Teams/Departments:** [Names/Roles, e.g., Software Development, Operations, Product Management] +* **Key Users/Customers:** [Representing the end-users of the project's outcomes] + +## High-Level Risks & Assumptions + +What are the major known risks that could impact the project's success? What critical assumptions are we making that, if proven false, could derail the project? + +* **Risks:** + * [e.g., Unforeseen technical complexity in legacy application migration.] + * [e.g., Lack of sufficient skilled resources for new cloud technologies.] +* **Assumptions:** + * [e.g., Key vendor XYZ will provide timely support for integration.] + * [e.g., Budget approval for cloud services will be granted as estimated.] + +## Related Documents & References (Optional) + +Links to any other relevant documents, research, or resources that provide additional context or detail. + +* [Link to High-Level Architectural Diagram] +* [Link to Initial Cost-Benefit Analysis] +* [Link to Regulatory Compliance Requirements] +``` diff --git a/content/templates/release-plan.md b/content/templates/release-plan.md new file mode 100644 index 0000000..0d8fd50 --- /dev/null +++ b/content/templates/release-plan.md @@ -0,0 +1,216 @@ +--- +title: Release plan +--- + +# Release plan + +Use this template to plan a release or deployment: what is in it, the steps, the checks, who does what, and how to roll back. + +## Template + +```markdown +--- +title: Release Plan / Deployment Plan - [Product/System Name] v[Version Number] +status: Draft / Reviewed / Approved / Executing / Completed +date: YYYY-MM-DD (Date of Plan) +release_date: YYYY-MM-DD (Target Release Date) +author: [Name or Role of Release Manager/Lead] +version: 1.0 +related_documents: + - System-Level Requirements - [Product Name] + - Test Plan - [Product Name] + - Technical Debt Register - [Product Name] +--- + +# Release Plan / Deployment Plan: [Product/System Name] v[Version Number] + +This document outlines the plan for releasing and deploying [Product/System Name] version [Version Number]. It details the scope, objectives, activities, responsibilities, and timeline for the release, ensuring a controlled and successful transition to the target environment. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Release Plan / Deployment Plan is to define the comprehensive strategy and detailed steps for delivering [Product/System Name] v[Version Number] to [target environment, e.g., "production," "customer sites," "app stores"]. It aims to: +* Ensure all necessary activities are planned and executed. +* Minimize risks and potential downtime during deployment. +* Provide clear roles and responsibilities. +* Facilitate effective communication among all stakeholders. + +### 1.2 Release Scope + +[Clearly define what is included in this release. List key features, bug fixes, or components.] + +* **New Features:** + * [Feature A] + * [Feature B] +* **Bug Fixes:** + * [Bug ID 1] + * [Bug ID 2] +* **Improvements/Technical Debt:** + * [Improvement X] + * [Technical Debt Item Y] +* **Affected Components/Systems:** + * [Component 1] + * [Component 2] + +### 1.3 Release Objectives + +What are the specific, measurable goals for this release? + +* [e.g., "Successfully deploy all new features to production by YYYY-MM-DD."] +* [e.g., "Achieve 99.9% uptime during and after deployment."] +* [e.g., "Ensure zero data loss during database migration (if applicable)."] +* [e.g., "Roll out firmware update to all beta devices within 24 hours."] + +### 1.4 Target Environment(s) + +[Specify the environment(s) to which this release will be deployed.] + +* [e.g., "Production Cloud Environment (AWS/Azure/GCP)"] +* [e.g., "Customer On-Premise Servers"] +* [e.g., "Mobile App Stores (iOS/Android)"] +* [e.g., "IoT Device Fleet"] + +## 2. Pre-Release Activities + +### 2.1 Code Freeze & Branching Strategy + +[Define when the code freeze will occur and the branching strategy for release.] + +* **Code Freeze Date:** YYYY-MM-DD HH:MM +* **Release Branch:** [e.g., `release/vX.Y.Z` created from `develop` or `main`] + +### 2.2 Final Testing & Quality Gates + +[Describe the final testing activities required before release.] + +* **Regression Testing:** [e.g., "All automated regression suites must pass."] +* **Performance Testing:** [e.g., "Load tests to simulate X concurrent users."] +* **Security Testing:** [e.g., "Final vulnerability scan, penetration test (if applicable)."] +* **User Acceptance Testing (UAT):** [e.g., "Business stakeholders sign-off on UAT results."] +* **Compliance Checks:** [e.g., "Regulatory compliance review completed."] + +### 2.3 Documentation Preparation + +[List all documentation that needs to be prepared or updated for the release.] + +* **Release Notes:** [Summary of new features, bug fixes, known issues.] +* **User Manuals/Guides:** +* **API Documentation:** +* **Installation/Configuration Guides:** +* **Operational Runbooks:** +* **Rollback Procedures:** + +### 2.4 Stakeholder Communication & Readiness + +[How will stakeholders be informed and prepared for the release?] + +* **Internal Communication:** [e.g., "Email to all-staff, Slack announcements."] +* **External Communication:** [e.g., "Customer release notes, marketing announcements."] +* **Training:** [e.g., "Support team training on new features."] + +## 3. Deployment Strategy + +### 3.1 Deployment Approach + +[Describe the chosen deployment method.] + +* **Method:** [e.g., "Blue/Green Deployment," "Canary Release," "Rolling Update," "Big Bang," "Phased Rollout"] +* **Automation:** [e.g., "Fully automated CI/CD pipeline," "Manual steps with scripts."] + +### 3.2 Deployment Steps (Detailed Checklist) + +[Provide a step-by-step checklist for the deployment process. Assign owners and estimated times for each step.] + +| Step # | Activity | Owner | Est. Time | Status | Notes | +|--------|----------|-------|-----------|--------|-------| +| 1. | Pre-deployment health check of current system. | Ops Team | 15 min | | | +| 2. | Backup critical databases. | Ops Team | 30 min | | | +| 3. | Deploy new application version to staging. | DevOps | 10 min | | | +| 4. | Run smoke tests on staging. | QA Team | 20 min | | | +| 5. | Deploy new application version to production (Blue environment). | DevOps | 15 min | | | +| 6. | Monitor production health metrics for 30 minutes. | Ops Team | 30 min | | | +| 7. | Switch traffic to new version. | DevOps | 5 min | | | +| 8. | Post-deployment verification tests on production. | QA Team | 15 min | | | +| 9. | Decommission old environment (Green environment). | DevOps | 30 min | | | +| 10. | Announce successful deployment. | Release Mgr | 5 min | | | + +### 3.3 Rollback Plan + +[Detail the steps to revert to the previous stable version in case of critical issues during or after deployment.] + +* **Rollback Trigger:** [What conditions will trigger a rollback? e.g., "Critical error rate > 5%," "Major functionality broken."] +* **Rollback Steps:** + 1. [Step 1: e.g., "Switch traffic back to old environment."] + 2. [Step 2: e.g., "Restore database from pre-deployment backup (if necessary)."] + 3. [Step 3: e.g., "Analyze logs to understand failure."] + +## 4. Post-Release Activities + +### 4.1 Monitoring & Verification + +[How will the system be monitored immediately after release to ensure stability and performance?] + +* **Key Metrics to Monitor:** [e.g., "Error rates," "Latency," "CPU/Memory usage," "User login success."] +* **Monitoring Tools:** [e.g., Prometheus, Grafana, ELK Stack.] +* **Alerting:** [Define critical alerts and response procedures.] + +### 4.2 Incident Management + +[Reference the Post-Mortem / Incident Review Template for handling any issues that arise post-release.] + +### 4.3 Feedback Collection + +[How will feedback from users and stakeholders be collected post-release?] + +* [e.g., "Customer support tickets," "User surveys," "Analytics data."] + +### 4.4 Lessons Learned + +[Plan for a Lessons Learned session after the release to capture insights for future improvements. Refer to the Lessons Learned Template.] + +## 5. Roles and Responsibilities + +[Identify the key roles and their responsibilities during the release and deployment.] + +* **Release Manager:** Overall coordination, communication, decision-making. +* **Development Leads:** Code readiness, bug fixes. +* **DevOps/Operations Team:** Deployment execution, infrastructure readiness, monitoring. +* **QA Team:** Final testing, post-deployment verification. +* **Product Owner:** Feature sign-off, stakeholder communication. +* **Support Team:** Customer readiness, incident handling. + +## 6. Communication Plan (During Release) + +[How will internal and external stakeholders be communicated with during the release process?] + +* **Pre-Release:** [e.g., "Scheduled downtime notification."] +* **During Deployment:** [e.g., "Live status updates in #release-status channel."] +* **Post-Deployment:** [e.g., "Success notification, known issues update."] +* **Incident Communication:** [Reference incident communication protocol.] + +## 7. Risks and Contingencies + +[Identify potential risks to the release and corresponding mitigation/contingency plans.] + +* **Risk 1:** [e.g., "Critical bug discovered during deployment."] + * *Mitigation:* [e.g., "Thorough pre-release testing."] + * *Contingency:* [e.g., "Execute rollback plan."] +* **Risk 2:** [e.g., "Performance degradation after deployment."] + * *Mitigation:* [e.g., "Rigorous load testing."] + * *Contingency:* [e.g., "Scale up resources, investigate bottlenecks, rollback if severe."] + +## 8. Approvals + +[Sign-off section for key stakeholders before release.] + +* **[Role 1]:** [Name, Date] +* **[Role 2]:** [Name, Date] +* ... + +## 9. References + +[List any other relevant documents (e.g., project plan, architectural diagrams, security reports) referenced in this plan.] +``` diff --git a/content/templates/requirements-traceability-matrix.md b/content/templates/requirements-traceability-matrix.md new file mode 100644 index 0000000..308af2f --- /dev/null +++ b/content/templates/requirements-traceability-matrix.md @@ -0,0 +1,79 @@ +--- +title: Requirements traceability matrix +--- + +# Requirements traceability matrix + +Use this template to trace each requirement to its design, its implementation and the tests that verify it. Generate it from your tools where you can; a matrix kept by hand goes out of date quickly. + +## Template + +```markdown +--- +title: Requirements Traceability Matrix (RTM) - [Product/System/Component Name] +status: Draft / In Progress / Finalised +date: YYYY-MM-DD +owner: [Name or Role of QA Lead/Architect] +version: 1.0 +--- + +# Requirements Traceability Matrix (RTM): [Product/System/Component Name] + +This Requirements Traceability Matrix (RTM) provides a mapping between different levels of requirements, design elements, and test cases for the [Product/System/Component Name]. It ensures that every requirement is addressed in the design, implemented in the code, and verified through testing. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this RTM is to establish and maintain traceability throughout the development lifecycle of the [Product/System/Component Name]. It helps to: +* Ensure all requirements are covered by design and testing. +* Facilitate impact analysis for changes to requirements. +* Provide evidence for compliance and quality assurance. + +### 1.2 Scope + +This RTM covers the traceability from [e.g., "System-Level Requirements"] to [e.g., "Detailed Requirements, Design Elements, and Test Cases"]. + +## 2. Traceability Matrix + +This matrix links requirements to their corresponding design components, source code, and test cases. + +| Requirement ID | Requirement Description (Brief) | Parent Requirement ID (if applicable) | Detailed Requirement ID(s) | Design Element ID(s) / ADR(s) | Source Code File(s) / Module(s) | Test Case ID(s) | Test Status (Latest) | Notes | +|----------------|---------------------------------|---------------------------------------|----------------------------|-------------------------------|-----------------------------------|-----------------|----------------------|-------| +| **SYS-FUN-001**| The system shall allow user login. | N/A | COMP-FUN-001 | ADR-005 (Auth Mechanism) | `auth_service.py` | TC-LOGIN-001 | Passed | | +| **SYS-FUN-002**| The system shall display product catalog. | N/A | COMP-FUN-002 | | `catalog_module.c` | TC-CAT-001 | Passed | | +| **SYS-NFR-PERF-001**| Login response time < 500ms. | N/A | COMP-NFR-PERF-001 | | `auth_service.py` | TC-LOGIN-PERF-001| Passed | | +| **SYS-SAFE-001**| System shall prevent over-temperature. | N/A | FIRM-SAFE-001, HW-SAFE-001 | ADR-010 (Thermal Mgmt) | `temp_sensor.c`, `hw_monitor.vhd` | TC-TEMP-001 | Passed | | +| **COMP-FUN-001**| The authentication module shall validate credentials. | SYS-FUN-001 | N/A | | `auth_module.c` | TC-AUTH-001 | Passed | | +| **FIRM-SAFE-001**| Firmware shall read temp sensor. | SYS-SAFE-001 | N/A | | `firmware/sensors.c` | TC-FIRM-TEMP-001| Passed | | +| **HW-SAFE-001**| Hardware shall include thermal cutoff. | SYS-SAFE-001 | N/A | | `hardware/schematic.pdf` | TC-HW-TEMP-001 | Passed | Needs HW Rev 2| + +## 3. Usage Guidelines + +* **Requirement ID:** Unique identifier for the requirement (e.g., from System-Level Requirements or Detailed Requirements document). +* **Requirement Description (Brief):** A short summary of the requirement. +* **Parent Requirement ID:** Links to the higher-level requirement from which this one is derived. +* **Detailed Requirement ID(s):** Links to more granular requirements (if this is a high-level RTM). +* **Design Element ID(s) / ADR(s):** Links to specific design documents, components, or Architecture Decision Records that address this requirement. +* **Source Code File(s) / Module(s):** References the specific code files or modules where the requirement is implemented. +* **Test Case ID(s):** Links to the test cases designed to verify this requirement. +* **Test Status (Latest):** The current status of the associated test cases (e.g., Passed, Failed, Not Run, Blocked). +* **Notes:** Any additional relevant information or context. + +## 4. Maintenance + +This RTM is a living document and should be updated whenever: +* Requirements are added, modified, or removed. +* Design decisions are made or changed. +* Code is implemented or refactored. +* Test cases are created, updated, or executed. + +## 5. References + +* [Link to System-Level Requirements Document] +* [Link to Detailed Requirements Document(s)] +* [Link to Test Plan Document(s)] +* [Link to Architecture Decision Records (ADRs) folder] +``` diff --git a/content/templates/risk-register.md b/content/templates/risk-register.md new file mode 100644 index 0000000..cf6d8cd --- /dev/null +++ b/content/templates/risk-register.md @@ -0,0 +1,126 @@ +--- +title: Risk register +--- + +# Risk register + +Use this template to track a project's risks, how likely and serious each one is, who owns it and what is being done about it. Review it regularly; a register nobody reads does not reduce any risk. + +## Template + +```markdown +--- +title: Risk Register - [Product/System/Project Name] +status: Active / Archived +date: YYYY-MM-DD +owner: [Name or Role of Risk Owner/Project Manager] +version: 1.0 +--- + +# Risk Register: [Product/System/Project Name] + +This Risk Register serves as a central repository for identified risks related to the [Product/System/Project Name]. It provides a structured way to track, assess, and manage potential issues that could impact the project's objectives, scope, schedule, budget, or quality. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Risk Register is to: +* Document all identified risks. +* Assess the likelihood and impact of each risk. +* Define clear mitigation strategies and contingency plans. +* Assign ownership for risk management activities. +* Facilitate continuous monitoring and review of risks. + +### 1.2 Scope + +This Risk Register covers risks associated with [briefly describe the scope, e.g., "the development and deployment of Product X," "the integration of System Y," "Project Z from inception to delivery"]. + +### 1.3 Definitions and Acronyms + +[List any risk-specific terms, definitions, or acronyms.] + +## 2. Risk Assessment Criteria + +### 2.1 Likelihood Scale + +| Level | Description | Probability | +|-------|---------------------|-------------| +| 1 | Very Low / Rare | < 10% | +| 2 | Low / Unlikely | 10-30% | +| 3 | Medium / Possible | 30-70% | +| 4 | High / Likely | 70-90% | +| 5 | Very High / Almost Certain | > 90% | + +### 2.2 Impact Scale + +| Level | Description | Impact on Schedule | Impact on Budget | Impact on Quality | +|-------|-------------|--------------------|------------------|-------------------| +| 1 | Very Low | Negligible | Negligible | Negligible | +| 2 | Low | Minor delay | Minor cost | Minor degradation | +| 3 | Medium | Moderate delay | Moderate cost | Noticeable degradation | +| 4 | High | Significant delay | Significant cost | Major degradation | +| 5 | Very High | Project threatened | Project threatened | Unacceptable degradation | + +### 2.3 Risk Score (Likelihood x Impact) + +[Optional: Define how the risk score is calculated and what ranges mean (e.g., Low, Medium, High, Critical).] + +## 3. Risk Register Log + +| Risk ID | Description (Brief) | Category | Likelihood (Current) | Impact (Current) | Score (Current) | Mitigation Strategy | Contingency Plan | Owner | Status | Date Raised | Date Last Updated | +|---------|---------------------|----------|----------------------|------------------|-----------------|---------------------|------------------|-------|--------|-------------|-------------------| +| **R-001** | Delay in critical hardware component delivery. | Supply Chain | 4 (High) | 4 (High) | 16 | Identify alternative suppliers, pre-order long-lead items. | Use simulation/emulation for software development. | Jane Doe | Open | YYYY-MM-DD | YYYY-MM-DD | +| **R-002** | Unforeseen technical complexity in AI model integration. | Technical | 3 (Medium) | 5 (Very High) | 15 | Conduct spike/PoC, bring in external expertise. | Revert to simpler, non-AI solution. | John Smith | Open | YYYY-MM-DD | YYYY-MM-DD | +| **R-003** | Scope creep due to evolving stakeholder requirements. | Project | 3 (Medium) | 3 (Medium) | 9 | Implement strict change control process, clear scope definition. | Prioritize ruthlessly, defer non-essential features. | Alice Brown | Open | YYYY-MM-DD | YYYY-MM-DD | +| **R-004** | Cybersecurity vulnerability in third-party library. | Security | 2 (Low) | 4 (High) | 8 | Regular security audits, use trusted libraries, keep dependencies updated. | Isolate affected module, prepare emergency patch. | Bob White | Open | YYYY-MM-DD | YYYY-MM-DD | +| **R-005** | Key team member leaves project unexpectedly. | Resource | 2 (Low) | 3 (Medium) | 6 | Cross-training, comprehensive documentation. | Reallocate tasks, hire contractor. | Carol Green | Open | YYYY-MM-DD | YYYY-MM-DD | + +## 4. Risk Categories + +[Define common categories for risks to help with analysis and reporting.] + +* **Technical:** (e.g., architecture, design, complexity, new technology, integration) +* **Project Management:** (e.g., schedule, budget, scope, resources, planning) +* **External:** (e.g., market changes, regulatory, supplier, economic) +* **Organisational:** (e.g., culture, stakeholder engagement, internal politics) +* **Operational:** (e.g., deployment, support, infrastructure, security) +* **Safety/Compliance:** (e.g., regulatory adherence, hazard control) + +## 5. Risk Management Process + +### 5.1 Identification + +[How will risks be identified? (e.g., brainstorming, checklists, lessons learned, expert judgment).] + +### 5.2 Analysis + +[How will risks be assessed for likelihood, impact, and score?] + +### 5.3 Mitigation & Response Planning + +[How will strategies be developed to reduce likelihood or impact? (e.g., avoid, reduce, transfer, accept).] + +### 5.4 Monitoring & Control + +[How will risks be continuously tracked, reviewed, and reported? (e.g., regular risk reviews, dashboards).] + +## 6. Roles and Responsibilities + +* **Risk Owner:** Responsible for monitoring assigned risks, implementing mitigation strategies, and reporting status. +* **Project Manager/Lead:** Overall responsibility for the Risk Register, facilitating risk reviews, and escalating critical risks. +* **Team Members:** Responsible for identifying and reporting new risks. + +## 7. Review Schedule + +[How often will the Risk Register be reviewed and by whom?] + +* **Frequency:** [e.g., "Weekly during team syncs," "Bi-weekly with Project Lead," "Monthly with Steering Committee."] +* **Attendees:** [e.g., "Core team," "Project Lead & Key Stakeholders."] + +## 8. References + +[List any other documents (e.g., project plan, requirements, ADRs) or standards referenced in this Risk Register.] +``` diff --git a/content/templates/sbom.md b/content/templates/sbom.md new file mode 100644 index 0000000..deb4508 --- /dev/null +++ b/content/templates/sbom.md @@ -0,0 +1,143 @@ +--- +title: Software bill of materials +--- + +# Software bill of materials + +Use this template to describe the components a product is built from, with their versions, licences and known vulnerabilities. For anything you ship, generate the bill of materials in a standard format such as CycloneDX or SPDX as part of the build, and keep this page as the summary. + +## Template + +```markdown +--- +title: Software Bill of Materials (SBOM) - [Product/System Name] v[Version Number] +status: Draft / Generated / Verified / Baselined +date: YYYY-MM-DD (Date of Generation) +author: [Name or Role of SBOM Generator/Owner] +version: 1.0 +related_documents: + - Release Plan - [Product Name] v[Version Number] + - Risk Register - [Product Name] +--- + +# Software Bill of Materials (SBOM): [Product/System Name] v[Version Number] + +This document lists all software components, including open-source and commercial, that are incorporated into [Product/System Name] version [Version Number]. It provides transparency into our software supply chain, enabling proactive vulnerability management, license compliance, and risk assessment. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Software Bill of Materials (SBOM) is to: +* Provide a comprehensive inventory of all software components within the product. +* Facilitate vulnerability management by identifying affected components when new vulnerabilities are disclosed. +* Ensure compliance with open-source and third-party software licenses. +* Support security auditing and risk assessment activities. +* Enhance transparency and trust in our software supply chain. + +### 1.2 Scope + +This SBOM covers all software components (libraries, frameworks, operating systems, firmware, etc.) that are part of, or linked into, the [Product/System Name] version [Version Number]. It includes both direct dependencies and transitive dependencies (dependencies of dependencies). + +### 1.3 Definitions and Acronyms + +* **Component:** A distinct software package, library, or module. +* **Direct Dependency:** A component explicitly listed in the product's build or manifest file. +* **Transitive Dependency:** A component that is a dependency of a direct dependency (or further down the chain). +* **CPE (Common Platform Enumeration):** A standardized naming scheme for software applications, operating systems, and hardware devices. +* **PURL (Package URL):** A URL string that identifies and locates a software package. +* **SPDX (Software Package Data Exchange):** An open standard for communicating SBOM information. +* **CycloneDX:** A lightweight SBOM standard designed for use in application security contexts and supply chain component analysis. + +## 2. SBOM Generation Details + +### 2.1 Generation Tool(s) + +[Specify the tool(s) used to generate this SBOM (e.g., Syft, Dependency-Track, Black Duck, custom scripts).] + +* **Tool Name:** [e.g., Syft] +* **Version:** [e.g., 0.80.0] +* **Configuration/Command:** [e.g., `syft dir:./ --output spdx-json=sbom.json`] + +### 2.2 Source of Analysis + +[Describe where the SBOM was generated from (e.g., source code repository, built artifact, container image).] + +* **Location:** [e.g., `git@github.com:your-org/your-repo.git#v1.0.0`] +* **Artifact Hash (if applicable):** [e.g., SHA256 hash of the final build artifact] + +### 2.3 SBOM Format + +[Specify the format of the generated SBOM (e.g., SPDX JSON, CycloneDX JSON, SPDX Tag-Value).] + +* **Format:** [e.g., SPDX JSON] + +## 3. Software Component Inventory + +This section provides a summary of the identified components. For a detailed, machine-readable list, refer to the attached/linked SBOM file. + +| Component Name | Version | Supplier/Author | License(s) | CPE / PURL (if available) | Hashes (SHA256) | Direct/Transitive | Vulnerabilities (Known) | Notes | +|----------------|---------|-----------------|------------|---------------------------|-----------------|-------------------|-------------------------|-------| +| `openssl` | `1.1.1u`| OpenSSL Project | OpenSSL | `pkg:deb/debian/openssl@1.1.1u` | `abcdef123...` | Direct | CVE-2023-XXXX | | +| `libjpeg-turbo`| `2.1.0` | libjpeg-turbo | BSD-3-Clause | `pkg:npm/libjpeg-turbo@2.1.0` | `1234567...` | Transitive | | | +| `react` | `18.2.0`| Meta | MIT | `pkg:npm/react@18.2.0` | `fedcba9...` | Direct | | | +| `sqlite` | `3.39.0`| SQLite | Public Domain | `pkg:generic/sqlite@3.39.0` | `9876543...` | Direct | | Used for local caching | +| `firmware-sdk` | `2.5.1` | Internal | Proprietary | | `zyxwvu...` | Direct | | | + +## 4. License Compliance Summary + +[Summarize the licensing posture of the components. Highlight any licenses requiring specific actions (e.g., copyleft licenses).] + +* **Total Unique Licenses:** [Number] +* **Common Licenses:** [e.g., MIT, Apache 2.0, BSD-3-Clause] +* **Licenses Requiring Attention:** [e.g., GPLv3 (if applicable), LGPLv2.1] + * **[License Type]:** [Action Required, e.g., "Ensure source code distribution compliance."] + +## 5. Vulnerability Management + +### 5.1 Known Vulnerabilities + +[Summarize any known vulnerabilities identified in the components listed in this SBOM. Refer to a separate vulnerability report if detailed.] + +* **Total Critical Vulnerabilities:** [Number] +* **Total High Vulnerabilities:** [Number] +* **Top 3 Vulnerabilities (CVE ID & Component):** + 1. [CVE-YYYY-XXXX - Component A vX.Y.Z] + 2. [CVE-YYYY-YYYY - Component B vA.B.C] + 3. [CVE-YYYY-ZZZZ - Component C vP.Q.R] + +### 5.2 Remediation Strategy (High-Level) + +[Describe the general approach to addressing identified vulnerabilities.] + +* [e.g., "Prioritize critical and high vulnerabilities for immediate patching/upgrading."] +* [e.g., "Monitor vulnerability databases (e.g., NVD, OSV) for new disclosures."] +* [e.g., "Regularly regenerate SBOMs to detect new components or changes."] + +## 6. SBOM Usage and Maintenance + +### 6.1 How to Use This SBOM + +* **Security Teams:** For vulnerability assessment and supply chain risk management. +* **Legal/Compliance Teams:** For open-source license compliance. +* **Development Teams:** For understanding dependencies and making informed decisions about new component adoption. +* **Operations Teams:** For understanding deployed software components. + +### 6.2 Maintenance + +This SBOM is a living artifact and should be regenerated and reviewed: +* [e.g., "Upon every major release."] +* [e.g., "After significant dependency changes."] +* [e.g., "Periodically (e.g., monthly/quarterly) for continuous monitoring."] +* [e.g., "When a new critical vulnerability is disclosed in a commonly used component."] + +## 7. References + +* [Link to the raw SBOM file (e.g., `sbom.json`)] +* [Link to Vulnerability Report (if separate)] +* [Link to Licensing Compliance Report (if separate)] +* [Official SPDX Specification] +* [Official CycloneDX Specification] +``` diff --git a/content/templates/software-design-specification.md b/content/templates/software-design-specification.md new file mode 100644 index 0000000..036c3d9 --- /dev/null +++ b/content/templates/software-design-specification.md @@ -0,0 +1,209 @@ +--- +title: Software design specification +--- + +# Software design specification + +Use this template for the detailed design of a system or module: its structure, interfaces, data and behaviour. It sits below the architecture document and above the code. + +## Template + +````markdown +--- +title: Software Design Specification - [System/Module Name] +status: Draft / Reviewed / Approved / Baselined / Evolving +date: YYYY-MM-DD +author: [Name or Role of Lead Developer/Architect] +version: 1.0 +related_requirements: + - SYS-FUN-XXX + - COMP-FUN-YYY +related_adrs: + - ADR-ZZZ +--- + +# Software Design Specification: [System/Module Name] + +This document specifies the architectural and detailed design for the [System/Module Name]. It describes how the software will be structured, how its components will interact, and how it will fulfill the defined requirements. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Software Design Specification (SDS) is to describe the design of the [System/Module Name]. It serves as a blueprint for implementation, providing sufficient detail for developers to build the software and for testers to verify its correctness. + +### 1.2 Scope + +This document covers the design aspects of [briefly describe the scope, e.g., "the entire [System Name] software," or "the [Module Name] within [System Name]"]. It directly addresses the requirements outlined in [link to System-Level Requirements or Detailed Requirements document]. + +### 1.3 Definitions and Acronyms + +[List any software-specific terms, definitions, or acronyms used in this document.] + +## 2. Architectural Design + +This section provides a high-level overview of the software architecture, showing major components and their relationships. + +### 2.1 Overall Architecture + +[Describe the chosen architectural style (e.g., Microservices, Layered, Event-Driven). Provide a high-level overview of the system's structure.] + +*Reference C4 Model Context/Container Diagrams here if applicable.* + +### 2.2 Major Components / Subsystems + +[Identify and briefly describe the main logical or physical components of the software. For each, state its primary responsibilities.] + +* **[Component A]:** [Description of responsibilities] +* **[Component B]:** [Description of responsibilities] +* ... + +### 2.3 Data Flow and Interactions + +[Describe how data flows through the system and how major components interact. Use diagrams where helpful.] + +```mermaid +graph TD + User[User Interface] --> API[API Gateway] + API --> AuthService[Authentication Service] + API --> ProductService[Product Service] + ProductService --> Database[Product Database] + AuthService --> UserDatabase[User Database] +``` +### 2.4 External Interfaces + +\[Describe any interfaces with external systems, hardware, or other software components outside the immediate scope of this design. Refer to ICDs if available.] + +* **\[External System X] Interface:** \[Description, e.g., "REST API for payment processing."] + +* **\[Hardware Component Y] Interface:** \[Description, e.g., "SPI communication with sensor."] + +## 3. Detailed Design + +This section delves into the internal design of individual components or modules. For each significant component or module identified in Section 2, provide the following details: + +### 3.1 \[Component/Module Name] Design + +#### 3.1.1 Purpose & Responsibilities + +\[Detailed description of what this component does and its specific responsibilities.] + +#### 3.1.2 Internal Structure + +\[Describe the internal breakdown of the component into smaller sub-components, classes, or functions. Use diagrams (e.g., class diagrams, sequence diagrams) as needed.] + +```mermaid +classDiagram + class OrderService { + +createOrder(OrderRequest request) + +getOrderDetails(String orderId) + -validateOrder(OrderRequest request) + -persistOrder(Order order) + } + class OrderRepository { + +save(Order order) + +findById(String id) + } + OrderService "1" -- "1" OrderRepository : uses +``` + +#### 3.1.3 Data Structures + +\[Specify the key data structures, objects, or data models used within this component, including their fields, types, and relationships.] + +#### 3.1.4 Algorithms + +\[Describe any complex or critical algorithms used within the component. Provide pseudocode or flowcharts if necessary.] + +#### 3.1.5 Error Handling & Fault Tolerance + +\[How will this component handle errors, exceptions, and unexpected conditions? What are its fault tolerance mechanisms?] + +#### 3.1.6 State Management (if applicable) + +\[Describe how the component manages its internal state and any state transitions.] + +### 3.2 \[Another Component/Module Name] Design + +\[Repeat Section 3.1 for other significant components/modules.] + +## 4. Interface Design (Internal APIs) + +\[Define the specific APIs or interfaces exposed by components for internal use by other components within the system.] + +* **\[API Name]:** \[e.g., `ProductService.getProductDetails(productId)`] + + * **Purpose:** \[Briefly describe.] + + * **Input Parameters:** \[Name, Type, Description] + + * **Return Value:** \[Type, Description] + + * **Error Codes/Exceptions:** \[List and describe.] + +## 5. Non-Functional Requirements (Design Considerations) + +How does this design address the non-functional requirements (performance, security, scalability, maintainability, etc.)? + +### 5.1 Performance + +\[How is performance considered in the design? (e.g., caching strategies, asynchronous processing, efficient algorithms).] + +### 5.2 Scalability + +\[How is the design scalable? (e.g., stateless components, horizontal scaling, load balancing).] + +### 5.3 Security + +\[How does the design enforce security? (e.g., authentication, authorization, input validation, encryption, secure communication).] + +### 5.4 Maintainability + +\[How does the design promote maintainability? (e.g., modularity, clear separation of concerns, coding standards, documentation).] + +### 5.5 Testability + +\[How does the design facilitate testing? (e.g., dependency injection, clear interfaces, logging).] + +## 6. Data Design + +\[If applicable, describe the persistent data storage design.] + +### 6.1 Database Schema / Data Model + +\[Provide the logical or physical database schema, including tables, relationships, and key fields. Or describe the data model for NoSQL databases.] + +### 6.2 Data Flow & Storage + +\[Describe how data is stored, retrieved, and managed by the system.] + +## 7. Security Considerations + +\[Detailed breakdown of security aspects beyond non-functional requirements, if needed. (e.g., threat modeling, access control mechanisms, data protection policies).] + +## 8. Deployment Considerations + +\[How will the software be deployed? (e.g., deployment units, environment requirements, dependencies).] + +## 9. Testing Considerations + +\[How does this design facilitate testing at various levels (unit, integration, system)? What specific testability features are included?] + +## 10. Traceability + +This section notes how this design traces back to requirements and forward to implementation and testing. + +* **Traces from:** \[Link to System-Level Requirements and Detailed Requirements] + +* **Traces to:** \[Link to Source Code Repositories, Test Plans] + +## 11. Open Issues / Future Work + +\[Any known design limitations, unresolved issues, or planned future enhancements.] + +## 12. References + +\[List any other documents (e.g., requirements, ADRs, external specifications) referenced in this design document.] +```` diff --git a/content/templates/systems-engineering-management-plan.md b/content/templates/systems-engineering-management-plan.md new file mode 100644 index 0000000..aaa17a6 --- /dev/null +++ b/content/templates/systems-engineering-management-plan.md @@ -0,0 +1,209 @@ +--- +title: Systems engineering management plan +--- + +# Systems engineering management plan + +Use this template on a larger multi-discipline project to describe how the systems engineering will be run. It covers the life cycle, the reviews, the technical baselines and who is responsible for what. + +## Template + +```markdown +--- +title: System Engineering Management Plan (SEMP) - [System/Project Name] +status: Draft / Reviewed / Approved / Baselined / Evolving +date: YYYY-MM-DD (Date of Plan) +author: [Name or Role of Lead System Engineer/Program Manager] +version: 1.0 +related_documents: + - Product Vision - [Product Name] + - Architecture Vision - [System Name] + - Risk Register - [System Name] +--- + +# System Engineering Management Plan (SEMP): [System/Project Name] + +This document defines the technical and management processes for the system engineering effort of the [System/Project Name]. It outlines the overall approach to planning, organizing, and controlling the system's development, from concept to deployment and beyond, ensuring a cohesive, integrated, and disciplined execution aligned with the Keystone framework. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this System Engineering Management Plan (SEMP) is to: +* Define the system engineering processes and activities for the [System/Project Name]. +* Establish a common understanding of the system engineering approach among all stakeholders. +* Outline the roles, responsibilities, and organizational structure for system engineering. +* Describe how technical and management activities will be integrated and controlled. +* Ensure traceability and adherence to requirements throughout the system lifecycle. + +### 1.2 Scope + +This SEMP covers the system engineering activities for the [System/Project Name], encompassing its [e.g., "software, firmware, hardware, and mechanical components"]. It details the management and technical processes from [e.g., "concept definition"] through [e.g., "development, integration, verification, validation, and preparation for deployment"]. + +### 1.3 System Lifecycle Model + +[Describe how the Keystone framework's "V with a Wavy Iterative Point" will be applied as the system lifecycle model for this project. Explain how the structured "Outer V" and the agile "Wavy Iterative Point" will be managed.] + +### 1.4 Definitions and Acronyms + +[List any system engineering-specific terms, definitions, or acronyms used in this document.] + +## 2. System Overview + +### 2.1 System Description + +[Provide a high-level description of the system, its purpose, and its primary functions. Refer to the Product Vision or Project Vision.] + +### 2.2 Key Stakeholders + +[Identify the primary stakeholders for the system engineering effort and their interests.] + +* **[Stakeholder Group 1]:** [e.g., "Customers/End Users"] +* **[Stakeholder Group 2]:** [e.g., "Program/Project Management"] +* **[Stakeholder Group 3]:** [e.g., "Engineering Teams (Software, Firmware, Hardware, Mechanical)"] +* **[Stakeholder Group 4]:** [e.g., "Quality Assurance/Test"] +* **[Stakeholder Group 5]:** [e.g., "Operations/Deployment"] +* **[Stakeholder Group 6]:** [e.g., "Regulatory Bodies/Compliance"] + +## 3. Technical Management Processes + +### 3.1 Requirements Management + +[Describe the process for eliciting, analyzing, documenting, tracing, and managing changes to system requirements.] + +* **Templates Used:** [e.g., "System-Level Requirements Template," "Detailed Requirements Template"] +* **Traceability:** [e.g., "Requirements Traceability Matrix (RTM) will be maintained."] +* **Tools:** [e.g., "Jira with Requirements Management plugin."] + +### 3.2 Architectural Design Management + +[Describe the process for defining, documenting, and evolving the system's architecture.] + +* **Templates Used:** [e.g., "Architecture Vision Template," "Architecture Principles Template," "C4 Model Diagrams Template," "Software Design Specification Template"] +* **Decision Logging:** [e.g., "Architecture Decision Records (ADRs) will be used for key decisions."] +* **Reviews:** [e.g., "Regular architectural reviews (e.g., SRR, PDR, CDR)."] + +### 3.3 Interface Management + +[Describe the process for identifying, defining, documenting, and controlling interfaces between system elements and external systems.] + +* **Templates Used:** [e.g., "Interface Control Documents (ICDs) Template"] +* **Process:** [e.g., "Formal review and baselining of ICDs."] + +### 3.4 Configuration Management + +[Describe the plan for identifying, controlling, and tracking changes to all system artifacts (requirements, design, code, test cases, documentation, hardware baselines).] + +* **Version Control:** [e.g., "Git will be used for all digital artifacts."] +* **Baselining:** [e.g., "Formal baselines will be established at key lifecycle gates."] +* **Tools:** [e.g., "Git, PLM system for hardware baselines."] + +### 3.5 Data Management + +[Describe the approach for managing system data, including data definitions, storage, access, and data integrity across disciplines.] + +### 3.6 Verification and Validation (V&V) Management + +[Describe the overall strategy for verifying that the system is built correctly and validates that it meets user needs.] + +* **Templates Used:** [e.g., "Test Plans (High-Level & Detailed) Template"] +* **Test Levels:** [e.g., "Unit, Component, Integration, System, Acceptance Testing."] +* **Traceability:** [e.g., "RTM will link requirements to test cases."] + +### 3.7 Risk Management + +[Describe the process for identifying, analyzing, mitigating, and monitoring technical and program risks.] + +* **Templates Used:** [e.g., "Risk Register Template"] +* **Process:** [e.g., "Regular risk reviews integrated into planning cycles."] + +### 3.8 Technical Performance Measurement (TPM) + +[Describe how key technical parameters will be measured and tracked to ensure performance objectives are being met.] + +* **Key Metrics:** [e.g., "System throughput, latency, power consumption, MTBF."] +* **Reporting:** [e.g., "Performance dashboards."] + +### 3.9 Decision Management + +[Describe the process for making, documenting, and communicating significant technical and project decisions.] + +* **Templates Used:** [e.g., "Decision Log Template," "Architecture Decision Record (ADR) Template"] + +## 4. Process Management + +### 4.1 Planning and Scheduling + +[Describe the high-level planning approach, including how iterative development cycles (from Keystone's "Wavy Iterative Point") integrate with overall program milestones.] + +### 4.2 Quality Management + +[Describe how quality will be assured throughout the system engineering lifecycle, including quality gates, reviews, and audits.] + +### 4.3 Continuous Improvement + +[Describe the mechanisms for continuous learning and process improvement.] + +* **Templates Used:** [e.g., "Lessons Learned Template," "Post-Mortem / Incident Review Template"] +* **Process:** [e.g., "Regular retrospectives, blameless post-mortems."] + +### 4.4 Tooling Strategy + +[Describe the overall strategy for selecting, integrating, and managing the tools used for system engineering activities.] + +* **Reference:** [e.g., "Tooling & Ecosystem Guides (Future Content)"] + +## 5. Organizational and Resource Management + +### 5.1 Roles and Responsibilities + +[Define the key roles within the system engineering organization and their responsibilities.] + +* **Chief System Engineer:** [Overall technical authority, architectural integrity.] +* **System Architects:** [Architectural design, cross-cutting concerns.] +* **System Engineers:** [Requirements analysis, V&V planning.] +* **Discipline Leads (Software, Hardware, Firmware, Mechanical):** [Detailed design, implementation.] + +### 5.2 Organizational Structure + +[Describe how the system engineering functions are organized and how they interact with development teams (e.g., alignment with Team Topologies concepts).] + +### 5.3 Resource Planning + +[Outline the plan for allocating and managing human resources, tools, facilities, and budget for system engineering activities.] + +## 6. Integration Management + +### 6.1 System Integration Strategy + +[Describe the approach for integrating different system elements (software, firmware, hardware) into a cohesive whole.] + +* **Phased Integration:** [e.g., "Incremental integration stages."] +* **Integration Environments:** [e.g., "Dedicated integration labs, HIL setups."] + +### 6.2 Supplier/Subcontractor Management + +[Describe how system engineering activities related to external suppliers or subcontractors will be managed.] + +## 7. Reviews and Audits + +### 7.1 Technical Reviews + +[Define the key technical reviews that will be conducted throughout the lifecycle.] + +* **System Requirements Review (SRR):** [Purpose, participants, timing.] +* **Preliminary Design Review (PDR):** +* **Critical Design Review (CDR):** +* **Test Readiness Review (TRR):** +* **Formal Qualification Review (FQR):** + +### 7.2 Audits + +[Describe any planned functional configuration audits (FCAs) or physical configuration audits (PCAs), and compliance audits.] + +## 8. References + +[List any other relevant documents (e.g., Program Management Plan, Quality Assurance Plan, Security Plan) referenced in this SEMP.] +``` diff --git a/content/templates/team-working-agreement.md b/content/templates/team-working-agreement.md new file mode 100644 index 0000000..f50232f --- /dev/null +++ b/content/templates/team-working-agreement.md @@ -0,0 +1,161 @@ +--- +title: Team working agreement +--- + +# Team working agreement + +Use this template when a team forms, or changes, to agree how it works: hours, meetings, communication, reviews and how it makes decisions. The team writes it together and changes it when it stops matching what they do. + +## Template + +```markdown +--- +title: Team Working Agreement - [Team Name] +status: Draft / Reviewed / Agreed / Evolving +date: YYYY-MM-DD +author: [Team Members' Names/Team Lead] +version: 1.0 +--- + +# Team Working Agreement: [Team Name] + +This document outlines the working agreements for the [Team Name]. It's a living document created by the team, for the team, to ensure clear communication, foster collaboration, and maintain a consistent approach to delivering value. We commit to upholding these agreements and reviewing them regularly to ensure they remain effective. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Team Working Agreement is to: +* Define our shared understanding of how we collaborate effectively. +* Establish clear criteria for when work is "ready" to be started. +* Articulate a comprehensive "Definition of Done" for our deliverables. +* Outline our expectations for user stories and other work items. +* Promote transparency, consistency, and continuous improvement within the team. + +### 1.2 Scope + +This agreement applies to all members of the [Team Name] and governs our daily working practices, ceremonies, and deliverable quality. + +### 1.3 Review and Evolution + +This is a living document. We will review this agreement [e.g., "at least quarterly," "during every retrospective"] and update it as needed to reflect our evolving practices and learnings. + +## 2. Our Team Values & Principles + +[Optional: Briefly state any core values or principles that guide your team's work, e.g., "Transparency," "Mutual Respect," "Continuous Learning," "Customer Focus."] + +* [Value/Principle 1] +* [Value/Principle 2] + +## 3. Team Collaboration & Communication + +### 3.1 Synchronization & Collaboration Points + +[Define the purpose, frequency, and expected participation for your team's essential synchronization and collaboration points, emphasizing flexibility over rigid ceremonies.] + +* **Daily Sync / Stand-up:** + * *Purpose:* Quick check-in to synchronize, identify immediate blockers, and align on daily priorities. + * *Frequency:* Daily, [Time] - [Time] [Timezone] (or "as needed via chat for distributed teams"). + * *Duration:* ~15 minutes (time-boxed). + * *Expectations:* Be concise, focus on progress, plans, and impediments. +* **Iteration Planning / Work Prioritization:** + * *Purpose:* Align on upcoming work, clarify scope, and ensure readiness for the next iteration. + * *Frequency:* [e.g., "Start of each iteration (e.g., bi-weekly)"] + * *Duration:* [e.g., "Flexible, typically 1-2 hours, or broken into smaller sessions."] + * *Expectations:* Active participation in understanding and sizing work items. +* **Review / Demo:** + * *Purpose:* Showcase completed work to stakeholders, gather feedback, and validate value delivery. + * *Frequency:* [e.g., "End of each iteration (e.g., bi-weekly)"] + * *Duration:* [e.g., "Flexible, typically 30-60 minutes."] + * *Expectations:* Prepare clear demonstrations, focus on outcomes, engage with feedback. +* **Retrospective / Continuous Improvement:** + * *Purpose:* Reflect on our process, identify what's working well, and pinpoint areas for improvement. + * *Frequency:* [e.g., "End of each iteration (e.g., bi-weekly)"] + * *Duration:* [e.g., "Flexible, typically 1 hour."] + * *Expectations:* Safe environment for honest feedback, focus on actionable improvements for our ways of working. + +### 3.2 Communication Channels + +[How will the team communicate for different purposes?] + +* **Urgent/Blockers:** [e.g., "Dedicated Slack/Teams channel," "Direct call to relevant person."] +* **General Discussion/Questions:** [e.g., "Team Slack/Teams channel."] +* **Longer Discussions/Decisions:** [e.g., "Scheduled meeting," "Confluence/Wiki page for documentation."] +* **Documentation:** [e.g., "All significant decisions and designs documented in Markdown/Confluence."] + +### 3.3 Conflict Resolution + +[How will the team address disagreements or conflicts?] + +* [e.g., "Address directly and respectfully with the individuals involved first."] +* [e.g., "If unresolved, escalate to the team lead/facilitator."] +* [e.g., "Use a structured discussion framework (e.g., ADRs for technical decisions)."] + +## 4. Definition of Ready (DoR) + +A work item (e.g., User Story, Task, Bug) is "Ready" to be pulled into a sprint/iteration when it meets the following criteria: + +* **Clear Value:** The business value or problem it solves is clearly understood. +* **Well-Defined:** The scope is clear, unambiguous, and understood by the team. +* **Estimable:** The team can provide a reasonable estimate of effort. +* **Testable:** Clear acceptance criteria are defined, allowing for verification. +* **Small Enough:** It can be completed within a single sprint/iteration. +* **Dependencies Identified:** All external dependencies are known and accounted for (e.g., APIs, hardware, other teams). +* **Technical Spikes/Research Done:** Any significant technical unknowns have been explored. +* **No Obvious Blockers:** No immediate impediments prevent starting work. +* **Linked to Higher-Level Requirements:** Traceability to System-Level Requirements is established (if applicable). + +## 5. Definition of Done (DoD) + +A work item is "Done" when it meets *all* of the following criteria: + +* **Code Complete:** All necessary code has been written. +* **Unit Tested:** All new/changed code has accompanying unit tests, and they pass. +* **Code Reviewed:** Code has been reviewed by at least one other team member and all feedback addressed. +* **Integrated:** Code has been successfully integrated into the main branch/repository. +* **Automated Tests Pass:** All relevant automated integration/system tests pass. +* **Acceptance Criteria Met:** All defined acceptance criteria for the work item have been verified (manual or automated). +* **Peer Tested / QA Approved:** (If applicable) Peer testing or formal QA sign-off has occurred. +* **Documentation Updated:** + * Relevant design documents (e.g., SDS, C4 diagrams) updated. + * API documentation updated. + * User-facing documentation (if applicable) drafted/updated. + * ADRs created for significant decisions. +* **Deployed to [Environment]:** Successfully deployed to a designated testing/staging environment. +* **Performance/Security Reviewed:** (If applicable) Basic performance and security checks passed. +* **No Known Critical Bugs:** No critical or major defects are outstanding for this feature. +* **Monitoring/Alerting Configured:** (If applicable) Necessary monitoring and alerting are in place. +* **Reviewed by Product Owner/Stakeholder:** (If applicable) Product Owner or relevant stakeholder has reviewed and accepted the work. + +## 6. User Story / Work Item Requirements + +When creating or refining a user story or other work item, it should ideally contain the following information: + +* **Title:** Concise and descriptive. +* **Description:** + * **As a [User Role], I want to [Action], so that [Benefit/Goal].** (for user stories) + * Or a clear statement of the problem/task for bugs/technical tasks. +* **Acceptance Criteria:** Clear, verifiable conditions that define when the story is complete. (e.g., Gherkin format: `Given-When-Then`). +* **Dependencies:** Any internal or external dependencies. +* **Effort Estimate:** (e.g., Story Points, T-shirt size, ideal days). +* **Priority:** (e.g., MoSCoW: Must, Should, Could, Won't; or High, Medium, Low). +* **Links:** To related requirements, design documents, bugs, or other stories. +* **Assigned To:** The individual(s) responsible for the work. +* **Discussion/Comments:** Space for ongoing dialogue and clarification. + +## 7. Our Definition of "Ready for Review" (Code/Design) + +Before submitting code or a design for review, ensure it meets these criteria: + +* [e.g., "Passes all automated tests (unit, integration)."] +* [e.g., "Adheres to coding standards/style guide."] +* [e.g., "Includes necessary comments and documentation."] +* [e.g., "No obvious compilation/build errors."] +* [e.g., "Design rationale clearly articulated (if a design review)."] + +## 8. References + +[List any other documents (e.g., company coding standards, architectural principles) referenced in this agreement.] +``` diff --git a/content/templates/technical-debt-register.md b/content/templates/technical-debt-register.md new file mode 100644 index 0000000..1b7de36 --- /dev/null +++ b/content/templates/technical-debt-register.md @@ -0,0 +1,146 @@ +--- +title: Technical debt register +--- + +# Technical debt register + +Use this template to record known technical debt, what it costs, and the plan to pay it down. Debt that is written down can be weighed against features; debt that is not just accumulates. + +## Template + +```markdown +--- +title: Technical Debt Register / Management Plan - [System/Product/Team Name] +status: Active / Archived +date: YYYY-MM-DD +owner: [Name or Role of Technical Lead/Architect/Team] +version: 1.0 +--- + +# Technical Debt Register / Management Plan: [System/Product/Team Name] + +This document serves as a central register for identified technical debt within the [System/Product/Team Name]. It provides a structured approach to understand, prioritize, and manage the remediation of technical debt, ensuring the long-term health, maintainability, and evolvability of our systems. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this Technical Debt Register and Management Plan is to: +* Provide a transparent record of identified technical debt items. +* Categorize and assess the impact and urgency of each debt. +* Facilitate informed decision-making regarding technical debt remediation. +* Assign ownership and track progress of remediation efforts. +* Ensure technical debt is managed proactively rather than accumulating uncontrollably. + +### 1.2 Scope + +This document covers technical debt related to [briefly describe the scope, e.g., "the software components of Product X," "the firmware for Device Y," "the entire Z System"]. + +### 1.3 Definitions + +* **Technical Debt:** Design or implementation constructs that are expedient in the short term but set up a technical context that will cost more to change later. It's a metaphor for the trade-off between short-term gains and long-term consequences. +* **Principal:** The initial cost of the expedient solution. +* **Interest:** The ongoing cost of living with the technical debt (e.g., slower development, more bugs, higher operational costs). +* **Remediation:** The process of paying down technical debt. + +### 1.4 Acronyms + +[List any relevant acronyms.] + +## 2. Technical Debt Register Log + +This log tracks individual technical debt items. + +| TD ID | Title (Brief) | Description | Category | Origin/Reason | Impact (Current) | Urgency (Current) | Owner | Status | Date Identified | Last Updated | Remediation Plan | Estimated Effort | +|-------|---------------|-------------|----------|---------------|------------------|-------------------|-------|--------|-----------------|--------------|------------------|------------------| +| **TD-001** | Hardcoded API endpoint in Frontend | Frontend API URL is hardcoded, requires manual change for environments. | Code Debt | Expedient for initial deployment. | High (deployment risk, error prone) | High (frequent deployments) | Jane Doe | Open | YYYY-MM-DD | YYYY-MM-DD | Externalize config to env vars. | 2 days | +| **TD-002** | Legacy Database Schema | `users` table lacks proper indexing, slow queries. | Design Debt | Historical; pre-dates current scale. | High (performance bottleneck) | Medium (intermittent user complaints) | John Smith | Open | YYYY-MM-DD | YYYY-MM-DD | Add B-tree indexes to `email` and `last_login` columns. | 5 days | +| **TD-003** | Manual Firmware Update Process | Firmware updates require manual flashing via USB. | Process Debt | Initial low volume; no automation built. | Medium (time consuming, error prone) | Low (infrequent updates) | Alice Brown | Open | YYYY-MM-DD | YYYY-MM-DD | Research OTA update solution. | 10 days (research) | +| **TD-004** | Outdated Library: `libxyz` | `libxyz` (v1.0) has known security vulnerabilities. | Dependency Debt | Not updated during previous release. | Critical (security risk) | High (critical vulnerability) | Bob White | Open | YYYY-MM-DD | YYYY-MM-DD | Upgrade `libxyz` to v2.1. | 3 days | +| **TD-005** | Lack of Unit Tests for Payment Module | Critical payment logic has no unit test coverage. | Test Debt | Tight deadline for MVP. | High (bug risk) | Medium (critical path) | Carol Green | Open | YYYY-MM-DD | YYYY-MM-DD | Write unit tests for `processPayment` function. | 4 days | + +## 3. Technical Debt Categories + +[Define common categories for technical debt to help with analysis and reporting.] + +* **Code Debt:** Issues within the code itself (e.g., duplicated code, complex logic, poor readability, lack of comments, hardcoding). +* **Design Debt:** Suboptimal architectural or system design choices (e.g., tight coupling, poor modularity, inadequate scalability). +* **Test Debt:** Insufficient or poor quality automated tests (e.g., low coverage, brittle tests, lack of integration tests). +* **Documentation Debt:** Missing, outdated, or unclear documentation (e.g., design docs, API specs, user manuals). +* **Process Debt:** Inefficient or manual development/operations processes (e.g., manual deployments, slow build times, lack of CI/CD). +* **Dependency Debt:** Outdated or insecure third-party libraries/frameworks. +* **Knowledge Debt:** Critical knowledge residing with only a few individuals, leading to single points of failure. +* **Environmental Debt:** Issues with development, testing, or production environments (e.g., inconsistent environments, outdated tools). + +## 4. Technical Debt Assessment Criteria + +### 4.1 Impact Scale + +[Similar to Risk Impact, but focused on the consequences of *not* addressing the debt.] + +| Level | Description | Impact on Development Velocity | Impact on Quality/Bugs | Impact on Operations | Impact on Business/Risk | +|-------|-------------|--------------------------------|------------------------|----------------------|-------------------------| +| 1 | Low | Minor slowdown | Infrequent minor bugs | Negligible | Negligible | +| 2 | Medium | Noticeable slowdown | Occasional bugs | Minor operational issues | Minor business impact | +| 3 | High | Significant slowdown | Frequent bugs, critical issues | Significant operational issues | Significant business risk | + +### 4.2 Urgency Scale + +[How quickly does this debt need to be addressed?] + +| Level | Description | Action Timeframe | +|-------|-------------|------------------| +| 1 | Low | Long-term (next quarter/year) | +| 2 | Medium | Mid-term (next sprint/release) | +| 3 | High | Short-term (this sprint/hotfix) | + +### 4.3 Prioritization (Optional: Urgency x Impact) + +[Define how the combined Urgency and Impact determine the prioritization of debt items.] + +## 5. Technical Debt Management Process + +### 5.1 Identification + +[How is technical debt identified? (e.g., code reviews, architectural reviews, static analysis tools, bug reports, retrospectives, developer feedback).] + +### 5.2 Assessment + +[How are identified debt items evaluated for category, impact, and urgency?] + +### 5.3 Prioritization + +[How are debt items prioritized relative to new feature development and other work?] +* **Dedicated Time:** Allocate a percentage of each sprint/iteration (e.g., 10-20%) for technical debt remediation. +* **"Fix as You Go":** Encourage developers to address minor debt in areas they are already working on. +* **Strategic Sprints/Releases:** Dedicate entire sprints or releases to major refactoring or debt payoff initiatives. +* **Product Backlog Integration:** Treat technical debt items as regular backlog items, prioritized by the Product Owner in collaboration with the engineering team. + +### 5.4 Remediation Planning + +[How are remediation plans developed? (e.g., specific tasks, estimated effort, dependencies).] + +### 5.5 Monitoring & Reporting + +[How will technical debt be continuously tracked, reviewed, and reported to stakeholders? (e.g., dashboards, regular reviews).] + +## 6. Roles and Responsibilities + +* **Technical Debt Owner:** Responsible for driving the remediation of specific debt items. +* **Team/Lead:** Responsible for identifying debt, contributing to remediation, and updating the register. +* **Architect/Engineering Manager:** Overall responsibility for the Technical Debt Register, facilitating prioritization discussions, and advocating for debt repayment. +* **Product Owner:** Involved in prioritizing technical debt against new features, understanding its impact on delivery velocity and product quality. + +## 7. Review Schedule + +[How often will the Technical Debt Register be reviewed and by whom?] + +* **Frequency:** [e.g., "Bi-weekly during sprint reviews," "Monthly architectural syncs," "Quarterly strategic planning."] +* **Attendees:** [e.g., "Development team," "Technical Leads & Product Owner," "Engineering Leadership."] + +## 8. References + +[List any other documents (e.g., Architecture Vision, ADRs, coding standards) referenced in this document.] +``` diff --git a/content/templates/test-plan.md b/content/templates/test-plan.md new file mode 100644 index 0000000..17fa696 --- /dev/null +++ b/content/templates/test-plan.md @@ -0,0 +1,182 @@ +--- +title: Test plan +--- + +# Test plan + +Use this template to plan how a product, system or component will be tested: the scope, the levels of testing, the environments, and the entry and exit criteria. + +## Template + +```markdown +--- +title: Test Plan - [Product/System/Component Name] ([High-Level/Detailed]) +status: Draft / Reviewed / Approved / Executing / Completed +date: YYYY-MM-DD +author: [Name or Role of Test Lead/Engineer] +version: 1.0 +test_level: High-Level / System / Integration / Component / Unit +related_requirements_doc: [Link to System-Level Requirements or Detailed Requirements Doc] +--- + +# Test Plan: [Product/System/Component Name] ([High-Level/Detailed]) + +This document outlines the test strategy, scope, objectives, and approach for verifying and validating the [Product/System/Component Name]. It serves as a guide for all testing activities, ensuring that the product meets its specified requirements and quality attributes. + +--- + +## 1. Introduction + +### 1.1 Purpose + +The purpose of this test plan is to define the overall strategy and approach for testing the [Product/System/Component Name]. It specifies the scope of testing, objectives, test levels, roles and responsibilities, and the resources required. + +### 1.2 Scope of Testing + +[Clearly define what will be tested and what will not be tested. Be specific about features, modules, interfaces, and environments.] + +* **In Scope:** + * [Feature/Module 1] + * [Interface X] + * [Environment Y] +* **Out of Scope:** + * [Feature/Module A (and why, e.g., "tested by another team," "deferred to next phase")] + +### 1.3 Objectives + +What are the specific goals of this testing effort? + +* [e.g., "Verify all high-level functional requirements (SYS-FUN-XXX) are met."] +* [e.g., "Validate that the system meets specified performance criteria (SYS-NFR-PERF-YYY)."] +* [e.g., "Identify and report defects to ensure product quality."] +* [e.g., "Ensure compliance with safety standard Z."] + +### 1.4 Test Levels + +[Specify the test levels covered by this plan (e.g., Unit, Component, Integration, System, Acceptance).] + +## 2. Test Strategy + +### 2.1 Overall Test Approach + +[Describe the general methodology for testing (e.g., "risk-based testing," "exploratory testing," "requirement-driven testing"). How will automation be leveraged?] + +### 2.2 Test Environment + +[Describe the hardware, software, network, and data configurations required for testing.] + +* **Hardware:** [e.g., "Target Device Rev X, Test Rig Z"] +* **Software:** [e.g., "Firmware Version A, Application Version B, OS Version C"] +* **Network:** [e.g., "Simulated IoT network, Isolated lab environment"] +* **Data:** [e.g., "Test data sets, database configurations"] + +### 2.3 Test Data Management + +[How will test data be created, managed, and maintained?] + +### 2.4 Test Tools + +[List the tools to be used for test management, execution, automation, and defect tracking.] + +* **Test Management:** [e.g., Jira with Xray, Azure DevOps Test Plans] +* **Automation:** [e.g., Pytest, Selenium, custom hardware test harnesses] +* **Defect Tracking:** [e.g., Jira, Azure DevOps] +* **Performance/Load Testing:** [e.g., JMeter, LoadRunner] + +## 3. Test Deliverables + +[What documents, reports, or artifacts will be produced as part of this testing effort?] + +* **Test Cases:** [e.g., "Detailed test cases (manual/automated)"] +* **Test Execution Logs:** +* **Defect Reports:** +* **Test Summary Report:** +* **Traceability Matrix (updated):** + +## 4. Roles and Responsibilities + +[Identify the key roles involved in testing and their responsibilities.] + +* **Test Lead:** [Overall test strategy, planning, reporting] +* **Test Engineers:** [Test case design, execution, defect reporting] +* **Developers:** [Unit testing, defect fixing] +* **Product Owner:** [Acceptance criteria definition, UAT] + +## 5. Test Cases (High-Level Plan) / Detailed Test Cases (Detailed Plan) + +### 5.1 High-Level Test Scenarios (for High-Level Test Plan) + +[Describe major test scenarios or functional areas to be covered.] + +* **Scenario 1: [Name of Scenario]** + * *Objective:* [What is this scenario trying to achieve?] + * *Associated Requirements:* [e.g., SYS-FUN-001, SYS-NFR-PERF-005] + * *Approach:* [Briefly describe the testing approach (e.g., "Automated end-to-end test," "Manual exploratory test").] +* **Scenario 2: [Name of Scenario]** + * ... + +### 5.2 Detailed Test Cases (for Detailed Test Plan) + +[For detailed plans, list specific test cases. Use a structured format.] + +* **Test Case ID:** [TC-XXX-001] +* **Test Case Name:** [Verify User Login with Valid Credentials] +* **Description:** [Ensure a registered user can successfully log in to the system.] +* **Pre-conditions:** [User account exists and is active.] +* **Steps:** + 1. [Navigate to login page.] + 2. [Enter valid username "testuser".] + 3. [Enter valid password "password123".] + 4. [Click "Login" button.] +* **Expected Result:** [User is redirected to the dashboard, and a "Welcome, testuser!" message is displayed.] +* **Priority:** [High/Medium/Low] +* **Type:** [Functional/Non-functional/Safety/Compliance] +* **Automation Status:** [Manual/Automated] +* **Trace to Requirement(s):** [e.g., COMP-FUN-005, SYS-FUN-010] + +## 6. Defect Management Process + +[How will defects be identified, reported, tracked, and retested?] + +* **Defect Lifecycle:** [e.g., New -> Open -> Fixed -> Retest -> Closed] +* **Severity Levels:** [e.g., Critical, Major, Minor, Cosmetic] +* **Priority Levels:** [e.g., P1 (Blocker), P2 (High), P3 (Medium), P4 (Low)] + +## 7. Reporting + +[How will test progress and results be reported to stakeholders?] + +* **Daily/Weekly Status Reports:** +* **Test Summary Reports:** [At key milestones or end of test cycles.] +* **Metrics:** [e.g., Test case execution rate, pass/fail rate, defect discovery rate, test coverage.] + +## 8. Exit Criteria + +[What conditions must be met for this testing phase to be considered complete?] + +* [e.g., "All critical and major defects are resolved and retested successfully."] +* [e.g., "All high-priority test cases have passed."] +* [e.g., "Test coverage targets (e.g., 80% code coverage) are met (if applicable)."] +* [e.g., "Stakeholder sign-off on test results."] + +## 9. Risks and Contingencies + +[What are the potential risks to this test plan, and what are the contingency plans?] + +* **Risk 1:** [e.g., "Test environment instability."] + * *Contingency:* [e.g., "Allocate dedicated environment support, have backup environment."] +* **Risk 2:** [e.g., "Delay in hardware delivery."] + * *Contingency:* [e.g., "Utilize simulation/emulation, prioritize software development."] + +## 10. Approvals + +[Sign-off section for key stakeholders.] + +* **[Role 1]:** [Name, Date] +* **[Role 2]:** [Name, Date] +* ... + +## 11. References + +[List any other documents (e.g., requirements, design, architecture) or standards referenced in this test plan.] +``` diff --git a/py-doc.yml b/py-doc.yml index 6741614..98b7aab 100644 --- a/py-doc.yml +++ b/py-doc.yml @@ -14,27 +14,150 @@ outputs: # in site/docs/blog/ for MkDocs Material's blog plugin. documents: guidelines: - # Guidelines. The section's own pull request lists the rest of its files here. + # Guidelines (from EngineeringProcess): the overview and core rules, the 26 guidelines in number order, + # then the principles, adoption, personal-project, agent and checklist pages. sources: - content/guidelines/index.md + - content/guidelines/core-rules.md + - content/guidelines/guidelines-governance.md + - content/guidelines/values-and-ways-of-working.md + - content/guidelines/quality-policy-and-attributes.md + - content/guidelines/documentation-as-code.md + - content/guidelines/ai-agent-usage.md + - content/guidelines/requirements-and-traceability.md + - content/guidelines/architecture-principles.md + - content/guidelines/decision-records.md + - content/guidelines/interfaces-and-co-design.md + - content/guidelines/components-reuse-and-variants.md + - content/guidelines/technology-and-third-party.md + - content/guidelines/coding-principles.md + - content/guidelines/coding-standard.md + - content/guidelines/version-control-and-versioning.md + - content/guidelines/pull-requests-and-review.md + - content/guidelines/pipelines-and-quality-gates.md + - content/guidelines/testing-principles-and-strategy.md + - content/guidelines/defects-and-technical-debt.md + - content/guidelines/logging-debug-and-diagnostics.md + - content/guidelines/observability-and-telemetry.md + - content/guidelines/release-update-and-production.md + - content/guidelines/security-and-data-protection.md + - content/guidelines/safety-regulatory-and-certification.md + - content/guidelines/planning-delivery-and-improvement.md + - content/guidelines/research-and-prototyping.md + - content/guidelines/competence-tooling-and-onboarding.md + - content/guidelines/principles-catalogue.md + - content/guidelines/business-principles.md + - content/guidelines/organisation-adoption.md + - content/guidelines/personal-projects.md + - content/guidelines/agent-rules.md + - content/guidelines/checklists.md outputs: {docx: {}, pdf: {}} standards: - # Standards. The section's own pull request lists the rest of its files here. + # Standards (from technical-documentation's specialised GU- documents): coding, design and interfaces, + # then connected devices. sources: - content/standards/index.md + - content/standards/c-coding-standard.md + - content/standards/misra-c.md + - content/standards/python.md + - content/standards/software-architecture-principles.md + - content/standards/api-design.md + - content/standards/unique-identifiers.md + - content/standards/lwm2m-objects.md + - content/standards/iot-device-architecture-principles.md + - content/standards/iot-battery-budgeting.md + - content/standards/constrained-binary-payloads.md + - content/standards/iot-edge-offline-storage.md + - content/standards/device-serialisation-and-factory-test.md outputs: {docx: {}, pdf: {}} guides: - # Guides. The section's own pull request lists the rest of its files here. + # Guides: each guide is a chapter, and a guide's numbered files are its sections (heading_offset 1). + # On the site each guide keeps its folder. sources: - content/guides/index.md + - {file: content/guides/c4-model/index.md, page_break: true} + - {file: content/guides/c4-model/01-c4-model-foundations.md, heading_offset: 1} + - {file: content/guides/c4-model/02-diagrams-as-code-tooling.md, heading_offset: 1} + - {file: content/guides/c4-model/03-system-context.md, heading_offset: 1} + - {file: content/guides/c4-model/04-container-diagrams.md, heading_offset: 1} + - {file: content/guides/c4-model/05-component-diagrams.md, heading_offset: 1} + - {file: content/guides/c4-model/06-structurizr-dsl-reference.md, heading_offset: 1} + - {file: content/guides/c4-model/07-deployment-and-dynamics.md, heading_offset: 1} + - {file: content/guides/cybersecurity/index.md, page_break: true} + - {file: content/guides/cybersecurity/01-threat-modeling-foundations.md, heading_offset: 1} + - {file: content/guides/cybersecurity/02-hardware-and-silicon-security.md, heading_offset: 1} + - {file: content/guides/cybersecurity/03-transport-and-network-security.md, heading_offset: 1} + - {file: content/guides/cybersecurity/04-secure-firmware-updates-ota.md, heading_offset: 1} + - {file: content/guides/cybersecurity/05-standards-and-compliance.md, heading_offset: 1} + - {file: content/guides/iot-endpoint-and-gateway-architecture/index.md, page_break: true} + - {file: content/guides/iot-endpoint-and-gateway-architecture/01-device-tiers-and-hardware-architecture.md, heading_offset: 1} + - {file: content/guides/iot-endpoint-and-gateway-architecture/02-connectivity-and-protocol-tradeoffs.md, heading_offset: 1} + - {file: content/guides/iot-endpoint-and-gateway-architecture/03-power-architecture-and-duty-cycling.md, heading_offset: 1} + - {file: content/guides/iot-endpoint-and-gateway-architecture/04-edge-storage-and-local-resilience.md, heading_offset: 1} + - {file: content/guides/iot-endpoint-and-gateway-architecture/05-edge-intelligence-and-processing.md, heading_offset: 1} + - {file: content/guides/iot-fleet-operations/index.md, page_break: true} + - {file: content/guides/iot-fleet-operations/01-zero-touch-device-onboarding.md, heading_offset: 1} + - {file: content/guides/iot-fleet-operations/02-scalable-telemetry-ingestion.md, heading_offset: 1} + - {file: content/guides/iot-fleet-operations/03-digital-twins-and-state-reconciliation.md, heading_offset: 1} + - {file: content/guides/iot-fleet-operations/04-fleet-scale-ota-campaigns.md, heading_offset: 1} + - {file: content/guides/iot-fleet-operations/05-fleet-observability-and-remote-diagnostics.md, heading_offset: 1} + - {file: content/guides/hil-and-automated-testing/index.md, page_break: true} + - {file: content/guides/hil-and-automated-testing/01-the-iot-testing-pyramid.md, heading_offset: 1} + - {file: content/guides/hil-and-automated-testing/02-host-based-firmware-testing.md, heading_offset: 1} + - {file: content/guides/hil-and-automated-testing/03-qemu-virtual-hardware-simulation.md, heading_offset: 1} + - {file: content/guides/hil-and-automated-testing/04-hil-bench-design-and-automation.md, heading_offset: 1} + - {file: content/guides/hil-and-automated-testing/05-network-simulation-and-chaos-testing.md, heading_offset: 1} + - {file: content/guides/scaling-architecture/index.md, page_break: true} + - {file: content/guides/scaling-architecture/01-advice-process.md, heading_offset: 1} + - {file: content/guides/scaling-architecture/02-alignment-mechanisms.md, heading_offset: 1} + - {file: content/guides/git/index.md, page_break: true} + - {file: content/guides/git/01-how-git-works.md, heading_offset: 1} + - {file: content/guides/git/02-branching-strategies.md, heading_offset: 1} + - {file: content/guides/git/03-choosing-a-version-control-system.md, heading_offset: 1} + - {file: content/guides/ways-of-working/index.md, page_break: true} + - {file: content/guides/ways-of-working/01-working-agreements.md, heading_offset: 1} + - {file: content/guides/ways-of-working/02-onboarding.md, heading_offset: 1} + - {file: content/guides/ways-of-working/03-development-process.md, heading_offset: 1} + - {file: content/guides/ways-of-working/04-ci-cd.md, heading_offset: 1} + - {file: content/guides/ways-of-working/05-documentation-standards.md, heading_offset: 1} + - {file: content/guides/ways-of-working/06-architecture-and-design-documentation.md, heading_offset: 1} + - {file: content/guides/ways-of-working/07-documentation-and-modelling-tools.md, heading_offset: 1} + - {file: content/guides/ways-of-working/08-metrics-and-feedback.md, heading_offset: 1} + - {file: content/guides/decision-records.md, page_break: true} + - {file: content/guides/software-design-specification.md, page_break: true} + - {file: content/guides/high-level-design.md, page_break: true} + - {file: content/guides/architecture-review-board.md, page_break: true} + - {file: content/guides/agile-requirements-and-user-stories.md, page_break: true} + - {file: content/guides/code-review.md, page_break: true} + - {file: content/guides/defect-management-and-debugging.md, page_break: true} + - {file: content/guides/deep-dives/index.md, page_break: true} + - {file: content/guides/deep-dives/cbor.md, heading_offset: 1} + - {file: content/guides/deep-dives/coap.md, heading_offset: 1} + - {file: content/guides/deep-dives/lwm2m.md, heading_offset: 1} + - {file: content/guides/deep-dives/nb-iot-and-cellular.md, heading_offset: 1} + - {file: content/guides/deep-dives/certificates-and-keys.md, heading_offset: 1} + - {file: content/guides/deep-dives/rtos-internals.md, heading_offset: 1} + - {file: content/guides/deep-dives/sysvars.md, heading_offset: 1} + - {file: content/guides/deep-dives/misra-c.md, heading_offset: 1} + - {file: content/guides/deep-dives/static-analysis.md, heading_offset: 1} + - {file: content/guides/deep-dives/git-internals.md, heading_offset: 1} + - {file: content/guides/deep-dives/docs-diagrams-and-architecture-as-code.md, heading_offset: 1} outputs: {docx: {}, pdf: {}} handbook: - # Architecture Handbook. The section's own pull request lists the rest of its files here. + # Architecture Handbook (technical-documentation's handbook): the overview, then eight chapters. sources: - content/handbook/index.md + - {file: content/handbook/01-introduction-to-architecture.md, page_break: true} + - {file: content/handbook/02-architecture-vision-and-principles.md, page_break: true} + - {file: content/handbook/03-architecture-patterns.md, page_break: true} + - {file: content/handbook/04-api-design-and-management.md, page_break: true} + - {file: content/handbook/05-cloud-architecture.md, page_break: true} + - {file: content/handbook/06-iot-and-embedded-systems.md, page_break: true} + - {file: content/handbook/07-security-and-standards.md, page_break: true} + - {file: content/handbook/08-software-development-best-practices.md, page_break: true} outputs: {docx: {}, pdf: {}} primers: @@ -121,16 +244,38 @@ documents: outputs: {docx: {}, pdf: {}} glossary: - # Glossary: the index, then one file per letter group. Each term is a ### heading, so it has - # its own anchor (content/glossary/redirects.tsv maps the old one-page-per-term URLs to them). + # Glossary: the index, then one page per letter, then Numbers and Symbols. Each term is a ## heading with + # its own anchor (content/glossary/redirects.tsv maps older URLs, including the letter-group pages, to them). sources: - content/glossary/index.md - - content/glossary/a-c.md - - content/glossary/d-f.md - - content/glossary/g-k.md - - content/glossary/l-n.md - - content/glossary/o-r.md - - content/glossary/s-z.md + - content/glossary/a.md + - content/glossary/b.md + - content/glossary/c.md + - content/glossary/d.md + - content/glossary/e.md + - content/glossary/f.md + - content/glossary/g.md + - content/glossary/h.md + - content/glossary/i.md + - content/glossary/j.md + - content/glossary/k.md + - content/glossary/l.md + - content/glossary/m.md + - content/glossary/n.md + - content/glossary/o.md + - content/glossary/p.md + - content/glossary/q.md + - content/glossary/r.md + - content/glossary/s.md + - content/glossary/t.md + - content/glossary/u.md + - content/glossary/v.md + - content/glossary/w.md + - content/glossary/x.md + - content/glossary/y.md + - content/glossary/z.md + - content/glossary/numbers.md + - content/glossary/symbols.md outputs: {docx: {}, pdf: {}} templates: @@ -145,29 +290,99 @@ documents: - content/templates/code-review-checklist.md - content/templates/debugging-checklist.md - content/templates/defect-report-checklist.md + - content/templates/project-vision.md + - content/templates/architecture-vision.md + - content/templates/architecture-principles.md + - content/templates/architecture-document.md + - content/templates/architecture-canvases.md + - content/templates/crosscutting-concerns.md - content/templates/adr.md - content/templates/rfc.md - content/templates/requirements.md + - content/templates/nfr-scenario.md + - content/templates/requirements-traceability-matrix.md - content/templates/interface-specification.md - - content/templates/architecture-document.md - - content/templates/architecture-canvases.md + - content/templates/software-design-specification.md - content/templates/experiment-record.md - - content/templates/postmortem.md + - content/templates/test-plan.md + - content/templates/systems-engineering-management-plan.md + - content/templates/team-working-agreement.md + - content/templates/meeting-minutes.md + - content/templates/decision-log.md + - content/templates/risk-register.md + - content/templates/technical-debt-register.md + - content/templates/release-plan.md + - content/templates/sbom.md - content/templates/pull-request-and-commits.md + - content/templates/postmortem.md + - content/templates/lessons-learned.md - content/templates/agent-task-brief.md + - content/templates/agents-md.md - content/templates/product-conformance-profile.md outputs: {docx: {}, pdf: {}} radar: - # Tech Radar. The section's own pull request lists the rest of its files here. + # Tech Radar (technical-documentation's radar): the methodology, the four quadrants and the directory. + # The interactive chart is a site page of its own (site/docs/radar-chart/), since it is HTML and script. sources: - content/radar/index.md + - {file: content/radar/languages-and-frameworks.md, page_break: true} + - {file: content/radar/platforms-and-operations.md, page_break: true} + - {file: content/radar/methods-and-patterns.md, page_break: true} + - {file: content/radar/tools-and-utilities.md, page_break: true} + - {file: content/radar/directory.md, page_break: true} outputs: {docx: {}, pdf: {}} firmitas: - # Firmitas. The section's own pull request lists the rest of its files here. + # Firmitas (Firmitas 2 from docs-WoW): front matter, the three parts with their chapters, and the + # appendices, which come from the first edition. Chapters sit one level below their part. sources: - content/firmitas/index.md + - {file: content/firmitas/01-preface.md, page_break: true} + - {file: content/firmitas/02-acknowledgements.md, page_break: true} + - {file: content/firmitas/03-introduction.md, page_break: true} + - {file: content/firmitas/part-1-why-programmes-fail/index.md, page_break: true} + - {file: content/firmitas/part-1-why-programmes-fail/01-the-programme-that-failed-predictably.md, heading_offset: 1} + - {file: content/firmitas/part-1-why-programmes-fail/02-estimation-why-it-fails-and-what-honest-looks-like.md, heading_offset: 1} + - {file: content/firmitas/part-1-why-programmes-fail/03-the-commitment-that-wasnt-theirs.md, heading_offset: 1} + - {file: content/firmitas/part-1-why-programmes-fail/04-what-the-system-did-with-the-information.md, heading_offset: 1} + - {file: content/firmitas/part-1-why-programmes-fail/05-the-learning-that-doesnt-happen.md, heading_offset: 1} + - {file: content/firmitas/part-1-why-programmes-fail/06-engineering-is-a-systems-discipline.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/index.md, page_break: true} + - {file: content/firmitas/part-2-the-framework/07-leadership-owns-the-system.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/08-delivery-knowledge-lives-closest-to-the-work.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/09-commitments-estimates-and-honest-plans.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/10-risk-issues-and-the-register-that-actually-works.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/11-requirements-from-customer-need-to-testable-specification.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/12-slack-flow-and-why-efficiency-thinking-fails.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/13-architecture-as-delivery-infrastructure.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/14-the-delivery-model.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/15-quality-as-a-system-property.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/16-governance-guardrails-and-decision-authority.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/17-teams-cognitive-load-and-organisation-for-flow.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/18-communication-information-flow-and-documentation.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/19-technical-debt-and-change-management.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/20-capability-people-and-building-for-the-long-term.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/21-portfolio-flow-and-prioritisation.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/22-business-alignment-economics-and-the-customer.md, heading_offset: 1} + - {file: content/firmitas/part-2-the-framework/23-ai-automation-and-human-accountability.md, heading_offset: 1} + - {file: content/firmitas/part-3-making-it-stick/index.md, page_break: true} + - {file: content/firmitas/part-3-making-it-stick/24-the-leader-who-has-to-change.md, heading_offset: 1} + - {file: content/firmitas/part-3-making-it-stick/25-assessing-readiness-and-adopting-progressively.md, heading_offset: 1} + - {file: content/firmitas/part-3-making-it-stick/26-common-failure-modes-and-anti-patterns.md, heading_offset: 1} + - {file: content/firmitas/part-3-making-it-stick/27-measuring-what-actually-matters.md, heading_offset: 1} + - {file: content/firmitas/part-3-making-it-stick/28-what-firmitas-cannot-fix.md, heading_offset: 1} + - {file: content/firmitas/part-3-making-it-stick/29-the-long-view.md, heading_offset: 1} + - {file: content/firmitas/appendices/index.md, page_break: true} + - {file: content/firmitas/appendices/a-glossary-of-terms.md, heading_offset: 1} + - {file: content/firmitas/appendices/b-firmitas-templates.md, heading_offset: 1} + - {file: content/firmitas/appendices/c-architectural-principles-for-firmitas.md, heading_offset: 1} + - {file: content/firmitas/appendices/d-minimal-balanced-and-regulated-firmitas-profiles.md, heading_offset: 1} + - {file: content/firmitas/appendices/e-end-to-end-worked-example.md, heading_offset: 1} + - {file: content/firmitas/appendices/f-leadership-and-delivery-anti-patterns-firmitas-is-designed-to-prevent.md, heading_offset: 1} + - {file: content/firmitas/appendices/g-bibliography-and-further-reading.md, heading_offset: 1} + - {file: content/firmitas/appendices/h-engineering-practice-across-disciplines.md, heading_offset: 1} + - {file: content/firmitas/appendices/i-quality-regulation-and-assurance.md, heading_offset: 1} outputs: {docx: {}, pdf: {}} @@ -184,10 +399,11 @@ site: dir: site # Only the sections that have moved in are published. Add each section here when its pull request # merges; until then its placeholder page stays off both sites. - documents: [glossary, templates] + documents: [guidelines, standards, guides, handbook, primers, radar, glossary, templates, firmitas] downloads: [pdf, docx] targets: github: mkdocs.yml # https://markbac.github.io/, personal home page tarf: mkdocs.tarf.yml # https://tarf.co.uk/, the tarf home page pages: # pages of our own, after the sections in the navigation + Tech Radar chart: radar-chart/index.md Blog: blog/index.md diff --git a/site/docs/blog/posts/2025-02-18-architecture-flashcards.md b/site/docs/blog/posts/2025-02-18-architecture-flashcards.md new file mode 100644 index 0000000..8aafc59 --- /dev/null +++ b/site/docs/blog/posts/2025-02-18-architecture-flashcards.md @@ -0,0 +1,220 @@ +--- +date: 2025-02-18T17:00:00 +authors: + - markbac +categories: + - Architecture +slug: architecture-flashcards +--- + +# Architecture Flashcards + + + +## **1\. Problem Definition** + +- What is the business problem or opportunity? +- Is the scope clearly defined and agreed? + +--- + +## **2\. Stakeholder Identification** + +- Who are the key stakeholders? +- Have you engaged with both business and technical stakeholders? + +--- + +## **3\. Context Diagram** + +- Have you defined the system boundaries? +- What external systems, actors, or processes interact with it? + +--- + +## **4\. Requirements** + +- Are the functional and non-functional requirements clear? +- Have they been prioritised and validated? + +--- + +## **5\. Architectural Vision** + +- What is the guiding vision for this architecture? +- Does it align with organisational strategy? + +--- + +## **6\. Architectural Patterns** + +- Which patterns are applicable (e.g., microservices, event-driven, layered)? +- Why are you choosing or rejecting specific patterns? + +--- + +## **7\. Decision Records** + +- Have you documented key architectural decisions? +- Are the trade-offs and rationale clear? + +--- + +## **8\. System Quality Attributes** + +- How will the architecture address performance, scalability, security, and maintainability? +- What are the primary quality attribute trade-offs? + +--- + +## **9\. Modelling** + +- Have you visualised the architecture using appropriate models (C4, UML, etc.)? +- Are the diagrams simple, clear, and focused? + +--- + +## **10\. Data Flow** + +- How does data flow through the system? +- Have you considered data integrity, consistency, and storage? + +--- + +## **11\. Integration** + +- How will the system integrate with other systems? +- Are integration points well-defined and robust? + +--- + +## **12\. Security** + +- Have you identified security threats and mitigations? +- How are authentication, authorisation, and data protection handled? + +--- + +## **13\. Resilience and Availability** + +- What happens if components fail? +- Is high availability or disaster recovery required? + +--- + +## **14\. Performance** + +- What are the performance expectations? +- Are there any known bottlenecks or constraints? + +--- + +## **15\. Scalability** + +- How will the system scale horizontally and/or vertically? +- Are there future growth considerations? + +--- + +## **16\. Deployment Strategy** + +- What is the deployment model (cloud, on-premises, hybrid)? +- How will you handle blue-green or canary deployments? + +--- + +## **17\. Observability** + +- Have you considered logging, monitoring, and alerting? +- How will you measure system health? + +--- + +## **18\. Technical Debt** + +- Are there areas of the architecture that introduce debt? +- How will you manage and pay it down? + +--- + +## **19\. Standards and Guidelines** + +- Are there coding, documentation, or architectural standards to follow? +- How will you ensure consistency across teams? + +--- + +## **20\. Governance** + +- What architectural governance processes are required? +- Are there any compliance or regulatory concerns? + +--- + +## **21\. Evolution and Extensibility** + +- Can the architecture evolve over time? +- How easy is it to add new features? + +--- + +## **22\. API Design** + +- Are APIs well-defined and RESTful, GraphQL, or event-driven as needed? +- Have you ensured proper versioning and backward compatibility? + +--- + +## **23\. Vendor Independence** + +- Are there any risks of vendor lock-in? +- What strategies will you use to maintain flexibility? + +--- + +## **24\. Technology Selection** + +- Is the technology stack suitable for the requirements? +- Have you considered longevity and community support? + +--- + +## **25\. Cost** + +- What are the cost implications of the architecture? +- Have you considered cloud costs, licensing, and operational expenses? + +--- + +## **26\. Documentation** + +- Is documentation lightweight but sufficient? +- Is it aimed at the right audience? + +--- + +## **27\. Collaboration** + +- Are you collaborating effectively with development, operations, and security teams? +- Have you communicated the architecture clearly? + +--- + +## **28\. MVP and Iteration** + +- What is the minimal viable product (MVP) for this architecture? +- How can it evolve iteratively? + +--- + +## **29\. Risks and Assumptions** + +- What are the key risks and assumptions? +- Are they documented and tracked? + +--- + +## **30\. Feedback and Validation** + +- How will you gather feedback and validate the architecture? +- Are there checkpoints for review and adaptation? diff --git a/site/docs/blog/posts/2025-02-18-embedded-architecture-process-mindmap.md b/site/docs/blog/posts/2025-02-18-embedded-architecture-process-mindmap.md new file mode 100644 index 0000000..2bda36b --- /dev/null +++ b/site/docs/blog/posts/2025-02-18-embedded-architecture-process-mindmap.md @@ -0,0 +1,168 @@ +--- +date: 2025-02-18T17:01:00 +authors: + - markbac +categories: + - Architecture + - Embedded +slug: embedded-architecture-process-mindmap +--- + +# Embedded Architecture Process Mindmap + + + +```mermaid +mindmap + root((Embedded Architecture Process)) + Product_Lifecycle + Concept_Stage + Market_Research + Vision_Statement + Feasibility_Study + Business_Case + Requirements_Definition + Functional_Requirements + Non-Functional_Requirements + Epics_and_User_Stories + Prioritisation_Sessions + Development_Stage + System_Architecture + High-Level_Architecture + Context_View + Building_Block_View + Detailed_Design + Component_Diagrams + Communication_Protocols + Prototyping_and_Feasibility + Hardware_Design + PCB_Design + Component_Selection + Power_Optimisation + Firmware_Development + Real-Time_OS + Driver_Development + Communication_Stack + Testing_and_Validation + Unit_Testing + Integration_Testing + System_Testing + Hardware-in-the-Loop + Launch_Stage + Pilot_Deployment + Production_Release + Customer_Feedback_Loop + Maintenance_Stage + Bug_Fixing + Feature_Rollout + Security_Patches + Version_Management + End_of_Life + Product_Decommissioning + Knowledge_Transfer + Data_Archiving + Project_Management + Planning + Scope_Definition + Resource_Planning + Risk_Assessment + Timeline_Estimation + Execution + Agile_Methodologies + Scrum + Sprint_Planning + Daily_Standups + Retrospectives + Kanban + Milestone_Tracking + Stakeholder_Communication + Monitoring_and_Control + Quality_Assurance + Risk_Mitigation + Budget_Tracking + Status_Reporting + Closure + Final_Review + Lessons_Learned + Documentation_Archiving + Governance_and_Architecture + Architectural_Reviews + Design_Reviews + Code_Reviews + Security_Assessments + Architecture_Documents + arc42_Template + Context_View + Building_Block_View + Runtime_View + Deployment_View + Risks_and_Technical_Decisions + Architecture_Decision_Records + Problem_Description + Considered_Options + Decision + Consequences + RFC_Process + Proposal_Documents + Design_Documents + Review_Cycle + Compliance_and_Standards + Safety_Standards + ISO_26262 + IEC_61508 + Security_Standards + OWASP + Secure_Coding_Practices + Regulatory_Audits + Documentation + Strategy_and_Overview + Vision_Statement + Product_Roadmap + Architecture_Canvas + Goals + Key_Decisions + Stakeholders + Open_Issues + Design_Documents + arc42_Communication_Canvas + Stakeholder_Communication + Information_Flow + Collaboration_Strategy + System_Design + UML_Diagrams + Sequence_Diagrams + Process_Documentation + Development_Workflows + Testing_Protocols + CI/CD_Pipeline_Docs + Docs_as_Code + Version_Controlled_Documentation + Continuous_Integration_for_Documentation + Markdown_and_Mermaid_Diagrams + Collaboration_and_Communication + Cross-Functional_Teams + Hardware-Software_Integration + DevOps_Coordination + QA_Collaboration + Communication_Strategies + Stand-Ups + Sprint_Reviews + Cross-Team_Retrospectives + Documentation_as_Code + Git_Workflows + Pull_Requests_for_Documentation + Tools_and_Methodologies + Agile_Practices + Continuous_Integration + Continuous_Deployment + Architecture_Modelling + C4_Model + arc42 + Visualisation_Tools + PlantUML + Mermaid + Automated_Testing + Unit_Testing + Hardware-in-the-Loop + Performance_Testing +``` diff --git a/site/docs/blog/posts/2025-03-11-expanding-my-nas-future-containers-to-run.md b/site/docs/blog/posts/2025-03-11-expanding-my-nas-future-containers-to-run.md new file mode 100644 index 0000000..73ecc7f --- /dev/null +++ b/site/docs/blog/posts/2025-03-11-expanding-my-nas-future-containers-to-run.md @@ -0,0 +1,47 @@ +--- +date: 2025-03-11T14:00:00 +authors: + - markbac +categories: + - Home lab +slug: expanding-my-nas-future-containers-to-run +--- + +# Expanding My NAS: Future Containers to Run + +With my Synology NAS already handling Docker workloads, I’m looking to expand its capabilities with some **new containers**. These will enhance **monitoring, automation, note-taking, development, and network management**. Here’s what’s on my list: + + + +## **Monitoring & Network Tools** + +* **Glances** – Real-time system monitoring for CPU, RAM, and storage. +* **Uptime Kuma** – Self-hosted status page for tracking service uptime. +* **NetAlert** – Notifies me when new devices connect to my network. +* **WatchYourLAN** – A lightweight LAN device monitor. + +## **Automation & Utility** + +* **Node-RED** – Visual programming for automating tasks. +* **Upsnap** – A simple Wake-on-LAN tool for remote device control. +* **Homarr** +* **Watchtower** + +## **Productivity & Development** + +* **Excalidraw** – A self-hosted collaborative whiteboard for quick diagrams. +* **Code-Server** – VS Code in the browser for lightweight development. +* **HedgeDoc** – Markdown-based collaborative note-taking. +* **Kanboard** – A Kanban board for tracking projects. +* **Blinko ** +* **Trilium** + +## **Other Useful Tools** + +* **IT Tools** – A web-based toolkit for network diagnostics. +* **Gramps** – A genealogy database for mapping family history. +* **Reubah** + +This will **expand my NAS into a full-fledged productivity and monitoring hub**. The key challenge? **Managing resources** while keeping everything lightweight and efficient. + +💡 **What other containers should I consider? Let me know your must-haves!** 🚀 diff --git a/site/docs/blog/posts/2025-03-11-monitoring-raspberry-pis-with-pistats-a-unified-dashboard-fo.md b/site/docs/blog/posts/2025-03-11-monitoring-raspberry-pis-with-pistats-a-unified-dashboard-fo.md new file mode 100644 index 0000000..06d1c78 --- /dev/null +++ b/site/docs/blog/posts/2025-03-11-monitoring-raspberry-pis-with-pistats-a-unified-dashboard-fo.md @@ -0,0 +1,182 @@ +--- +date: 2025-03-11T13:00:00 +authors: + - markbac +categories: + - Home lab +slug: monitoring-raspberry-pis-with-pistats-a-unified-dashboard-fo +--- + +# Monitoring Raspberry Pis with PiStats & A Unified Dashboard for All Devices + +As my self-hosted network grows, keeping an eye on my **Raspberry Pis** is becoming more important. With multiple **Pi-hole, Home Assistant, and testing boards**, I need a way to track their **CPU usage, temperature, storage, and overall health**—ideally from a **single dashboard** that also monitors other key devices. + + + +In this post, I’ll cover: + +* **PiStats** – A simple way to monitor Raspberry Pi devices. +* **Unified Dashboards** – How to centralise monitoring for all devices, including Synology NAS and other hardware. + +*** + +## **PiStats: A Simple Raspberry Pi Monitoring Tool** + +### **What is PiStats?** + +PiStats is a **lightweight monitoring tool** designed for Raspberry Pi devices. It provides real-time stats on:\ +✅ CPU usage\ +✅ Temperature\ +✅ Memory & storage usage\ +✅ Uptime + +It’s a great alternative to setting up **Grafana & Prometheus**, which, while powerful, can be overkill for basic monitoring needs. + +### **How to Install PiStats on a Raspberry Pi** + +Since PiStats is a simple Python script, setup is quick. + +1️⃣ **Install dependencies** + +```bash +sudo apt update && sudo apt install python3-pip -y +pip3 install flask +``` + +2️⃣ **Download PiStats** + +```bash +git clone https://github.com/Spirotot/PiStats.git +cd PiStats +``` + +3️⃣ **Run PiStats** + +```bash +python3 app.py +``` + +4️⃣ **Access the Web UI** + +* Open a browser and go to `http://:5000` +* You’ll see a basic dashboard with CPU, temperature, and memory stats. + +### **Should You Use PiStats?** + +✅ **Pros:** + +* Super lightweight (just a small Python script). +* Simple web-based UI with key metrics. +* No need for a database or complex setup. + +❌ **Cons:** + +* No long-term data storage (only real-time stats). +* No multi-device monitoring—only shows one Raspberry Pi at a time. + +PiStats is great for quick, **per-device** monitoring, but if you want a **single dashboard for all Raspberry Pis and other devices**, you’ll need something more **scalable**. + +*** + +## **Building a Unified Dashboard for Raspberry Pis & Other Devices** + +Since I want **one dashboard** to monitor **all Raspberry Pis, my Synology NAS, and other devices**, PiStats alone won’t cut it. Here are **two solid options**: + +*** + +### **Option 1: Glances + Web UI (Quick & Simple)** + +Glances is a lightweight system monitoring tool that can show CPU, RAM, disk, and network usage across **multiple devices**. + +#### **How to Install & Set Up Glances** + +1️⃣ **Install Glances on all Raspberry Pis** + +```bash +sudo apt update && sudo apt install glances -y +``` + +2️⃣ **Run Glances in Web Mode** + +```bash +glances -w +``` + +* Access it via `http://:61208` + +3️⃣ **Centralised Dashboard:** + +* Install Glances on **one device** (e.g., NAS or a Raspberry Pi). +* Add all other Pis as remote agents: + +```bash +glances --browser +``` + +* Now, you can monitor all devices in a **single web interface**. + +✅ **Pros:** + +* Quick to set up, no database required. +* Works on **Raspberry Pi, Synology NAS, and other Linux devices**. +* Lightweight compared to Prometheus/Grafana. + +❌ **Cons:** + +* No historical data storage—only real-time monitoring. + +*** + +### **Option 2: Grafana + Prometheus (Powerful & Scalable)** + +If you want **long-term monitoring, alerts, and detailed analytics**, Grafana + Prometheus is a **more advanced solution**. + +#### **How to Set Up a Centralised Grafana Dashboard** + +1️⃣ **Install Prometheus on a central device (NAS or Pi)** + +```bash +docker run -d --name=prometheus -p 9090:9090 prom/prometheus +``` + +2️⃣ **Install Node Exporter on Raspberry Pis** + +```bash +wget https://github.com/prometheus/node_exporter/releases/latest/download/node_exporter-armv7 +chmod +x node_exporter-armv7 +./node_exporter-armv7 & +``` + +* This sends Pi metrics to Prometheus. + +3️⃣ **Install Grafana** (on NAS or another Pi) + +```bash +docker run -d -p 3000:3000 grafana/grafana +``` + +* Access Grafana at `http://:3000` +* Add Prometheus as a **data source**. +* Create dashboards to track **CPU, temperature, RAM, uptime**, and more. + +✅ **Pros:** + +* Scales to **multiple Raspberry Pis, NAS, and other devices**. +* Stores historical data for analysis. +* Custom dashboards with alerts. + +❌ **Cons:** + +* More complex to set up. +* Uses more resources than Glances. + +*** + +## **What’s Best for My Setup?** + +Since I want a **single dashboard** that covers all my Raspberry Pis **and** my NAS, my approach will likely be: + +* **For quick monitoring:** **Glances Web UI** +* **For long-term data & alerts:** **Grafana + Prometheus** + +PiStats is great as a **basic, per-device tool**, but if you want true **multi-device monitoring**, Glances or Grafana are the way to go. diff --git a/site/docs/blog/posts/2025-03-11-self-hosting-my-smart-home-network-current-setup-and-future.md b/site/docs/blog/posts/2025-03-11-self-hosting-my-smart-home-network-current-setup-and-future.md new file mode 100644 index 0000000..e131d8b --- /dev/null +++ b/site/docs/blog/posts/2025-03-11-self-hosting-my-smart-home-network-current-setup-and-future.md @@ -0,0 +1,74 @@ +--- +date: 2025-03-11T12:00:00 +authors: + - markbac +categories: + - Home lab +slug: self-hosting-my-smart-home-network-current-setup-and-future +--- + +# Self-Hosting My Smart Home & Network: Current Setup and Future Plans + +Living in rural Lancashire with no close neighbours has its perks—peace, privacy, and the freedom to build a powerful self-hosted home network without worrying about ISP restrictions or poor infrastructure. My setup is built around **Synology, Raspberry Pi, and Docker-powered services**, giving me complete control over my network, automation, and devices. Here’s what I have running today and what I’m planning next. + + + +*** + +## **My Current Home Network Setup** + +### **Core Infrastructure** + +* **Synology RT6600ax Router** – Handles routing and VLANs like a champ. +* **Synology DS723+ NAS (32GB RAM, NVMe cache)** – The backbone of my network, running Docker containers and hosting services. +* **Portainer** – Makes managing Docker containers much easier. +* **Heimdall** – A clean homepage for quick access to my self-hosted services. +* **Fing** – A dedicated network monitoring device to track all connected devices and detect new ones. + +### **Raspberry Pi Fleet** + +* **Raspberry Pi 4 (Pi-hole)** – Blocks ads and trackers at the network level. +* **Raspberry Pi 5 & Pi 400** – Used for testing, tinkering, and lightweight workloads. +* **Raspberry Pi (Home Assistant)** – The core of my smart home automation. + +*** + +## **What Works Well?** + +✅ **Reliable Infrastructure** – The NAS and Raspberry Pis each handle their own dedicated tasks.\ +✅ **Network Visibility** – **Fing** helps track connected devices and alerts me to any new ones.\ +✅ **Ad-Free Browsing** – Pi-hole ensures the internet is cleaner.\ +✅ **Containerisation** – Running most services in Docker keeps everything modular and easy to manage.\ +✅ **Smart Home Control** – Home Assistant integrates my devices seamlessly. + +*** + +## **What’s Next? Future Plans & Additions** + +### **Monitoring & Dashboards** + +🔹 **Flame Dashboard** – A sleek, modern alternative to Heimdall with search functionality.\ +🔹 **Uptime Kuma** – A self-hosted service to monitor uptime and network status.\ +🔹 **Glances** – A real-time monitoring tool to track CPU, RAM, and system stats across devices. + +### **Raspberry Pi Monitoring** + +I want better visibility into the **health and performance** of my Raspberry Pis. Some potential solutions: + +* **Grafana + Prometheus** – Powerful but requires some setup. +* **Glances with Agent Mode** – Simple and lightweight. +* **PiStats** – A dashboard-style web tool specifically for monitoring Pi devices. +* **Raspberry Pi Monitoring Script** – A simple cron job that logs CPU temp, RAM, and storage usage. + +### **Automation & Updates** + +🔹 **Watchtower** – Automates Docker container updates (used with caution).\ +🔹 **Dozzle** – A lightweight web-based log viewer for Docker. + +*** + +## **Why Self-Host?** + +For me, self-hosting is about **control, reliability, and performance**. I don’t rely on external cloud services, I avoid monthly subscription costs, and I can fine-tune everything to my specific needs. + +With **better monitoring, dashboards, and automation**, my setup will become even more streamlined and resilient. Now, the only question is: **What should I add first?** 🤔 diff --git a/site/docs/blog/posts/2025-03-11-setting-up-an-nginx-reverse-proxy-with-ssl-and-local-dns-for.md b/site/docs/blog/posts/2025-03-11-setting-up-an-nginx-reverse-proxy-with-ssl-and-local-dns-for.md new file mode 100644 index 0000000..baed0e8 --- /dev/null +++ b/site/docs/blog/posts/2025-03-11-setting-up-an-nginx-reverse-proxy-with-ssl-and-local-dns-for.md @@ -0,0 +1,219 @@ +--- +date: 2025-03-11T11:00:00 +authors: + - markbac +categories: + - Home lab +slug: setting-up-an-nginx-reverse-proxy-with-ssl-and-local-dns-for +--- + +# Setting Up an Nginx Reverse Proxy with SSL and Local DNS for a Synology NAS and Other Services (home.example) + +> **For a private home network using a Synology NAS, Synology Router, and Raspberry Pis** + +## **What This Guide Covers** + +* ✅ **Use a local domain (`*.home.example`)** to access internal services. +* ✅ **Secure access with a wildcard SSL certificate (Let's Encrypt).** +* ✅ **Run Nginx Proxy Manager on Synology NAS without port conflicts.** +* ✅ **Pi-hole for local DNS resolution (without Unbound).** +* ✅ **Use Cloudflare API for automatic SSL certificate renewal.** +* ✅ **Works for all devices on the network (PC, Mac, iOS, Android, Smart TVs).** +* ❌ **No external access required.** + +*** + +## **1⃣ Devices & Network Setup** + +For this setup, we assume the following hardware: + + + +| Device | IP Address | Role | +| -------------------------- | ------------------ | ----------------------------------- | +| **Synology NAS** | `192.0.2.203` | Nginx Proxy Manager (Reverse Proxy) | +| **Home Assistant** | `192.0.2.101` | Home automation | +| **Heimdall Dashboard** | `192.0.2.203:5050` | Service launcher | +| **Pi-hole (Raspberry Pi)** | `192.0.2.10` | Local DNS | + +*** + +## **2⃣ Configure Local DNS with Pi-hole (No Unbound)** + +Pi-hole ensures `*.home.example` resolves correctly inside your network. + +### **2.1 Install Pi-hole on a Raspberry Pi** + +1. **Run the installer:** + ```sh + curl -sSL https://install.pi-hole.net | bash + ``` + +2. **Follow the setup:** + + * Choose the Raspberry Pi’s **network interface**. + * **Set a static IP** (`192.0.2.10` recommended). + * **Upstream DNS Provider:** Choose **"Custom"** (we’ll configure NextDNS later). + * Enable the **web interface** for easy management. + +### **2.2 Set Up Local DNS Records in Pi-hole** + +1. Open the **Pi-hole Admin Panel**:\ + `http://pi.hole/admin` +2. Go to **Settings → Local DNS → DNS Records**. +3. **Add your internal services:** + ``` + nas.home.example → 192.0.2.203 + ha.home.example → 192.0.2.101 + heim.home.example → 192.0.2.203 + ``` +4. **Save changes**. + +Now, any device using Pi-hole will resolve `*.home.example` to the correct local IP. + +*** + +## **3⃣ Set Pi-hole as the Network’s Primary DNS** + +### **On Synology Router (Best Option)** + +1. **Log into your Synology Router** (`http://router.synology.com`). +2. **Go to Network Centre → Internet → DHCP Server**. +3. **Set Primary DNS to:** `192.0.2.10` (Pi-hole). +4. **Remove any Secondary DNS** (to prevent bypassing Pi-hole). +5. **Apply changes**. + +*** + +## **4⃣ Configure NextDNS as Pi-hole’s Upstream** + +Since we’re not using Unbound, **Pi-hole needs an external resolver for non-local queries**. + +1. Open **Pi-hole Admin Panel**. + +2. **Go to Settings → DNS**. + +3. **Disable all default upstream DNS providers**. + +4. **Set NextDNS as the upstream DNS**: + + ``` + 45.90.28.0 + 45.90.30.0 + ``` + + * If using **DNS-over-HTTPS (DoH)**: + + ```sh + sudo apt install cloudflared -y + ``` + + * Add this config to `/etc/dnsmasq.d/02-cloudflared.conf`: + ```ini + server=https://dns.nextdns.io/YOUR_NEXTDNS_ID + ``` + * Restart Pi-hole: + ```sh + sudo systemctl restart pihole-FTL + ``` + +Now, **Pi-hole resolves local domains (`*.home.example`) and forwards external lookups to NextDNS.** + +*** + +## **5⃣ Install Nginx Proxy Manager on Synology NAS** + +### **5.1 Install Docker on Synology NAS** + +1. **Go to Synology Package Centre**. +2. **Install Docker**. + +### **5.2 Deploy Nginx Proxy Manager** + +1. **SSH into your NAS** and create an NPM folder: + ```sh + mkdir -p /volume1/docker/npm && cd /volume1/docker/npm + ``` +2. **Create a `docker-compose.yml` file:** + ```sh + nano docker-compose.yml + ``` +3. **Modify the port mappings to avoid conflicts with Synology DSM:** + ```yaml + version: "3" + services: + npm: + image: 'jc21/nginx-proxy-manager:latest' + restart: always + ports: + - "8080:80" # HTTP → Change to 8080 + - "8443:443" # HTTPS → Change to 8443 + - "81:81" # NPM Web UI (unchanged) + volumes: + - ./data:/data + - ./letsencrypt:/etc/letsencrypt + ``` +4. **Start NPM:** + ```sh + docker-compose up -d + ``` + +Now, NPM will listen on: + +* **`http://NAS_IP:8080`** instead of `80`. +* **`https://NAS_IP:8443`** instead of `443`. + +*** + +## **6⃣ Configure Let’s Encrypt SSL with Cloudflare API** + +### **6.1 Get a Cloudflare API Token** + +1. **Go to** [Cloudflare Dashboard](). +2. **Select your domain (`home.example`)**. +3. **Go to "API Tokens" → "Create Token".** +4. **Select "Edit Zone DNS"** as the permission scope. +5. **Restrict it to `home.example`**. +6. **Generate and copy the token**. + +### **6.2 Add SSL Certificate in NPM** + +1. **Log into Nginx Proxy Manager (`http://NAS_IP:81`)**. +2. **Go to "SSL Certificates" → "Add SSL Certificate".** +3. **Choose "Let’s Encrypt".** +4. **Enter `*.home.example` as the domain.** +5. **Use DNS Challenge → Cloudflare.** +6. **Paste your Cloudflare API Token**. +7. **Click "Save" and issue the certificate.** + +✅ **Now, Let’s Encrypt will automatically renew your wildcard SSL every 90 days.** + +*** + +## **7⃣ Configure Proxy Hosts in Nginx Proxy Manager** + +1. **Go to "Hosts" → "Proxy Hosts"**. + +2. **Add services (e.g., `nas.home.example`, `ha.home.example`)**: + + * **Domain**: `nas.home.example` + * **Forward IP**: `192.0.2.203` + * **Forward Port**: `5000` + * **Force SSL** + +3. **Repeat for other services**. + +*** + +## **Final Setup Summary** + +| Component | Port | +| -------------------------------- | ---------------------------------------- | +| **Synology DSM** | Uses ports **80 & 443** | +| **Nginx Proxy Manager (Docker)** | Now runs on **8080 & 8443** | +| **Synology Reverse Proxy** | Routes **80 → 8080 & 443 → 8443** | +| **Pi-hole (Raspberry Pi)** | Runs on **192.0.2.10 (port 53 for DNS)** | + +✅ **No port conflicts with Synology NAS.**\ +✅ **Wildcard SSL auto-renewal via Cloudflare API.**\ +✅ **All services accessible with `*.home.example` and HTTPS.** diff --git a/site/docs/blog/posts/2025-04-25-arp-spoofing-mac-filtering-and-network-fingerprinting.md b/site/docs/blog/posts/2025-04-25-arp-spoofing-mac-filtering-and-network-fingerprinting.md new file mode 100644 index 0000000..6ed99a9 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-arp-spoofing-mac-filtering-and-network-fingerprinting.md @@ -0,0 +1,136 @@ +--- +date: 2025-04-25T09:22:00 +authors: + - markbac +categories: + - Home lab +slug: arp-spoofing-mac-filtering-and-network-fingerprinting +--- + +# ARP Spoofing, MAC Filtering and Network Fingerprinting + +*Part 22 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Ah, Layer 2—the unsung hero of networking. It’s like the midfield in a football team—often overlooked but absolutely critical for passing the ball securely from one player to another. In networking terms, we're talking about the data link layer, where frames are transferred over physical mediums. Yet, just like a match can be compromised by a sneaky opponent, your network can be vulnerable to various Layer 2 threats if you're not careful. + + + +## Importance of Understanding These Risks + +While most folks pay attention to higher-level attacks like SQL injections or DDoS attacks, ignoring Layer 2 vulnerabilities is a bit like locking your front door but leaving your windows wide open. So today, let's dive into some advanced Layer 2 attacks such as ARP spoofing and network fingerprinting and explore how MAC address filtering can be both a help and a hindrance. + +## ARP Spoofing (Detailed) + +First up on our agenda is ARP spoofing—a bit like someone pretending to be your mate down the pub to get free drinks on your tab. + +### How ARP Spoofing Works + +Here's how it works: + +- **ARP Basics**: The Address Resolution Protocol (ARP) is used to map IP addresses to MAC addresses. +- **Spoofing Attack**: An attacker sends fake ARP messages over a local network, linking their MAC address with the IP address of a legitimate device. +- **Man-in-the-Middle**: Once successful, the attacker can intercept data meant for that IP address or alter communications. + +```mermaid +graph TD; + A[Victim] -->|Sends Data| B[Router]; + B -->|Responds| A; + C[Attacker] -->|Sends Fake ARP| A; + A -->|Data Hijacked| C; +``` + +### Consequences of ARP Spoofing + +The fallouts are rather grim: + +- **Data Interception**: Sensitive information (think passwords) can be captured. +- **Network Disruption**: Legitimate communication gets diverted or blocked. +- **Impersonation**: The attacker can impersonate devices on the network. + +### Detection and Prevention Techniques + +So how do we combat this villainy? + +- **Static ARP Entries**: Hardcode MAC-IP mappings to prevent learning new entries dynamically. +- **ARP Inspection**: Use features like Dynamic ARP Inspection (DAI) in managed switches. +- **Detection Tools**: Employ tools like `arpwatch` or `XArp` for monitoring unusual changes. + +> **Tip:** Implementing VLAN segmentation can also limit the scope of an attack within the network. + +## MAC Address Filtering + +Next on the docket is MAC address filtering—a bit like using a VIP list at a club. Only certain devices are allowed in. + +### How MAC Filtering Works + +In essence: + +- Network devices have lists of approved MAC addresses. +- Only traffic from these addresses is allowed through. + +### Limitations and Bypasses + +However, it's not foolproof: + +- **MAC Spoofing**: An attacker can easily change their device's MAC address to match an authorised one. +- **Device Limitations**: Keeping lists updated in large networks can be cumbersome. + +### When MAC Filtering Might Be Useful + +Still, it has its place: + +- Useful in small networks where you know all devices. +- Acts as a basic barrier against casual snoopers. + +> **Caution:** Don’t solely rely on MAC filtering; it should be just one layer of your security strategy. + +## Network Fingerprinting + +Moving on, let’s talk about network fingerprinting. It’s akin to someone knowing which brand of tea you drink by looking at your shopping habits. + +### Passive vs Active Fingerprinting + +Two approaches here: + +- **Passive Fingerprinting**: Observes traffic patterns without injecting packets—stealthy but slower. +- **Active Fingerprinting**: Sends packets to elicit responses—faster but more detectable. + +### Information That Can Be Gathered + +Both methods aim to collect data such as: + +- Operating systems and versions +- Open ports and running services +- Devices' roles within the network + +### Countermeasures + +To obscure your fingerprints: + +- Use firewalls to control outbound information. +- Employ IDS/IPS systems for anomaly detection. +- Regularly update software to close known vulnerabilities. + +## Real-World Scenarios + +Let's bring all this theory into the real world. + +### Attacks on Local Networks + +Think of a university campus where students might try to sniff each other's traffic—naughty but not unheard of. Implementing VLANs and using network monitoring tools can mitigate such risks effectively. + +### Security Audits + +During audits, ethical hackers might use fingerprinting techniques to assess vulnerabilities. Here, layers of security measures will showcase your defensive robustness. + +## Conclusion: Proactive Layer 2 Security Measures + +In summary, while Layer 2 might not always steal the spotlight, ignoring it is akin to ignoring rust on scaffolding—it'll bite you eventually. Adopt multiple defensive strategies: + +- Be vigilant about ARP spoofing with static entries and inspections. +- Use MAC filtering wisely and as part of a broader security plan. +- Obscure your network's fingerprint through good practices and up-to-date defences. + +> **Takeaway:** Layer 2 security isn't just nice-to-have; it's essential. Like keeping a good brew at hand during winter—a necessity for survival! + +That wraps up our look into these fascinating aspects of network security. Now go forth and lock down those low-level layers! diff --git a/site/docs/blog/posts/2025-04-25-authentication-authorisation-accounting-aaa.md b/site/docs/blog/posts/2025-04-25-authentication-authorisation-accounting-aaa.md new file mode 100644 index 0000000..0b01bbe --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-authentication-authorisation-accounting-aaa.md @@ -0,0 +1,137 @@ +--- +date: 2025-04-25T09:13:00 +authors: + - markbac +categories: + - Networking +slug: authentication-authorisation-accounting-aaa +--- + +# Authentication, Authorisation, Accounting (AAA) + +*Part 13 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +In the world of network security, AAA doesn't stand for "Amateur Athletes Association" or anything to do with batteries, but rather, it represents a foundational concept: Authentication, Authorisation, and Accounting. These three pillars are essential in ensuring secure access to networks and data, whether you're managing an enterprise network or setting up a secure home environment. + + + +Understanding AAA is like understanding the rules of a football match; you need to know who’s allowed on the pitch, what they’re allowed to do, and how to keep track of their actions. This post will delve into each component of AAA, showing you how they apply in real-world scenarios such as Wi-Fi networks, VPNs, and logging systems. + +## Importance of AAA in Network Security + +Imagine your network as a castle. You wouldn't want just anyone waltzing in through the front gate, would you? AAA helps you: + +- **Authenticate**: Verify identities to ensure only legitimate users enter. + +- **Authorise**: Decide what these users can access once they're inside. + +- **Account**: Keep track of user activities for auditing and troubleshooting. + +AAA is critical because it helps maintain the integrity, confidentiality, and availability of your network resources. Now, let's explore each component in detail. + +## Authentication + +### Verifying User Identity + +Authentication is like checking IDs at a club entrance. It's all about verifying that users are who they claim to be before granting them access. + +### Methods of Authentication + +- **Passwords**: The traditional method. Simple but prone to being compromised if not managed well (think of weak passwords as leaving the back door open). + +- **Certificates**: Digital certificates act like electronic passports, providing stronger security by encrypting communications. + +- **Biometrics**: Fingerprints or facial recognition offer high-security levels but raise privacy concerns. + +```mermaid +graph TD; + A[User Attempts Access] --> B{Authentication Method}; + B -->|Password| C(Verify Password); + B -->|Certificate| D(Verify Certificate); + B -->|Biometric| E(Scan Biometric); + C --> F{Access Granted}; + D --> F; + E --> F; +``` + +### Note +Always use multi-factor authentication (MFA) where possible. It's like having multiple locks on your door—extra layers make it much harder for intruders. + +## Authorisation + +### Determining User Privileges + +Once authenticated, authorisation determines what a user can and cannot do. Think of this as assigning roles on a football team—everyone has specific duties based on their skills. + +### Access Control Lists (ACLs) + +ACLs list permissions attached to an object (like files or directories), specifying who can access them and what operations they can perform. + +- **Example**: An ACL may allow users to read a file but not write or delete it. + +### Role-Based Access Control (RBAC) + +RBAC assigns permissions based on roles within an organisation—like midfielders and defenders have different responsibilities on the pitch. + +- **Example**: A manager may have more access rights than a regular employee. + +## Accounting + +### Tracking User Activity + +Accounting is about keeping score—tracking what users do in your network. This helps with: + +- Auditing +- Troubleshooting +- Compliance with regulations + +### Logging and Auditing + +Logs are records of user activities and system events. Regularly reviewing logs can help detect anomalies early—like spotting unusual movements in a football match indicating foul play. + +### Tip +Implement automated log analysis tools to sift through large volumes of data efficiently. Think of them as your assistant referees watching every angle for potential fouls. + +## AAA Protocols + +Now that we understand what AAA does, let's look at some protocols that implement these concepts: + +### RADIUS (Remote Authentication Dial-In User Service) + +RADIUS is widely used for network access control—especially effective for Wi-Fi authentication and VPN connections. It's like having a central referee making decisions across various games simultaneously. + +### TACACS+ (Terminal Access Controller Access-Control System Plus) + +TACACS+ offers greater granularity than RADIUS by separating authentication, authorisation, and accounting processes. It's akin to having specialised officials managing different aspects of the game—one for offside calls, another for penalties. + +## AAA in Different Environments + +AAA isn't just for big corporate networks; it applies to various environments: + +### Enterprise Wi-Fi (802.1X) + +IEEE 802.1X is an authentication standard used in enterprise Wi-Fi networks. It uses RADIUS servers to authenticate users before granting them network access—similar to ticket checks before entering a stadium. + +### VPN Authentication + +VPNs use AAA protocols to ensure secure remote access. Authentication confirms user identity while authorisation controls accessible resources within the private network—much like ensuring team members have the right passes during an away game. + +### Network Device Access + +Network devices often require secure access controls. Implementing AAA ensures only authorised personnel configure routers or switches—like allowing only qualified mechanics under the hood of your car. + +## Conclusion + +The importance of AAA cannot be overstated when it comes to secure access control. It ensures: + +- Only legitimate users gain entry (Authentication) +- Users have appropriate privileges (Authorisation) +- Activities are monitored and recorded (Accounting) + +Whether securing an enterprise network or setting up advanced home systems, understanding and implementing AAA is crucial. It’s like having reliable referees overseeing every aspect of your game plan—keeping matches fair and square while ensuring smooth operations. + +--- + +## Key Takeaway + +AAA forms the bedrock of any robust security strategy. By effectively authenticating users, authorising their access levels, and accounting for their actions, you create a comprehensive shield against unauthorised intrusion—whether you're defending your business empire or fortifying your smart home setup against cyber threats. diff --git a/site/docs/blog/posts/2025-04-25-common-application-protocols.md b/site/docs/blog/posts/2025-04-25-common-application-protocols.md new file mode 100644 index 0000000..3300a20 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-common-application-protocols.md @@ -0,0 +1,160 @@ +--- +date: 2025-04-25T09:11:00 +authors: + - markbac +categories: + - Networking +slug: common-application-protocols +--- + +# Common Application Protocols: HTTP, HTTPS, FTP, SMTP, IMAP + +*Part 11 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## Introduction + +Imagine the Internet as a sprawling city. There are roads (networks) and buildings (servers), but what really matters to us are the services—shops, offices, and homes—that we access using these roads. Application-layer protocols are like the signposts and rules that guide us through this city to get what we need. They dictate how our data is packaged, sent, received, and interpreted across networks. Today, we'll take a tour of some popular ones: HTTP, HTTPS, FTP, SMTP, and IMAP. We’ll explore how they interact with users and why they're important. + + + +## HTTP (Hypertext Transfer Protocol) + +### Web Browsing + +HTTP is like the postal service for web pages. It’s the protocol used by the World Wide Web to transfer HTML pages from a server to a client (your browser). When you type a URL into your browser or click a link, you're effectively sending an "HTTP request" to a server asking for a page. + +### Client-Server Model + +HTTP operates on a client-server model: + +- **Client**: Your browser +- **Server**: The web server hosting the pages + +The client sends an HTTP request and the server responds with an HTTP response containing the requested content. + +```mermaid +sequenceDiagram + participant Client as Browser + participant Server as Web Server + Client->>Server: HTTP GET /index.html + Server-->>Client: 200 OK + HTML Content +``` + +### Stateless Nature + +HTTP is stateless, meaning each request from a client to server is independent. There’s no memory of past interactions. This simplicity makes it fast but requires additional mechanisms (like cookies) for maintaining sessions. + +### Port Usage + +- **Port**: 80 (default) + +## HTTPS (HTTP Secure) + +### Encryption and Security + +HTTPS wraps HTTP in a secure layer using SSL/TLS encryption. It's like putting your sensitive letter in a tamper-proof envelope before posting it—ensuring privacy and integrity between your browser and the website. + +### SSL/TLS + +- **SSL/TLS**: Protocols used to encrypt data +- **Purpose**: Ensure data confidentiality and integrity + +### Importance for Sensitive Data + +Using HTTPS is crucial when handling sensitive information like login credentials or payment details. Browsers often warn users about non-HTTPS sites because unencrypted data can be intercepted by malicious actors. + +### Port Usage + +- **Port**: 443 (default) + +## FTP (File Transfer Protocol) + +### File Transfer Between Systems + +FTP is akin to loading up a lorry with goods (files) from one warehouse (system) and delivering them to another. It’s used for transferring files between computers on a network. + +### Separate Control and Data Connections + +FTP uses two connections: + +1. **Control Connection**: Manages session commands (like logging in) +2. **Data Connection**: Transfers file data + +The separation allows commands and data transfer to occur independently. + +```mermaid +sequenceDiagram + participant User as User's Computer + participant Server as FTP Server + User->>Server: FTP Command Channel + Note right of Server: Establishes control session + User->>Server: FTP Data Channel + Note right of Server: Transfers files +``` + +### Port Usage + +- **Ports**: 21 (control), 20 (data) + +## SMTP (Simple Mail Transfer Protocol) + +### Sending Email Messages + +SMTP is like the traditional postman for emails—responsible for sending messages from client to mail server or between servers until it reaches its destination. + +### Mail Servers and Relaying + +SMTP works by relaying emails between mail servers: + +- **Sending Client**: Connects to an SMTP server to send emails. +- **Mail Servers**: Relay emails towards the recipient's mail server. + +SMTP doesn’t handle email retrieval—that's where IMAP comes in. + +### Port Usage + +- **Port**: 25 (default), 587 or 465 for secure transport with TLS/SSL + +## IMAP (Internet Message Access Protocol) + +### Retrieving Email Messages + +IMAP is like visiting your mailbox at the local post office—it allows you to view and manage your email stored on the mail server without downloading them permanently. + +### Folder Management and Synchronization + +With IMAP: + +- Emails are stored on the server. +- You can organise emails into folders. +- Changes sync automatically across devices. + +This protocol suits users who want access to their email from multiple devices without losing track of organisation. + +### Port Usage + +- **Port**: 143 (default), 993 for secure connections with TLS/SSL + +## Protocol Summary Table + +| Protocol | Description | Port | Use Case | +|----------|------------------------------------------------|-------|------------------------------| +| HTTP | Web browsing | 80 | Accessing websites | +| HTTPS | Secure web browsing | 443 | Secure access to websites | +| FTP | File transfer | 21/20 | Transferring files | +| SMTP | Sending emails | 25/587/465 | Email sending | +| IMAP | Retrieving/managing emails | 143/993 | Email retrieval | + +## Conclusion + +Application-layer protocols are essential tools in our digital toolbox. Whether you're browsing websites with HTTP/HTTPS, transferring files via FTP, or managing emails through SMTP/IMAP, understanding these protocols helps you navigate and secure your online activities more effectively. + +--- + +### Key Takeaways + +- **HTTP/HTTPS** provide the backbone for web browsing; always prefer HTTPS for security. +- **FTP** is vital for moving larger files between systems but consider secure variants like FTPS or SFTP. +- **SMTP/IMAP** work together to handle email communication; use secure ports whenever possible. + +Understanding these protocols not only aids in troubleshooting but also equips you with knowledge to make informed decisions about online security and efficiency. diff --git a/site/docs/blog/posts/2025-04-25-development-and-documentation-tools-in-containers.md b/site/docs/blog/posts/2025-04-25-development-and-documentation-tools-in-containers.md new file mode 100644 index 0000000..04300fa --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-development-and-documentation-tools-in-containers.md @@ -0,0 +1,169 @@ +--- +date: 2025-04-25T09:28:00 +authors: + - markbac +categories: + - Home lab +slug: development-and-documentation-tools-in-containers +--- + +# Development and Documentation Tools in Containers + +*Part 28 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +In today's fast-paced development world, having a robust and flexible environment is crucial. There's no need for a fancy office setup or a massive server farm to get started. With Docker containers, you can self-host your development tools right at home, even on a modest NAS setup. This post will explore how to run developer productivity tools in Docker, transforming your NAS into a powerful dev environment. + + + +## Self-hosting Development Tools + +Self-hosting your tools offers control and privacy. Whether it's about editing code remotely, sketching diagrams, or hosting a knowledge wiki, doing it all within your own network means you can customise the setup to your heart's content without worrying about third-party changes or outages. + +## Benefits of Docker for Development + +Docker is like the Swiss Army knife of the software world: + +- **Consistency**: "Works on my machine" isn't an issue; it works everywhere. +- **Isolation**: Each container is its own entity, preventing messy dependency conflicts. +- **Portability**: Move your containers between machines or scale with ease. +- **Efficiency**: Containers are lightweight and start up quickly. + +It's akin to having a player who can perform consistently regardless of the pitch conditions—a manager's dream! + +## Development Tools + +Let's dive into some specific tools that can transform your development workflow. + +### code-server (VS Code in the Browser) + +Visual Studio Code has become a staple for many developers. With `code-server`, you can run VS Code in your browser: + +- **Remote Accessibility**: Access your IDE from any device with internet access. +- **Extensions Support**: All the familiar extensions are available. +- **Customisability**: Tailor the environment to fit your needs. + +Imagine being able to tweak your code from anywhere—like adjusting tactics from the sideline during a match. + +### Gitpod (Brief Mention) + +If you're after something more managed but still want remote access, Gitpod provides cloud-based workspaces. However, it might not fully align with the self-hosted ethos if you're keen on keeping everything under one roof. + +## Documentation Tools + +Proper documentation is as vital as the development itself—it's like having clear game plans before each match. + +### Wiki.js or MediaWiki (for a Wiki) + +These tools allow you to maintain comprehensive documentation: + +- **Wiki.js**: Modern interface, supports Markdown, and easy to set up. +- **MediaWiki**: The backbone of Wikipedia; robust but requires more setup effort. + +Both options turn your documentation into an accessible resource for anyone who needs it—think of them as your tactical playbook. + +### Excalidraw (for Diagrams) + +Excalidraw offers a straightforward way to sketch diagrams: + +- **Intuitive Interface**: Drag-and-drop elements make diagramming simple. +- **Collaboration Ready**: Share and edit with colleagues in real-time. + +It's like drawing up plays on a whiteboard during half-time—a quick and effective way to convey ideas. + +### Draw.io (Brief Mention) + +For those needing more detailed technical diagrams, Draw.io offers more features. However, it can be overkill for simpler tasks where Excalidraw excels. + +## Setup and Configuration + +Now onto setting these tools up using Docker Compose—a tool that simplifies managing multi-container applications. Here's how you can do it: + +### Docker Compose Deployment + +A typical `docker-compose.yml` might look like this: + +```yaml +version: '3' +services: + code-server: + image: codercom/code-server + ports: + - "8443:8443" + volumes: + - /path/to/data:/home/coder/project + wiki-js: + image: requarks/wiki + ports: + - "3000:3000" + volumes: + - /path/to/wiki:/wiki/data + excalidraw: + image: excalidraw/excalidraw + ports: + - "5000:5000" +``` + +### Persistence and Data Management + +It's crucial to manage data persistence properly: + +- **Volumes** ensure data isn't lost when containers are recreated. +- Regular backups prevent unexpected data losses—akin to ensuring there's always another ball on standby during training. + +## Workflow Integration + +Let's look at how all these tools fit together: + +```mermaid +graph TD; + A[Code] --> B[Diagram]; + B --> C[Documentation]; + C --> D[Deployment]; +``` + +This flow illustrates how code changes lead to updated diagrams and documentation before culminating in deployment—a seamless pipeline like moving the ball fluidly from defence to attack. + +### Connecting to Your NAS + +Integrating these tools with your NAS makes them accessible across your home network. Most modern NAS systems support Docker natively—turning them into versatile platforms beyond just storage. + +## Use Cases + +Here's where rubber meets road—how can these setups be used? + +### Remote Development + +Develop from anywhere without needing your main machine—it's like having home advantage even when playing away. + +### Collaborative Documentation + +Teams spread across locations can contribute and access shared resources anytime—like having scouts feeding live updates during matches. + +### Knowledge Sharing + +Host internal wikis for sharing expertise across teams—ensuring everyone understands their roles and responsibilities on the field. + +## Best Practices + +A few tips for maintaining a healthy self-hosted ecosystem: + +### Security + +Ensure only authorised users have access by setting strong passwords and enabling HTTPS—like securing the dressing room so only players and staff enter. + +### Backups + +Regularly back up both data volumes and configuration files using scheduled tasks—remember Murphy's Law! + +### Updates + +Keep images updated to protect against vulnerabilities while benefiting from new features—it’s akin to keeping players fit for peak performance. + +## Conclusion + +Empowering developers with self-hosted tools enables flexibility and control over their environment. By leveraging Docker on your NAS or similar hardware, you create an adaptable workspace tailored precisely for your needs. It's less about having flashy stadiums but ensuring fundamentals are spot-on—the foundation of any championship-winning side! + +--- + +> **Takeaway** +> Embrace self-hosting with Docker not just as cost-saving but enhancing productivity through customisation whilst maintaining full control over security aspects—turning every project into potential silverware! diff --git a/site/docs/blog/posts/2025-04-25-dns-the-internets-directory-service.md b/site/docs/blog/posts/2025-04-25-dns-the-internets-directory-service.md new file mode 100644 index 0000000..ab15883 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-dns-the-internets-directory-service.md @@ -0,0 +1,105 @@ +--- +date: 2025-04-25T09:10:00 +authors: + - markbac +categories: + - Networking +slug: dns-the-internets-directory-service +--- + +# DNS: The Internet's Directory Service + +*Part 10 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## Introduction - What is DNS? (Domain Name System) + +Imagine needing to call someone but only having their phone number. You'd have to remember every single one, which would make your brain ache more than listening to a manager who just discovered Agile! Luckily, we have phonebooks (or contact lists on mobiles these days), so we can use names instead of numbers. + + + +The Domain Name System (DNS) works similarly for the internet. It translates human-friendly domain names like `www.example.com` into IP addresses like `192.0.2.1`, which computers use to communicate with each other. Without DNS, navigating the web would be as pleasant as a soggy match day at Old Trafford. + +## DNS Query Process + +### Recursive vs. Iterative Queries + +When you type a web address into your browser, a DNS query kicks off to find the corresponding IP address. Think of it like sending your mate to the pub to find out if it's open; they might ask several people along the way. + +- **Recursive Query**: Your DNS resolver takes responsibility for fetching the answer, like your mate asking everyone until they get an answer. +- **Iterative Query**: You get redirected from person to person (or server to server) until you finally reach the right bloke who knows when the pub opens. + +### DNS Resolvers and Servers + +There are different types of DNS servers involved in this process: + +- **Root Servers**: The starting point for all queries. They don't know much themselves but will point you in the right direction. +- **TLD Servers**: Responsible for top-level domains like `.com`, `.uk`, etc. +- **Authoritative Servers**: These hold the actual records you're looking for, akin to asking the barman directly. + +Here's a diagram that illustrates this process: + +```mermaid +graph TD; + A[User] --> B[Local Resolver]; + B --> C[Root Server]; + C --> D[TLD Server]; + D --> E[Authoritative Server]; + E --> B; + B --> A; +``` + +## DNS Caching + +Caching in DNS is like jotting down your mate's answer about the pub's open hours, so you don't need to ask again. + +- **Benefits**: + - **Speed**: Faster response times as you don't need to traverse multiple steps. + - **Reduced Load**: Less traffic for upstream servers, saving them from a digital sweat. + +- **TTL (Time to Live)**: This defines how long a cached query stays valid before you need to check again—like remembering that pubs don't stay open forever and sometimes things change! + +## DNS Security + +As with most things online, security is an issue. DNS wasn't initially designed with security in mind, so additional measures are needed. + +### DNSSEC (DNS Security Extensions) + +DNSSEC adds a layer of authentication using cryptographic signatures, ensuring that the data hasn't been tampered with—much like checking the seal on your pint glass isn't broken. + +### DoH (DNS over HTTPS) + +Encrypts DNS queries via HTTPS, similar to whispering secrets in someone's ear rather than shouting across a crowded room. This helps prevent eavesdropping by third parties. + +### DoT (DNS over TLS) + +Another encryption method that secures queries using TLS—think of it as another whispering technique but through a different method. Both improve privacy and security significantly. + +## Practical Applications + +### Home Network DNS Settings + +Configuring your home network's DNS can improve speed and reliability. Consider using public resolvers like Google or Cloudflare for better performance and privacy compared to default ISP settings. + +### Pi-hole for Ad-blocking + +Pi-hole acts as a network-wide ad blocker by refusing requests for known ad-serving domains—a bit like putting up a "No Junk Mail" sign on your virtual letterbox. It's straightforward to set up on a Raspberry Pi: + +1. Install Raspbian on your Raspberry Pi. +2. Run `curl -sSL https://install.pi-hole.net | bash`. +3. Configure your router or devices to use Pi-hole as their primary DNS server. + +This setup can significantly cut down on ads and trackers across all devices connected to your network. + +## Conclusion - Importance of DNS for Internet Functionality + +DNS is crucial for the internet's smooth operation, much like referees in football—they often go unnoticed until something goes wrong! It helps translate memorable domain names into IP addresses computers understand while offering security measures like DNSSEC and encrypted protocols like DoH and DoT for privacy protection. + +### TL;DR + +- **DNS**: Translates domain names into IP addresses. +- **Query Process**: Involves recursive and iterative methods with root, TLD, and authoritative servers. +- **Caching**: Speeds up responses and reduces load. +- **Security Enhancements**: Includes DNSSEC, DoH, and DoT. +- **Practical Uses**: Configure home networks efficiently and use Pi-hole for ad-blocking. + +With this knowledge under your belt, you're better equipped to understand and manage how your devices connect online—ensuring faster, safer, and more reliable internet browsing experiences. diff --git a/site/docs/blog/posts/2025-04-25-encryption-basics-protecting-data-in-transit-and-at-rest.md b/site/docs/blog/posts/2025-04-25-encryption-basics-protecting-data-in-transit-and-at-rest.md new file mode 100644 index 0000000..e5cd72d --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-encryption-basics-protecting-data-in-transit-and-at-rest.md @@ -0,0 +1,86 @@ +--- +date: 2025-04-25T09:14:00 +authors: + - markbac +categories: + - Networking +slug: encryption-basics-protecting-data-in-transit-and-at-rest +--- + +# Encryption Basics: Protecting Data in Transit and at Rest + +*Part 14 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Ah, encryption. It's the secret sauce that keeps your data safe from prying eyes, much like a lock on your front door keeps out unwanted visitors. In this post, we'll take a stroll through the fundamental principles of encryption, touching on symmetric key methods, asymmetric encryption, and examples such as HTTPS and full-disk encryption. Whether you're new to this or just need a refresher, grab a brew, and let's dive in. + + + +## Introduction + +### What is Encryption? + +In simple terms, encryption is the process of converting information or data into a code to prevent unauthorised access. Think of it as turning your plain-text messages into an unreadable jumble unless you have the right key to decipher it. + +### Importance of Protecting Data + +Why all this fuss about encryption? Well, with cyber threats lurking like foxes in the henhouse, protecting data in transit (as it moves across networks) and at rest (stored on devices) is crucial. Without encryption, sensitive information like financial records or personal communications would be as exposed as a sheep in an open field. + +## Symmetric Encryption + +Symmetric encryption uses a single key for both encrypting and decrypting data. Imagine having one key that locks and unlocks your front door—simple but effective. + +- **Examples**: AES (Advanced Encryption Standard) is one of the most widely used algorithms. +- **Speed and Efficiency**: Symmetric encryption is quick and efficient because it's less computationally intensive. + + ```mermaid + graph TD; + Plaintext -->|Encrypt| Ciphertext; + Ciphertext -->|Decrypt| Plaintext; + Key[Single Key] --> Encrypt; + Key --> Decrypt; + ``` + +- **Key Distribution Challenges**: The main headache? Safely sharing the key without it falling into the wrong hands. It's like passing your house key through the post; not ideal if someone else intercepts it. + +## Asymmetric Encryption + +Asymmetric encryption involves two keys: a public key for encrypting data and a private key for decrypting it. This method is akin to sending a locked box (publicly available lock) to someone who can unlock it with their unique key (private). + +- **Examples**: RSA (Rivest-Shamir-Adleman) is the go-to algorithm here. +- **Key Exchange and Digital Signatures**: Facilitates secure key exchange over insecure channels and provides digital signatures to verify identities. Much like signing a letter with your unique signature to prove authenticity. + +## TLS (Transport Layer Security) + +When we talk about encryption in transit, TLS springs to mind. It's what makes HTTPS secure by encrypting data between your browser and the server. + +- **HTTPS and Web Security**: Every time you see that little padlock icon next to a URL, TLS is working its magic. It ensures that any data sent over the web remains confidential and tamper-proof. + +## Encryption at Rest + +This form of encryption protects data that's stored on devices—much like locking up valuable tools in a shed when they're not being used. + +- **Full-Disk Encryption**: As the name suggests, this encrypts everything on a disk drive. +- **File Encryption**: Targets specific files rather than entire disks. Handy if you want to protect sensitive documents without encrypting everything. + +## Practical Applications + +### Securing Communications + +Whether it's emails, messages, or video calls, encryption ensures your conversations are private—like whispering secrets instead of shouting them across the room. + +### Protecting Stored Data + +Encryption keeps dormant data safe from unauthorised access. Think of it as fitting your castle with thick walls against would-be invaders. + +## Conclusion + +Encryption plays a fundamental role in modern cybersecurity by safeguarding both moving and stationary data. It's an essential tool in our digital toolbox that helps maintain privacy and security in an ever-connected world. + +### TL;DR + +- **Symmetric Encryption**: Quick but needs secure key distribution. +- **Asymmetric Encryption**: More secure for exchanging keys but computationally heavier. +- **TLS**: Essential for securing web communication. +- **Encryption at Rest**: Protects stored data on devices. + +Remember, just like keeping your car locked when parked or watching out for offside traps in football, using encryption wisely can keep your digital assets secure from potential threats. diff --git a/site/docs/blog/posts/2025-04-25-essential-dashboards-for-home-labs-heimdall-homarr-flame.md b/site/docs/blog/posts/2025-04-25-essential-dashboards-for-home-labs-heimdall-homarr-flame.md new file mode 100644 index 0000000..18dd8cb --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-essential-dashboards-for-home-labs-heimdall-homarr-flame.md @@ -0,0 +1,116 @@ +--- +date: 2025-04-25T09:27:00 +authors: + - markbac +categories: + - Home lab +slug: essential-dashboards-for-home-labs-heimdall-homarr-flame +--- + +# Essential Dashboards for Home Labs: Heimdall, Homarr, Flame + +*Part 27 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## The Need for a Central Dashboard + +If you're anything like me, juggling multiple services in your home lab can feel like herding cats. Each service is running in its container, doing its bit to make your smart home or DIY server setup tick. But keeping track of all these services? That's another story. You need a central dashboard—a place where you can see everything at a glance, just like the control room at Old Trafford (without the football drama). + + + +## Organising Home Lab Services + +When you've got containers scattered across machines and networks, organising them effectively becomes crucial. A good dashboard tool acts as a launchpad, giving you quick access to manage and monitor each service without having to SSH into every device or scroll through endless bookmarks. + +## Dashboard Tools + +Let's dive into three popular dashboard tools: Heimdall, Homarr, and Flame. Each has its quirks and strengths, much like choosing between a trusty Land Rover, a sleek Jaguar, or an eco-friendly Nissan Leaf. + +### Heimdall + +Heimdall is my go-to—simple yet powerful. It's like the Swiss Army knife of dashboards: compact but incredibly versatile. It doesn’t try to do too much. Instead, it focuses on being an efficient application dashboard with a focus on user-friendliness. + +### Homarr + +Homarr offers a slightly different approach, focusing on aesthetics and ease of use. Think of it as the Apple of dashboards—sleek design with intuitive usability. It's ideal if you want something that looks good on your big screen display in the living room. + +### Flame + +Flame is another solid option. It's open-source and very community-driven, offering flexibility to tinkerers who love customisation—like those modding their cars or setting up hi-fi systems from scratch. + +## Key Features + +Each dashboard tool comes with features that make managing your home lab services easier: + +- **Service Listing**: View all your services in one place. +- **Icon Support**: Use icons for quick identification—like road signs for your digital motorway. +- **Search Functionality**: Quickly find services without scrolling endlessly. +- **Customisation**: Tailor the appearance to suit your personal taste or match your home decor. +- **Integration with Docker**: Easily connect to your Docker containers and update automatically. + +```mermaid +graph LR + A[Dashboard] --> B[Service 1] + A --> C[Service 2] + A --> D[Service 3] +``` + +## Setup and Configuration + +### Docker Compose Deployment + +Most of these dashboards can be easily deployed using Docker Compose, making setup a breeze: + +```yaml +version: '3' +services: + heimdall: + image: linuxserver/heimdall + container_name: heimdall + ports: + - "80:80" + volumes: + - /path/to/config:/config + restart: unless-stopped +``` + +### Adding Services + +Adding services involves pointing the dashboard to your existing services. Think of it as adding new fixtures to your team sheet before kick-off. + +### Customizing Appearance + +Whether you prefer dark mode for late-night tinkering or bright colours to match your garden shed's vibrant paint job, customisation options are plentiful. + +## Comparison + +### Strengths and Weaknesses + +Here's how they stack up: + +- **Heimdall**: Simple and effective; lacks some advanced features. +- **Homarr**: Beautiful UI; might lack deeper integrations some users need. +- **Flame**: Highly customisable; requires more setup effort. + +### Choosing the Right Tool for Your Needs + +It boils down to what you're looking for: + +- Want simplicity? Go Heimdall. +- After aesthetics? Homarr's your mate. +- Love tweaking? Flame will keep you busy. + +## Use Cases + +These dashboards serve various purposes beyond just being pretty faces: + +- **Quick Access to Services**: Instantly jump into any service without fumbling around. +- **Monitoring Service Status**: Keep an eye on what's running smoothly or needs attention. +- **Sharing Access with Family Members**: Give others in the household access without overwhelming them with complexity. + +## Conclusion + +Enhancing your home lab experience with a centralised dashboard is akin to equipping your workshop with power tools instead of relying on elbow grease alone. Whether it's Heimdall's straightforwardness, Homarr's polished look, or Flame's flexibility, there's something here for every tech enthusiast. + +Takeaway? Choose the tool that fits your style and needs best—your digital workshop will thank you for it. + +Happy tinkering! 🍻 diff --git a/site/docs/blog/posts/2025-04-25-firewalls-what-they-are-and-why-you-need-one.md b/site/docs/blog/posts/2025-04-25-firewalls-what-they-are-and-why-you-need-one.md new file mode 100644 index 0000000..aa6fd98 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-firewalls-what-they-are-and-why-you-need-one.md @@ -0,0 +1,135 @@ +--- +date: 2025-04-25T09:12:00 +authors: + - markbac +categories: + - Networking +slug: firewalls-what-they-are-and-why-you-need-one +--- + +# Firewalls: What They Are and Why You Need One + +*Part 12 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Picture this: your network is like a bustling football stadium. You’ve got fans coming in and out, each with their own agenda. Some just want to enjoy the match (your regular traffic), while others might be up to no good (malicious actors). The firewall is your security team, checking tickets and ensuring only the right people get through. Without it, chaos reigns supreme. + + + +In this post, we'll dive into the nitty-gritty of firewalls—what they are, how they work, and why they’re not just important, but essential for securing your network. + +## Introduction + +### What is a Firewall? + +A firewall acts as a barrier between your internal network and potential threats from the outside world. It filters traffic based on pre-established security rules, much like a bouncer at a club deciding who gets in and who stays out. + +### Importance of Firewalls in Network Security + +- **Protects against cyber threats**: Shields your network from malicious attacks and unauthorised access. +- **Ensures data privacy**: Keeps sensitive information safe from prying eyes. +- **Regulates traffic**: Manages incoming and outgoing network traffic efficiently. + +In essence, without a firewall, your data is as vulnerable as leaving your front door wide open. + +## Basic Firewall Concepts + +### Packet Filtering + +At its core, packet filtering involves analysing small chunks of data (packets) against a set of filters. It's like checking each fan's ticket before entering the stadium. If the ticket doesn’t match the criteria, they’re not getting in. + +### Stateful vs. Stateless Firewalls + +- **Stateless Firewalls**: These operate purely on rules. They don’t remember previous packets—each one is judged on its own merits. +- **Stateful Firewalls**: A bit like having a memory. They track active connections and make decisions based on the state of these connections. + +```mermaid +graph TD; + A[Incoming Packet] -->|Check Rules| B{Stateless Firewall} + B -->|Allow/Deny| C[Network] + A -->|Track State| D{Stateful Firewall} + D -->|Allow/Deny| C +``` + +### Firewall Rules and Policies + +Rules are essentially "if-then" statements that dictate what happens when certain types of traffic are encountered. Policies are collections of these rules that govern overall network behaviour. + +## Firewall Types + +### Network Firewalls + +These protect entire networks by filtering traffic between different network segments. Think of them as the main gates to your stadium. + +### Host-Based Firewalls + +Installed on individual devices, these act like personal bodyguards for each computer or server, adding an extra layer of protection. + +### Hardware vs. Software Firewalls + +- **Hardware Firewalls**: Physical devices placed between your network and the gateway to the internet. +- **Software Firewalls**: Applications installed on devices within your network. + +*Tip*: For home users, a software firewall on each device plus a router with built-in firewall capabilities can provide robust protection. + +## Advanced Firewall Features + +### Application-Layer Firewalls + +These delve deeper than traditional firewalls by inspecting data at the application level. Imagine checking not just tickets but also IDs to ensure legitimacy. + +### Intrusion Prevention Systems (IPS) + +An IPS works alongside firewalls to detect and prevent known threats in real-time—like having sniffer dogs at the gate to catch anything suspicious. + +### Network Address Translation (NAT) and Firewalls + +NAT hides internal IP addresses from the outside world, adding another layer of security by making your network less visible to potential attackers. + +## Home and Enterprise Use Cases + +### Home Network Firewall Setup + +Most modern routers come with built-in firewall features. For example, an Asus router can be configured with simple steps: + +1. Log in to router admin panel. +2. Navigate to firewall settings. +3. Enable packet filtering and set up basic rules—block all except essential services like HTTP/HTTPS. + +### Enterprise Firewall Deployment + +In an enterprise setting, firewalls are critical for maintaining secure operations across multiple offices or branches: + +- Deploy hardware firewalls at key points within the corporate network. +- Use advanced firewalls with IPS capabilities for sensitive departments handling critical data. + +## Best Practices + +### Rule Management + +Regularly review and update firewall rules to adapt to new threats and changes within your network environment. + +### Logging and Monitoring + +Keep an eye on logs for any unusual activity—this helps in spotting potential breaches early on. + +### Regular Updates + +Ensure firmware and software updates are applied promptly to close off vulnerabilities as they’re discovered. + +*Note*: An outdated firewall is akin to using last season's playbook—it leaves you exposed! + +## Conclusion + +Firewalls are indispensable players in your cybersecurity lineup. They offer the first line of defence against malicious intrusions while also managing network traffic efficiently. Whether it's for a home setup or an enterprise deployment, understanding their role ensures you're well-equipped to keep your data safe from harm. + +So next time you connect to the internet, think of your firewall as not just a guardian but an ever-watchful coach guiding you towards victory in the digital arena. + +--- + +**Takeaway Summary** + +- Firewalls are essential for protecting networks from unauthorised access. +- Understanding basic concepts like packet filtering and stateful inspection is key. +- Choose the right type of firewall based on specific needs—network vs host-based, hardware vs software. +- Advanced features like IPS add layers of security. +- Regular rule updates and monitoring are crucial for effective defence. diff --git a/site/docs/blog/posts/2025-04-25-home-network-design-tips-from-a-real-setup.md b/site/docs/blog/posts/2025-04-25-home-network-design-tips-from-a-real-setup.md new file mode 100644 index 0000000..c4d6c18 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-home-network-design-tips-from-a-real-setup.md @@ -0,0 +1,160 @@ +--- +date: 2025-04-25T09:19:00 +authors: + - markbac +categories: + - Home lab +slug: home-network-design-tips-from-a-real-setup +--- + +# Home Network Design: Tips from a Real Setup + +*Part 19 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## Importance of a Well-Designed Home Network + +Imagine your home network as the plumbing in a house. If it's well-designed, everything flows smoothly, and you hardly notice it. But if it's poorly planned, you'll constantly be dealing with blockages and leaks. In today's connected world, where everything from your fridge to your light bulbs might be online, having a robust and efficient home network is as important as having good electrics or central heating. + + + +## Overview of My Home Network Setup + +In this post, I'll give you a peek into my own home network setup. It's not just a bunch of cables and blinking lights; it's a carefully planned architecture that supports advanced features like VLANs, ad-blocking, and local DNS. I'll walk you through my choices and share some nuggets of wisdom I’ve picked up along the way. + +## Network Components + +Let's start by looking at the key players in my network lineup: + +### Asus BT8 Router + +- **Role:** This is the backbone of my network. It handles routing, DHCP, and basic firewall duties. +- **Why I Chose It:** Solid performance and reliability. Plus, it supports advanced features like VLANs out of the box. + +### Synology 723+ NAS + +- **Role:** Storage and service hub. +- **Why I Chose It:** Versatile with great software support for Docker, making it ideal for running various services. + +### Raspberry Pi Devices + +- **Pi-hole:** This acts as my network-wide ad blocker. +- **Home Assistant:** Manages all my home automation tasks. + +These little Pis are like Swiss Army knives — small but incredibly handy. + +## Network Architecture + +Now, let’s dive into the architecture — the blueprint that ensures everything works seamlessly together. + +### VLANs for Segmentation + +Think of VLANs (Virtual LANs) as invisible fences within your network. They keep different types of traffic separate: + +- **Guest Network:** Isolated from your main devices for security. +- **IoT Devices:** Kept on their own VLAN to prevent any dodgy fridge from snooping on your PC. + +### Wired and Wireless Setup + +A mix of wired connections for speed-critical devices (e.g., PCs, NAS) and wireless for convenience (e.g., phones, tablets). + +```mermaid +graph TD; + A[Internet] --> B[Asus BT8 Router]; + B --> C[VLAN 1 - Main Devices]; + B --> D[VLAN 2 - IoT Devices]; + B --> E[VLAN 3 - Guest Network]; + C --> F[Synology 723+ NAS]; + C --> G[Raspberry Pi - Pi-hole]; + C --> H[Raspberry Pi - Home Assistant]; +``` + +## Key Services + +Let’s talk about the services that make my network tick. + +### Pi-hole for Ad-blocking + +- **Function:** Blocks ads network-wide. +- **Benefit:** Cleaner browsing experience without needing browser plugins. + +### Home Assistant for Home Automation + +This is like having a butler who never sleeps: + +- **Capabilities:** Controls smart lights, thermostats, even the coffee maker. +- **Integration:** Works with almost any smart device out there. + +### Synology NAS for Storage and Services + +Not just a storage box: + +- **Docker & Portainer:** Run various containers for different services. Portainer makes managing them easy. +- **Heimdall Dashboard:** Acts as a homepage for all services — think of it as a digital control panel. + +## Setup and Configuration + +Here’s where we get our hands dirty configuring things. + +### Router Configuration + +Set up basic networking rules: + +- Assign static IPs to key devices. +- Configure port forwarding for external access to selected services. + +### VLAN Setup + +Involves setting up different networks within your router: + +1. Define VLAN IDs in the router settings. +2. Assign devices to appropriate VLANs based on their role. + +### Service Integration + +Ensure all services can communicate where necessary: + +- Connect Home Assistant with other devices using MQTT or similar protocols. +- Use Pi-hole's DNS capabilities to streamline service discovery. + +## Tips and Best Practices + +Let's touch on some practical advice that can save you headaches down the line. + +### Security Considerations + +Just like locking your doors at night: + +- Use strong passwords everywhere. +- Enable firewall rules to block unwanted traffic. + +> **Tip**: Regularly update firmware/software to patch vulnerabilities. + +### Performance Optimisation + +Make sure your network runs like a well-oiled machine: + +- Prioritise bandwidth for critical devices using QoS (Quality of Service). +- Monitor network performance regularly to spot bottlenecks early. + +### Expansion and Future-Proofing + +Plan ahead like you're building an extension on your house: + +- Leave room in your IP address range for new devices. +- Consider future technologies like Wi-Fi 6 or mesh networks. + +## Conclusion + +Having a customised home network isn't just about bragging rights; it brings real benefits: + +- Enhanced security by isolating devices with VLANs. +- Improved user experience through ad-blocking and streamlined device management. +- Flexibility to add new technology without overhauling everything. + +Setting up an advanced home network might seem daunting at first, but breaking it down into manageable steps makes it achievable. With careful planning and consideration, you can create a setup that not only meets current needs but also adapts to whatever the future throws at it. Now put the kettle on – you've earned it! + +--- + +**Takeaway Summary** + +A well-planned home network is crucial for seamless connectivity in today's tech-driven homes. By strategically using components like routers, NAS systems, and Raspberry Pi devices, along with implementing VLANs for segmentation, you can achieve both high performance and robust security. Always plan with an eye towards future expansion and keep security practices up-to-date. diff --git a/site/docs/blog/posts/2025-04-25-how-devices-join-the-network-dhcp-explained.md b/site/docs/blog/posts/2025-04-25-how-devices-join-the-network-dhcp-explained.md new file mode 100644 index 0000000..36ecd61 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-how-devices-join-the-network-dhcp-explained.md @@ -0,0 +1,118 @@ +--- +date: 2025-04-25T09:07:00 +authors: + - markbac +categories: + - Networking +slug: how-devices-join-the-network-dhcp-explained +--- + +# How Devices Join the Network: DHCP Explained + +*Part 7 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Imagine you're setting up a small farm, and you need to make sure each animal has its own space. You could manually assign spots, but that’s a faff, especially as the herd grows. Instead, you use a system that automatically assigns each one a spot based on what's available. This is basically what DHCP does for your network devices. + + + +## Introduction: Manual vs. Automatic IP Configuration + +Back in the day, assigning IP addresses manually was like herding cats. Each device needed an IP address typed in by hand—prone to errors and definitely not scalable. Enter DHCP: Dynamic Host Configuration Protocol. It's like having a digital shepherd overseeing your network, ensuring everyone has a place without stepping on each other's toes. + +### Benefits of DHCP + +- **Efficiency**: Automates IP address assignment. +- **Scalability**: Easily handles large networks. +- **Flexibility**: Quickly adapts to changes in the network. +- **Reduced Errors**: Minimises human error in configuration. + +## DHCP Process + +DHCP operates through a four-step dance between client and server. Let's break it down: + +1. **DHCP Discover**: A device (client) sends out a broadcast request saying, "I need an IP!" +2. **DHCP Offer**: The DHCP server replies with an offer, like a friendly wave, saying "Here's an available IP." +3. **DHCP Request**: The client responds, "I'll take that IP, please." +4. **DHCP Acknowledgement (ACK)**: Finally, the server confirms, "It's all yours!" + +Here’s how it looks in a diagram: + +```mermaid +sequenceDiagram + participant Client + participant Server + + Client->>Server: DHCP Discover + Server->>Client: DHCP Offer + Client->>Server: DHCP Request + Server->>Client: DHCP ACK +``` + +This process ensures each device gets a unique IP address without manual intervention. + +## DHCP Leases + +When a device gets an IP address via DHCP, it's not permanent; it's leased. + +- **Lease Duration**: Think of it like renting a flat; you can stay as long as the lease lasts. + - Short leases are good for networks with lots of new devices frequently connecting. + - Longer leases reduce DHCP traffic but may risk running out of available addresses if too many devices join. + +- **Lease Renewal**: Before the lease expires, devices must request to renew their lease to keep the same IP address. + +## DHCP Reservations + +Sometimes you want certain devices to always have the same IP address—like reserving the best parking spot for your tractor. + +- **Assigning Specific IPs**: You can configure reservations based on MAC addresses so specific devices always receive the same IP. + +- **Use Cases**: + - Servers where consistent addressing is crucial. + - Network printers that multiple users access. + +## Static vs. Dynamic IPs + +Understanding when to use static or dynamic IPs is key to effective network management. + +### Differences and Use Cases + +- **Static IPs**: + - Manually assigned and never change unless you intervene. + - Ideal for servers and network equipment where stability is paramount. + +- **Dynamic IPs**: + - Assigned by DHCP and can change over time. + - Perfect for general devices like laptops and smartphones. + +### When to Use Each + +- Use static IPs for critical infrastructure. +- Use dynamic IPs for general devices to ease management. + +## DHCP Options + +Beyond just handing out IPs, DHCP can configure other settings: + +- **Gateway Configuration**: Tells devices which router to use as their gateway to the outside world. +- **DNS Server Configuration**: Informs devices which DNS servers to use for translating domain names into IP addresses. + +These options ensure all networked devices are singing from the same hymn sheet. + +## Conclusion + +DHCP is like the unsung hero on your network—a digital traffic warden managing device addresses seamlessly. Without it, managing any sizeable network would be tedious at best and chaotic at worst. + +With its ability to dynamically allocate and manage addresses while providing flexibility and reducing errors, DHCP is vital for modern networks. Whether you're running an enterprise-scale setup or just keeping your home gadgets connected smoothly, understanding DHCP will help you keep everything running like clockwork. + +--- + +## Key Takeaways + +- **DHCP automates** the process of assigning IP addresses on a network. +- It involves a four-step process of discover, offer, request, and acknowledge. +- Leases define how long an IP address is assigned to a device before requiring renewal. +- Reservations help ensure critical devices have consistent addressing. +- Static IPs suit stable infrastructure; dynamic ones suit mobile or general-purpose devices. +- Additional options allow configuration of gateways and DNS servers through DHCP. + +Remember this next time you set up or troubleshoot your network—it might just save you from pulling your hair out! diff --git a/site/docs/blog/posts/2025-04-25-how-the-internet-actually-works-isps-ixps-routing.md b/site/docs/blog/posts/2025-04-25-how-the-internet-actually-works-isps-ixps-routing.md new file mode 100644 index 0000000..f206ba6 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-how-the-internet-actually-works-isps-ixps-routing.md @@ -0,0 +1,143 @@ +--- +date: 2025-04-25T09:17:00 +authors: + - markbac +categories: + - Networking +slug: how-the-internet-actually-works-isps-ixps-routing +--- + +# How the Internet Actually Works (ISPs, IXPs, Routing) + +*Part 17 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## Introduction + +Imagine the internet as a sprawling metropolis, full of roads, bridges, and highways. It's vast, complex, and constantly buzzing with activity. But how does it all come together? This post will take you on a journey through the architecture of the internet, exploring the key components like ISPs, IXPs, and routing protocols that make it all tick. + + + +We'll delve into: + +- **ISPs (Internet Service Providers):** The gatekeepers connecting us to the internet. +- **IXPs (Internet Exchange Points):** The hubs where networks collaborate. +- **Routing:** The method by which data finds its way across this digital landscape. + +So grab your brew, settle in, and let's unravel the mystery of how the internet actually works. + +## ISPs (Internet Service Providers) + +### Tiers of ISPs + +Think of ISPs as the utility companies of the internet world. They provide the infrastructure that connects our homes and businesses to this massive network. + +- **Tier 1 ISPs:** These are the big players—global companies like AT&T or BT that own large portions of the internet's backbone. They don't pay for transit because they peer with each other. + +- **Tier 2 ISPs:** These often serve regional areas. They connect to Tier 1 networks and may peer with other Tier 2s to reduce costs. + +- **Tier 3 ISPs:** These are your local ISPs, providing service directly to consumers and small businesses. They usually purchase access from Tier 2 providers. + +### How ISPs Connect Users to the Internet + +ISPs use a mix of fibre optics, copper lines, and wireless technologies to bring the internet to your doorstep. Here’s how it generally works: + +1. **Local Loop:** This is the connection from your home to your local exchange—think of it as your driveway leading to a main road. + +2. **Network Backbone:** Once data leaves your local loop, it travels over high-capacity lines managed by Tier 1 or Tier 2 providers. + +3. **Peering & Transit:** Data hops between different ISP networks at peering points to reach its destination. + +## IXPs (Internet Exchange Points) + +### Peering and Transit + +IXPs are like bustling roundabouts where multiple roads converge, allowing traffic from different networks to flow smoothly. + +- **Peering:** When two networks exchange traffic directly without charging each other—like two neighbours agreeing to share their garden hose without any money changing hands. + +- **Transit:** One network pays another for access to parts of the internet they can't reach directly—like paying a toll to use someone else’s motorway. + +### Reducing Latency and Improving Efficiency + +By facilitating direct connections between networks, IXPs play a crucial role in reducing latency (the time it takes for data to travel from point A to B) and improving efficiency. Imagine fewer roundabouts on your morning commute; you get there faster with less hassle. + +```mermaid +graph TD; + A[Local Network] -->|Peering| B[ISP 1]; + B --> C[IXP]; + C -->|Transit| D[ISP 2]; + D --> E[Global Internet]; +``` + +## Routing + +### How Data Packets Travel Across the Internet + +Data on the internet travels in packets—small chunks that find their way through a maze of routers and switches. Picture each packet as a letter in an envelope with an address written on it. + +Here's how it happens: + +1. **Packet Creation:** Your device breaks down data into packets. + +2. **Routing Decision:** Each router along the way decides which path is best for forwarding these packets based on various factors like speed or cost. + +3. **Destination Arrival:** Packets reach their destination where they’re reassembled into usable information. + +### Autonomous Systems (AS) + +An Autonomous System is a collection of IP networks and routers under the control of one entity—like a single team managing all the buses in a city. + +- Each AS is assigned a unique number (ASN). +- ASes make independent routing decisions but coordinate through peering agreements. + +### BGP (Border Gateway Protocol) + +BGP is like air traffic control for data packets—it decides which routes data should take across different ASes. + +**Key Features:** + +- **Path Selection:** Chooses routes based on policies rather than shortest paths alone. + +- **Robustness:** Designed to handle network failures by finding alternative paths. + +```plaintext +AS1 --> AS2 --> AS3 --> Destination +AS1 <-- AS4 <-- AS5 <-- Source +``` + +## Traceroute + +### Visualizing the Path of Data + +Traceroute is a handy tool that helps you see the path data takes from your computer to its destination—a bit like following bread crumbs through a forest. + +**Usage:** + +```bash +traceroute example.com +``` + +This command reveals each hop along the route, helping diagnose where delays or problems might occur. + +### Troubleshooting Network Issues + +When something goes wrong on your network excursion: + +- Use traceroute to pinpoint where packets are getting lost or delayed. +- Identify if it's an issue with your ISP or further upstream. + +## Internet Governance + +Just like we have rules for driving on roads, there are organisations that set standards for how data travels over networks: + +- **ICANN (Internet Corporation for Assigned Names and Numbers):** Manages domain names and IP addresses. + +- **IETF (Internet Engineering Task Force):** Develops technical standards that ensure smooth communication between devices on different networks. + +## Conclusion + +The internet is a marvel of collaboration—a patchwork quilt stitched together by thousands of different networks working in harmony. Whether you're streaming your favourite show or sending an email halfway around the world, it's this intricate dance between ISPs, IXPs, and routing protocols that makes it all possible. + +Takeaway: Next time you click 'send', remember there's more going on behind those bits than meets the eye! + +If you've made it this far without nodding off—well done! You've got a better grasp than most on how this fantastic virtual world operates. Now go impress someone with your newfound knowledge over another brew! diff --git a/site/docs/blog/posts/2025-04-25-iot-networking-smart-devices-and-their-challenges.md b/site/docs/blog/posts/2025-04-25-iot-networking-smart-devices-and-their-challenges.md new file mode 100644 index 0000000..09f9b1e --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-iot-networking-smart-devices-and-their-challenges.md @@ -0,0 +1,124 @@ +--- +date: 2025-04-25T09:21:00 +authors: + - markbac +categories: + - Home lab +slug: iot-networking-smart-devices-and-their-challenges +--- + +# IoT Networking: Smart Devices and Their Challenges + +*Part 21 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## What is IoT (Internet of Things)? + +Imagine your kettle talking to your fridge. No, it's not the start of a bad joke; it's the Internet of Things (IoT). In simple terms, IoT refers to a network of physical objects—"things"—embedded with sensors, software, and other technologies to connect and exchange data with other devices and systems over the Internet. It's like having a team of very chatty assistants running around your house or workplace. + + + +## Growth and Impact of IoT Devices + +The growth of IoT has been monumental. From smart thermostats to industrial sensors, these devices are transforming how we live, work, and play. According to some estimates, there will be over 75 billion connected IoT devices by 2025. That's more than the number of cups of tea consumed in Britain each year! + +But this growth comes with its fair share of challenges, especially when it comes to networking. Let's dive into how these devices connect and what hurdles they throw our way. + +## IoT Connectivity + +### Wi-Fi + +Wi-Fi is like the bustling high street of IoT connectivity: ubiquitous, convenient, but often crowded. It provides ample bandwidth for data-hungry applications but can suffer from interference and range limitations. + +### Zigbee and Z-Wave + +These are like those narrow country lanes: less travelled but vital for specific routes. Zigbee and Z-Wave are low-power protocols ideal for home automation due to their mesh networking capabilities. However, they typically require a hub for internet connectivity. + +### Cellular + +Think of cellular networks as the motorways of IoT—fast, wide-reaching, but with tolls (data costs). Ideal for mobile or remote devices where Wi-Fi isn't available, cellular connectivity supports higher data rates but can be expensive. + +### Other Technologies (Bluetooth, LoRaWAN) + +- **Bluetooth**: Great for short-range connections; think headphones or smartwatches. +- **LoRaWAN**: Offers long-range communication at low power and is perfect for agricultural or environmental monitoring where devices are spread out like sheep on a hillside. + +## IoT Protocols + +### MQTT (Message Queuing Telemetry Transport) + +MQTT is like a postal service for small packages—efficient and lightweight. Designed for constrained environments with limited bandwidth, it uses a publish-subscribe model. Devices publish messages to a broker, which then distributes them to subscribers. + +```mermaid +graph LR + A[Device 1] -- Publish --> B(MQTT Broker) + B -- Distribute --> C[Device 2] + B -- Distribute --> D[Device 3] +``` + +### CoAP (Constrained Application Protocol) + +CoAP is akin to HTTP's little brother—designed for simplicity and low overhead. It's ideal for small IoT devices with limited resources and supports multicast communications, making it suitable for group messaging. + +## IPv6 and IoT + +### Addressing the Scalability of IoT + +Remember when everyone switched from analogue TVs? Well, IPv6 is the digital switchover for IP addresses. With billions more addresses available compared to IPv4, IPv6 ensures that we don't run out of addresses as IoT devices flood the market. + +### IPv6 Adoption + +Adoption has been slow—like getting kids to eat their veg—but it's happening. Many ISPs now offer IPv6 support, which is crucial as we integrate more IoT devices into our networks. + +## IoT Security Challenges + +### Vulnerabilities and Attacks + +IoT devices can be as vulnerable as an open barn door without proper security measures. Common attacks include: + +- **Botnets**: Devices hijacked to perform malicious activities. +- **Data breaches**: Sensitive information being leaked or stolen. + +### Privacy Concerns + +There's also the issue of privacy. Imagine your smart speaker eavesdropping on conversations about Aunt Mildred's dodgy knee—that's not something anyone wants! + +### Device Management + +Managing numerous devices can feel like herding cats. Ensuring firmware updates and consistent configurations across all devices is critical yet challenging. + +## Securing IoT Networks + +### Network Segmentation (VLANs) + +Think of VLANs as fences in a field keeping different flocks apart. By segmenting your network, you can isolate IoT devices from critical parts of your system. + +### Strong Authentication + +Use strong authentication methods—like a shepherd's whistle—to ensure only authorised users can access your network. + +### Regular Updates + +Keep your systems updated regularly. It's like servicing your car; neglect leads to breakdowns at the worst times. + +### Example: Securing IoT Devices on a Home Assistant Network + +Consider setting up separate VLANs for your Home Assistant setup. It keeps your smart light bulbs from meddling with sensitive data traffic on your main network. + +```mermaid +graph TD + A[Main Network] + B[VLAN 1 - Home Assistant] -->|Isolated| A + C[VLAN 2 - Guest Network] -->|Isolated| A +``` + +## Conclusion + +Navigating the complexities of IoT networking isn't always straightforward—it's more like navigating Yorkshire moors in the fog! However, by understanding connectivity options, protocols, security challenges, and best practices, you can create robust and secure environments for your smart devices. + +### Takeaways + +- **Understand Protocols**: Use MQTT or CoAP where appropriate. +- **Embrace IPv6**: It's the future-proof solution. +- **Prioritise Security**: Segment networks and keep everything updated. + +By keeping these tips in mind, you'll be well-equipped to handle the wild world of IoT networking without losing your bearings—or any data! diff --git a/site/docs/blog/posts/2025-04-25-ip-addresses-and-subnetting-demystified.md b/site/docs/blog/posts/2025-04-25-ip-addresses-and-subnetting-demystified.md new file mode 100644 index 0000000..b28ab4d --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-ip-addresses-and-subnetting-demystified.md @@ -0,0 +1,130 @@ +--- +date: 2025-04-25T09:05:00 +authors: + - markbac +categories: + - Networking +slug: ip-addresses-and-subnetting-demystified +--- + +# IP Addresses and Subnetting Demystified + +*Part 5 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Networking can seem a bit like trying to understand the offside rule in football: it sounds simple until you try to explain it. Yet, just like mastering the offside trap, getting your head around IP addresses and subnetting is crucial for anyone dabbling in networking. So, grab a brew and let’s break down these concepts into digestible chunks. + + + +## Introduction + +### What is an IP address? + +An IP address is essentially a unique identifier for a device on a network, much like how your home address identifies where you live. It allows data to find its way across the vast expanse of networks to reach the correct device. Think of it as the logical addressing system of the internet. + +### Why are IP addresses necessary? + +Without IP addresses, sending data over the internet would be akin to posting a letter with no address – it’d never reach its destination. They ensure every device can communicate efficiently and effectively, whether it’s your laptop streaming Netflix or your smart fridge ordering more milk. + +## IP Address Structure + +### IPv4 vs. IPv6 + +- **IPv4**: The older version, using 32 bits for an address (e.g., `192.168.1.1`), which supports about 4.3 billion unique addresses. +- **IPv6**: Introduced due to IPv4 exhaustion, using 128 bits (e.g., `2001:0db8:85a3:0000:0000:8a2e:0370:7334`), supporting a mind-boggling number of addresses. + +The transition from IPv4 to IPv6 is much like shifting from a village of cottages to a sprawling city – plenty more room! + +### Octets and Dotted Decimal Notation + +IPv4 addresses are split into four octets (8-bit groups) separated by dots. Each octet can range from 0 to 255 – so next time you see `192.168.0.1`, think of it as four houses on a street. + +### Network and Host Portions + +IP addresses consist of two parts: + +- **Network Portion**: Identifies the specific network. +- **Host Portion**: Identifies the specific device on that network. + +Imagine a football team where the club name represents the network portion and individual player names are the host portion. + +## IP Address Classes (A, B, C) + +Back in the day, IP addresses were divided into classes to make allocation easier: + +- **Class A**: Supports large networks with many devices (`1.0.0.0` to `126.0.0.0`) +- **Class B**: Supports medium-sized networks (`128.0.0.0` to `191.255.0.0`) +- **Class C**: Caters to smaller networks (`192.0.0.0` to `223.255.255.0`) + +Each class had a default subnet mask defining which part of an address was network versus host – like default formations in football tactics. + +## Subnetting + +### Why Subnetting? + +Subnetting divides a large network into smaller, manageable chunks: + +- **Efficiency**: Reduces broadcast traffic, making each subnet less congested. +- **Security**: Isolates segments, preventing one compromised subnet from affecting others. + +Think of subnetting as dividing fields for crop rotation – better yield and reduced risk of disease spreading. + +### Subnet Mask Explained + +A subnet mask defines which part of an IP address is network vs host: + +- Example: Given `192.168.1.10` with subnet mask `255.255.255.0`, the first three octets (`192.168.1`) denote network, and the last (`10`) denotes host. + +### Creating Subnets (Simple Examples) + +To create subnets: +1. Determine how many subnets you need. +2. Borrow bits from the host portion for additional network portions. + +For instance: +- Starting with `192.168.1.x`, use subnet mask `255.255.255.192` (or `/26`) for four subnets with 62 hosts each. + +```mermaid +graph TB + A[Network] --> B[Subnet 1] + A --> C[Subnet 2] + A --> D[Subnet 3] + A --> E[Subnet 4] +``` + +## CIDR (Classless Inter-Domain Routing) + +### CIDR Notation (/8, /16, /24) + +CIDR offers more flexibility than traditional class-based addressing by allowing arbitrary-length prefixes: + +- `/8`: Large networks +- `/16`: Medium-sized +- `/24`: Smaller networks + +### Benefits of CIDR (Flexibility) + +CIDR helps in efficient address allocation and reduces wastage – akin to fitting more passengers on a bus without empty seats. + +## Practical Applications + +### Home Network Subnetting + +For home setups: +- Use CIDR notation like `/24` for easy management. +- Separate IoT devices from personal computers to enhance security. + +### Enterprise Network Examples + +In enterprises: +- Allocate subnets per department using CIDR for scalability. +- Use VLANs alongside subnets for even greater control and isolation. + +## Conclusion + +Understanding IP addressing and subnetting forms the bedrock of robust network management – much like knowing your formations in football ensures tactical success on the pitch. + +## TL;DR + +IP addresses are essential for identifying devices on a network; subnetting splits large networks into manageable sections for efficiency and security; CIDR provides flexibility in address allocation. + +So next time you're configuring your home router or planning an enterprise network, remember these fundamentals – they’ll keep your data flowing smoothly! diff --git a/site/docs/blog/posts/2025-04-25-keeping-docker-in-check-with-watchtower.md b/site/docs/blog/posts/2025-04-25-keeping-docker-in-check-with-watchtower.md new file mode 100644 index 0000000..08028fa --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-keeping-docker-in-check-with-watchtower.md @@ -0,0 +1,116 @@ +--- +date: 2025-04-25T09:30:00 +authors: + - markbac +categories: + - Home lab +slug: keeping-docker-in-check-with-watchtower +--- + +# Keeping Docker in Check with Watchtower + +*Part 30 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Ah, Docker. It's like the Swiss Army knife of modern software development, isn't it? Containers have become the go-to solution for deploying applications efficiently and reliably. But with great power comes great responsibility, as some superhero once said. Keeping those containers updated is crucial—not just for security, but for stability and new features too. + + + +## Importance of Keeping Containers Updated + +Think of your Docker containers as a fleet of tractors on your farm. Each one needs regular maintenance to work smoothly, from oil changes (security patches) to upgrading that old carburettor (new features). Ignoring updates can leave you open to vulnerabilities or mean you're missing out on improved performance. + +## Automation vs Manual Updates + +Now, you could manually update each container like a meticulous farmer tending to each tractor—rewarding but time-consuming. Enter automation! Tools like Watchtower can automatically update your containers, acting as your trusty farmhand. But there's a balance to strike between automation and control, which we'll explore in this post. + +## Watchtower + +Let's delve into Watchtower—your new best mate for keeping Docker containers up-to-date without lifting a finger. + +### How Watchtower Works + +Watchtower monitors your running Docker containers and checks if any updates are available for the images they're based on. If an update is found, it gracefully stops the current container, pulls the latest image, and starts it again. + +Here's a simple way to get Watchtower up and running: + +```bash +docker run -d \ + --name watchtower \ + -v /var/run/docker.sock:/var/run/docker.sock \ + containrrr/watchtower +``` + +### Automatic Updates + +Watchtower checks for updates at regular intervals (you can configure this), so it's a bit like having a farmhand who wanders around checking each tractor's oil level every hour. + +### Configuration Options + +You can tweak Watchtower's behaviour with various configuration options: + +- **Interval**: Set how often Watchtower checks for updates. +- **Notifications**: Get alerts via Slack or email when updates occur. +- **Rolling Updates**: Control how updates are applied across multiple containers. + +## Pros of Automatic Updates + +Why should you let Watchtower do its thing? Here are some compelling reasons: + +- **Security Patches**: Automatically get critical security updates applied promptly. +- **New Features**: Enjoy the latest features without manual intervention. +- **Bug Fixes**: Benefit from resolved issues that might affect your application. + +## Cons of Automatic Updates + +Of course, automation isn't all sunshine and rainbows. Here's where things might go awry: + +- **Breaking Changes**: An auto-update might introduce changes that break compatibility. +- **Instability**: New versions can be less stable than their predecessors. +- **Lack of Control**: You lose some oversight over what's being changed. + +> **Warning:** Before automating everything, consider whether you need to maintain strict version control over certain critical services. + +## Version Pinning + +Now, sometimes you'll want to pin versions of specific containers to avoid unexpected surprises—like ensuring all your tractors use the same type of fuel. + +### When to Pin Container Versions + +Pin versions when: + +- You rely on specific functionality that newer versions might alter or remove. +- You're managing complex dependencies across multiple services. + +```yaml +image: postgres:12.3 +``` + +The above ensures you're always running PostgreSQL version 12.3 and not accidentally pulling in the latest (potentially incompatible) version. + +## Best Practices + +To keep your system running smoothly while leveraging automation, consider these best practices: + +### Testing Updates + +Always test new versions in a staging environment before rolling them out. Think of it as taking a prototype tractor for a spin before deploying it across the farm. + +### Backups + +Regular backups are essential. If something goes pear-shaped after an update, you'll want a way back to the previous state. + +### Monitoring + +Implement monitoring solutions to detect anomalies quickly following an update. This way, you catch issues before they cause serious downtime. + +## Conclusion + +Balancing automation with control in Docker updates is like managing a football team—you need strategy and adaptability. Automated tools like Watchtower make it easier to stay on top of updates without micromanaging every detail, but they require careful consideration and testing. + +### Key Takeaways + +- **Automate Wisely**: Use Watchtower for routine updates but keep an eye out for critical systems where manual checks are necessary. +- **Version Pinning**: Pin known-stable versions when reliability is paramount. +- **Continuous Improvement**: Regularly review and refine your update strategy based on new insights and technologies. + +And there you have it—a straightforward guide to keeping Docker containers updated with Watchtower while maintaining control over your environment. Now go put the kettle on; you've earned it! diff --git a/site/docs/blog/posts/2025-04-25-mac-addresses-and-arp-finding-devices-on-a-network.md b/site/docs/blog/posts/2025-04-25-mac-addresses-and-arp-finding-devices-on-a-network.md new file mode 100644 index 0000000..1b42333 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-mac-addresses-and-arp-finding-devices-on-a-network.md @@ -0,0 +1,130 @@ +--- +date: 2025-04-25T09:06:00 +authors: + - markbac +categories: + - Networking +slug: mac-addresses-and-arp-finding-devices-on-a-network +--- + +# MAC Addresses and ARP: Finding Devices on a Network + +*Part 6 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +When you think of a MAC address, imagine it as the fingerprint of your device on a network. It's as unique as a bespoke football kit number, identifying each piece of kit in the field. While it's not the only factor that makes up your "team" (or network), it's crucial for determining who's who in the lineup. + + + +This blog post will take you on a journey through the fascinating world of MAC addresses and ARP (Address Resolution Protocol). By the end, you'll understand how devices are identified on a network, why ARP is essential, and the potential pitfalls lurking in this seemingly straightforward process. + +## MAC Address Definition + +A **MAC address** (Media Access Control address) is often referred to as a physical address because it's hardwired into your device's network interface card (NIC). It's used to uniquely identify devices on a local network segment. Here's what you need to know: + +- **Uniqueness and Purpose**: Each MAC address is supposed to be unique across all network interfaces worldwide. This uniqueness ensures that data packets reach the correct destination within a local network. +- **Analogy**: Think of a MAC address as the VIN (Vehicle Identification Number) of your car—unique to every vehicle and vital for identification. + +## MAC Address Structure + +### 48-Bit Address + +A MAC address consists of 48 bits, typically represented as six pairs of hexadecimal digits. Imagine it like a postcode—it’s got its structure, and each part tells you something. + +### OUI (Organizationally Unique Identifier) + +The first three octets (24 bits) represent the **Organizationally Unique Identifier** (OUI). It identifies the manufacturer of the device. The rest is assigned by the manufacturer to ensure uniqueness. + +- **Example**: In `00-14-22-01-23-45`, `00-14-22` is the OUI, indicating the manufacturer. + +## ARP (Address Resolution Protocol) + +With all these unique identifiers flying around, you might wonder why we need anything else. Enter ARP—the protocol that bridges IP addresses with MAC addresses. + +### Why ARP is Needed + +While IP addresses are great for routing traffic across different networks, they don't directly map to how data physically travels along wires or through switches in your local network. That's where ARP comes into play: + +- **Purpose**: Resolving an IP address to its corresponding MAC address, ensuring data packets reach their correct physical destination. + +### ARP Request and Reply Process + +Here's how it works: + +1. **ARP Request**: A device broadcasts an ARP request packet asking, "Who has IP address X.X.X.X? Tell me your MAC." +2. **ARP Reply**: The device with IP X.X.X.X responds with its MAC address. + +```mermaid +sequenceDiagram + participant A as Device A + participant B as Device B + A->>B: Who has IP X.X.X.X? + B-->>A: I have IP X.X.X.X, my MAC is XX:XX:XX:XX:XX:XX +``` + +### ARP Tables + +Devices maintain an **ARP table**, a sort of contacts list mapping IP addresses to MAC addresses for quick reference—much like keeping a list of regular players and their positions in your favourite Sunday league team. + +## Layer 2 Networking + +Layer 2 is where networking rubber meets the road—here's how switches and MAC addresses interact: + +### Switches and MAC Address Forwarding + +Switches use MAC addresses to forward data frames only to intended recipients within a local area network (LAN). They maintain a **MAC address table** which maps ports to MAC addresses, ensuring efficient delivery without unnecessary broadcasts. + +### Broadcast Domains + +A broadcast domain is the set of all devices that can reach each other by broadcast at layer 2. All devices on a single Ethernet switch share one broadcast domain unless VLANs are used. + +## ARP Spoofing + +Now for something more sinister—**ARP spoofing**, where attackers attempt to associate their own MAC address with an IP address belonging to another device on the network. + +### What is ARP Spoofing? + +An attacker sends fake ARP messages onto the network, tricking devices into sending data meant for another IP address to them instead. + +### Potential Consequences + +- **Man-in-the-Middle Attacks**: Intercepted traffic allows attackers to eavesdrop or alter communication. +- **Denial of Service**: Disrupts legitimate communication between devices. + +### Basic Mitigation Techniques + +- **Static ARP Entries**: Manually configure critical devices with fixed mappings. +- **Use Secure Protocols**: Encrypt sensitive traffic using SSL/TLS or VPNs. + +> [!Caution] +> Despite being useful, static ARP entries can become cumbersome in large networks due to manual configuration complexity. + +## MAC Filtering + +Another layer of defence is **MAC filtering**, which controls access based on whether a device's MAC address appears on an allowed list. + +### How MAC Filtering Works + +Here's how it functions: + +- Network hardware checks incoming frames against an allowed list before forwarding data. + +### Limitations and Security Considerations + +While helpful for basic security: + +- **Spoofing Threats**: Attackers can spoof allowed MAC addresses. + +> [!Tip] +> Use in combination with other security measures like strong encryption and regular monitoring for best results. + +## Conclusion + +As we've seen, understanding how MAC addresses and ARP work together is fundamental for managing and securing modern networks. Whether you're configuring your home IoT setup or managing enterprise-level infrastructure, awareness of these concepts helps prevent potential security issues. + +### Key Takeaways + +- **Essential Role**: MAC addresses are vital identifiers within local networks. +- **ARP's Importance**: Bridges logical IP addressing with physical hardware locations. +- **Security Awareness**: Stay vigilant against threats like ARP spoofing; consider layered security approaches including encryption and access control measures. + +By mastering these basics, you're better equipped to tackle both everyday networking tasks and more advanced challenges down the line—much like knowing both when to pass short in midfield and when to hoof it clear from defence on a rainy day at St James' Park! diff --git a/site/docs/blog/posts/2025-04-25-nas-what-it-is-and-why-you-might-want-one.md b/site/docs/blog/posts/2025-04-25-nas-what-it-is-and-why-you-might-want-one.md new file mode 100644 index 0000000..538958a --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-nas-what-it-is-and-why-you-might-want-one.md @@ -0,0 +1,167 @@ +--- +date: 2025-04-25T09:20:00 +authors: + - markbac +categories: + - Home lab +slug: nas-what-it-is-and-why-you-might-want-one +--- + +# NAS: What It Is and Why You Might Want One + +*Part 20 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Ah, Network Attached Storage, or NAS for short. It's one of those techy terms that sounds like it belongs in a sci-fi novel but is actually quite down-to-earth and incredibly useful. Whether you’re looking to keep your family photos safe, stream your favourite films across the house, or share files with your small business team without shouting across the office, a NAS might just be your new best friend. + + + +## What is a NAS? + +In plain English, a NAS is a dedicated storage device connected to your network. Think of it as a personal library where all your digital goodies live, accessible from any device connected to your home or office network. Unlike an external hard drive that connects directly to one computer, a NAS serves data over the network, making it the communal cookie jar for all things digital. + +## Benefits of a NAS + +- **Centralised Storage:** All your files in one place—no more "where did I save that?" moments. +- **Accessibility:** Files available to anyone on the network without needing to plug in and out like musical chairs. +- **Scalability:** Start small and expand as needed; just add more drives. +- **Energy Efficiency:** Consumes less power than a full-blown server. + +## NAS Functionality + +Alright then, let’s dive into what makes a NAS tick. + +### Centralised Storage + +Gone are the days of USB drives hidden in desk drawers. With a NAS, you have: + +- **Easy Organisation:** Like having a well-labelled filing cabinet for everything digital. +- **Remote Access:** Access your files from anywhere with internet access (great for when you're off gallivanting). + +### File Sharing + +Think of this like passing notes in class but much more efficient and less likely to get confiscated by the teacher. + +- **Collaboration:** Share documents with colleagues or family members easily. +- **Cross-device Compatibility:** Works with PCs, Macs, smartphones—whatever you’ve got. + +### Media Streaming + +Fancy turning your living room into a mini cinema? A NAS can transform into: + +- **Media Server:** Stream films, music, and photos to smart TVs and media players. +- **Plex Integration:** With Plex Media Server, organise and stream content effortlessly. + +### Backup Solutions + +A bit like having an insurance policy for your data: + +- **Automated Backups:** Scheduled backups mean no more manual saving. +- **Versioning & Snapshots:** Recover previous versions if you make a mess of something. + +## Storage Protocols + +Understanding some of the lingo here helps when setting things up. + +### SMB (Server Message Block) + +The go-to protocol for Windows machines. It allows seamless file sharing across networks—just like sharing biscuits at tea time. + +### NFS (Network File System) + +Favoured by Linux and Unix systems. It provides faster performance compared to SMB but requires a bit more setup elbow grease. + +### AFP (Apple Filing Protocol) + +For those living in Apple's orchard, AFP ensures smooth sailing when accessing files from Mac devices. + +## NAS Use Cases + +Let's look at where a NAS can really shine. + +### Home Media Server (Plex) + +Turn your home into an entertainment hub: + +- Organise films and music with Plex. +- Stream content to any device around the house—perfect for family film night. + +### Time Machine Backups + +For Mac users who want easy-peasy backups: + +- Set up Time Machine to back up directly to the NAS without lifting a finger. + +### Personal Cloud Storage + +Imagine Dropbox but without monthly fees: + +- Store and access files from anywhere. +- Maintain control over who gets access—no peeking neighbours here! + +### Small Office File Sharing + +Ideal for small teams needing shared resources: + +- Central file repository reduces email clutter. +- Enables collaborative work on documents without duplication nightmares. + +## NAS Hardware and Configuration + +Choosing the right gear is crucial—like picking the right size welly boots for mucking about in fields. + +### Drive Configurations (RAID) + +RAID isn't just about avoiding traffic jams: + +- **RAID 0:** Speed boost but no redundancy—like driving fast with no seatbelt. +- **RAID 1:** Mirroring for redundancy—two drives holding identical data. + +Mermaid Diagram: +```mermaid +graph LR + A[Single Disk] -->|No Redundancy| B(RAID 0) + A -->|Redundancy| C(RAID 1) +``` + +### Network Connectivity + +Ensure your NAS has good Wi-Fi or Ethernet capabilities—it’s how it talks to everything else. + +### NAS Operating Systems (e.g., Synology DSM) + +Like choosing between Android or iOS but for storage: + +- Intuitive interfaces make management straightforward. + +## NAS Considerations + +Before you rush out to buy one, here are some things to chew over: + +### Cost + +Not just initial purchase but potential upgrades over time. Don’t buy more than you need initially; you can scale later. + +### Performance + +Consider what you'll use it for—media streaming demands more horsepower than simple file storage. + +### Security + +With great power comes great responsibility: + +- Ensure firmware updates are regular. +- Set up user permissions carefully to avoid unwanted guests snooping around. + +## Conclusion + +A NAS is like having your own private server without the faff of running one. It’s versatile enough for both home use and small office setups, offering benefits from centralised storage to media streaming. So whether you're looking to beef up your backup game or just want easier access to your files from anywhere, a NAS could be just what you need—without breaking the bank or requiring an IT degree. + +--- + +**Takeaway Summary:** + +1. **Centralised Storage & Sharing**: Keep all digital assets in one place while ensuring easy access across devices. +2. **Versatile Use Cases**: From media servers to personal cloud storage—even small business solutions. +3. **Cost & Security**: Mindful investment with attention towards security settings will pay dividends in peace of mind. + +There you have it—a comprehensive look at why a NAS might be worth considering next time you're pondering how best to manage your digital life. Grab yourself a brew and mull it over! diff --git a/site/docs/blog/posts/2025-04-25-nat-and-port-forwarding-talking-to-the-outside-world.md b/site/docs/blog/posts/2025-04-25-nat-and-port-forwarding-talking-to-the-outside-world.md new file mode 100644 index 0000000..26edea1 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-nat-and-port-forwarding-talking-to-the-outside-world.md @@ -0,0 +1,131 @@ +--- +date: 2025-04-25T09:08:00 +authors: + - markbac +categories: + - Networking +slug: nat-and-port-forwarding-talking-to-the-outside-world +--- + +# NAT and Port Forwarding: Talking to the Outside World + +*Part 8 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Imagine your home as a bustling little village with a single post office (your router) that handles all incoming and outgoing mail (data). Now, each house in this village has its own address, but to the outside world, they only see one big address — the post office's. This is the basic premise of Network Address Translation (NAT). + + + +In today's digital landscape, almost every device we own wants to talk to the outside world — whether it's our smartphones, smart TVs, or gaming consoles. However, there's a finite number of public IP addresses available. So how do all these devices communicate with the internet while using just one public IP address? Enter NAT. + +Let's dive into the nuts and bolts of how NAT and port forwarding work in your home network. + +## NAT (Network Address Translation) + +### How NAT Works + +At its core, NAT translates private IP addresses to a single public IP address — like translating local dialects into a language everyone understands. + +- **Private IP Addresses**: These are like house numbers in our village analogy. They're used within your local network (192.168.x.x, 10.x.x.x). +- **Public IP Addresses**: This is akin to the post office's address. It's how other networks identify your home network on the internet. + +When a device sends data to the internet, NAT changes the device's private IP address to the router's public IP address. The router keeps track of which private address corresponds to each outgoing request so it can correctly route incoming responses back to the right device. + +### Hiding Internal IPs + +NAT acts as a curtain between your internal network and the wider internet. By masking internal addresses, it helps keep your devices from being directly exposed to external threats. + +### Benefits of NAT + +- **Security**: By hiding internal addresses, NAT provides an additional layer of security against unsolicited external connections. +- **IP Conservation**: With IPv4 addresses being scarce, NAT conserves them by enabling multiple devices to share a single public IP. + +## Port Forwarding + +While NAT is great for outgoing requests, what if you want external traffic to reach specific devices inside your network? That's where port forwarding steps in. + +### Allowing External Access to Internal Services + +Port forwarding works like directing certain types of mail (say, football magazines) to a specific home in our village. It tells your router how to handle specific types of incoming traffic and which internal device should receive it. + +### Port Numbers and Their Purpose + +Ports are analogous to specific doors at a house; they represent different services or applications: + +- **HTTP (Web Servers)**: Port 80 +- **HTTPS (Secure Web Servers)**: Port 443 +- **FTP (File Transfer Protocol)**: Port 21 + +Each service listens on its designated port(s), and port forwarding ensures that incoming requests reach the correct service on your network. + +### Configuring Port Forwarding on a Router + +Most modern routers offer user-friendly interfaces for setting up port forwarding: + +1. Log into your router's web interface. +2. Navigate to the port forwarding section. +3. Enter: + - The internal IP address of the device. + - The port number(s) you wish to forward. + - The protocol (TCP/UDP) if necessary. +4. Save changes and test connectivity. + +## Home Networking Examples + +Let's explore some real-world scenarios where you'd use NAT and port forwarding. + +### Web Server Hosting + +If you're hosting a website from home: +- Set up port forwarding for HTTP/HTTPS ports (80/443) pointing to your server's private IP. +- This setup will direct incoming web traffic correctly, allowing external users to access your site. + +```mermaid +graph LR + Internet -->|Request| Router -->|Forwarded| WebServer +``` + +### Game Server Setup + +For gaming enthusiasts wanting friends to join their hosted games: +- Determine which ports the game uses. +- Configure port forwarding for those ports so players can connect directly to your gaming PC or console. + +### Remote Access to Devices + +Want remote access to a home security camera? +- Forward its specific management port so you can monitor it from anywhere with an internet connection. + +## Double NAT + +### What is Double NAT? + +Double NAT occurs when two routers are performing NAT on your network — for example, an ISP-supplied modem/router combo connected to another personal router. This can complicate things like online gaming or remote desktop access because packets have two layers of translation before reaching their destination. + +### Issues and Solutions + +Problems caused by double NAT include: +- Difficulty hosting online games. +- Issues with VPNs or remote access tools. + +Solutions: +- **Bridge Mode**: Configure one device (typically the ISP modem/router) in bridge mode so it acts solely as a modem. +- **DMZ Settings**: Use DMZ settings on one router for an easier setup, effectively bypassing one layer of NAT for particular devices. + +## Conclusion: Importance of NAT and Port Forwarding + +NAT and port forwarding are essential tools in modern networking: + +- They allow multiple devices within a home network to share a single public IP address efficiently. +- They enable controlled access from external networks while maintaining security through restricted exposure. + +When setting up these features, always consider security implications — ensure only necessary ports are open and regularly review configurations for any vulnerabilities. + +--- + +### Key Takeaways + +- **NAT** allows multiple devices on a local network to communicate with external networks using one public IP address. +- **Port Forwarding** directs specific types of inbound traffic to particular devices within your network. +- Understanding and configuring these elements effectively can enhance both functionality and security in home networking setups. + +By grasping these concepts, you'll be better equipped to manage your home network like a seasoned pro! diff --git a/site/docs/blog/posts/2025-04-25-network-tools-and-automation-in-docker.md b/site/docs/blog/posts/2025-04-25-network-tools-and-automation-in-docker.md new file mode 100644 index 0000000..f4cd5cd --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-network-tools-and-automation-in-docker.md @@ -0,0 +1,157 @@ +--- +date: 2025-04-25T09:29:00 +authors: + - markbac +categories: + - Home lab +slug: network-tools-and-automation-in-docker +--- + +# Network Tools and Automation in Docker + +*Part 29 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## Introduction + +Ah, the allure of a well-oiled smart home and network that runs as smoothly as your favourite football team on a good day. But let's face it, managing networks can sometimes feel like herding cats—unpredictable and occasionally chaotic. Enter Docker, your new best mate for taming network chaos with automation and documentation. Whether you're a tinkerer, a home automation enthusiast, or just someone who likes to keep things organised, Dockerised tools can really make your life easier. + + + +In this blog post, we'll explore how you can leverage Docker to automate your smart home, document architectures, and track those pesky IP/network changes. Ready? Let’s dive in. + +## Enhancing Network Management with Docker + +Docker is like having a toolbox with all your favourite spanners and screwdrivers neatly arranged—everything's contained, reproducible, and portable. By containerising network management tools, you not only streamline deployment but also enhance scalability and maintainability. + +### Automation and Documentation + +- **Automation**: Automate repetitive network tasks using visual programming or integrated systems. +- **Documentation**: Capture and visualise network architecture to avoid the "what does this wire do?" syndrome. + +## Network Automation + +### Node-RED + +Node-RED is like the Swiss Army knife of network automation—versatile and powerful. It’s a flow-based programming tool that lets you connect hardware devices, APIs, and online services in new and interesting ways. Perfect for visual learners who prefer not to drown in lines of code. + +- **Visual Programming**: Drag-and-drop interface to create automation flows. +- **Integration**: Connects easily with various IoT devices and services. + +### Home Assistant Integration + +Home Assistant is your go-to platform for smart home control. Integrating Node-RED with Home Assistant is like putting peanut butter with jam—each great on its own but even better together. + +- **Control Centre**: Manage all smart devices from one interface. +- **Automation**: Use Node-RED flows to automate tasks like turning lights on at sunset or adjusting the thermostat when you leave home. + +## Network Documentation + +### Structurizr + +Documenting network architecture can be tedious work akin to writing assembly instructions for flat-pack furniture. Structurizr simplifies this by allowing you to create C4 model diagrams that are both comprehensive and easy on the eyes. + +```mermaid +graph TD; + A[User] --> B[Web Server]; + B --> C[Application Server]; + C --> D[Database]; +``` + +### Archi (Brief Mention) + +For those who prefer ArchiMate for enterprise architecture modelling, Archi is worth a look. It's a bit more heavyweight but offers robust features for those deep dives into complex systems. + +## Network Monitoring + +### IP Snap + +Is there anything more annoying than an IP address change wreaking havoc on your carefully curated setup? IP Snap helps track these changes so you're never caught off guard again. + +- **IP Tracking**: Monitors IP address changes in real-time. +- **Alerts**: Get notified when changes occur. + +### Rejex (Log Analysis) + +Think of Rejex as the log detective—sifting through data to find clues about what's going wrong (or right) in your system. + +- **Log Parsing**: Analyse logs efficiently using regex patterns. +- **Insightful Reporting**: Generate reports to identify trends or issues. + +## Setup and Configuration + +Deploying these tools using Docker Compose is like setting up a production line—each component has its place, making it easy to manage and update. + +```yaml +version: '3' +services: + nodered: + image: nodered/node-red + ports: + - "1880:1880" + homeassistant: + image: homeassistant/home-assistant + ports: + - "8123:8123" +``` + +### Connecting Tools Together + +The beauty of Docker is how easily these tools can be interconnected: + +- Use Node-RED to trigger actions in Home Assistant. +- Monitor changes with IP Snap alerts feeding into structured notifications. + +## Use Cases + +### Smart Home Automation Flows + +Create flows that automate mundane daily tasks: + +- Turn off lights when no movement is detected for 30 minutes. +- Automatically lock doors at night. + +### Network Architecture Diagrams + +Visualise your network setup: + +- Map out connections using Structurizr diagrams. +- Document dependencies between services. + +### Change Management + +Track network changes: + +- Use IP Snap to log address alterations. +- Analyse logs with Rejex for performance tweaks. + +## Best Practices + +### Security + +Like locking the barn door before the horse bolts: + +- Ensure containers are updated regularly. +- Limit access to sensitive components with firewalls. + +### Version Control for Flows/Diagrams + +Keep track of changes and revisions: + +- Use Git repositories for managing Node-RED flows. +- Track Structurizr diagram versions similarly. + +### Documentation + +Don’t skip documenting setups; future-you will thank present-you: + +- Maintain clear descriptions within Node-RED flows. +- Use markdown files alongside diagrams for detailed explanations. + +## Conclusion + +By now, you should have a solid grasp of how Dockerised tools can simplify automating, documenting, and monitoring your network setup. Whether you're just starting out or already knee-deep in cables and configurations, these tools provide a robust framework for managing complexity without losing sight of the bigger picture. So grab a cuppa, fire up Docker, and start tinkering—your future self will thank you! + +--- + +### Takeaway Summary +Docker offers a comprehensive solution for automating network management tasks, documenting architectures, and tracking changes efficiently. By integrating tools like Node-RED, Home Assistant, Structurizr, IP Snap, and Rejex into your workflow, you can achieve greater control over your environment while keeping everything tidy and manageable. diff --git a/site/docs/blog/posts/2025-04-25-network-topologies-star-bus-mesh-and-more.md b/site/docs/blog/posts/2025-04-25-network-topologies-star-bus-mesh-and-more.md new file mode 100644 index 0000000..99eb31d --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-network-topologies-star-bus-mesh-and-more.md @@ -0,0 +1,174 @@ +--- +date: 2025-04-25T09:02:00 +authors: + - markbac +categories: + - Networking +slug: network-topologies-star-bus-mesh-and-more +--- + +# Network Topologies: Star, Bus, Mesh and More + +*Part 2 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +So, you're curious about network topologies. Fair enough—it's the kind of topic that can seem as tangled as a football club's finances. But once you get your head around it, you'll see it's not just about connecting computers; it's about choosing the right structure to keep data flowing smoothly. Let's dig in. + + + +## Introduction + +### What is Network Topology? + +To put it simply, network topology is how you arrange and connect the various parts of a network. Think of it like a road map for data packets, showing them how to get from A to B. + +- **Physical Topology**: This is all about the actual layout—the cables, switches, routers and whatnot. + +- **Logical Topology**: This focuses on how data moves within the network, regardless of its physical design, similar to how traffic might flow through a city's road system. + +### Why Topology Matters + +Choosing the right topology affects two main things: + +- **Performance**: Some layouts are faster or more efficient than others. + +- **Reliability**: Some designs can keep working even if parts of the network fail; others can't. + +## Star Topology + +### Description + +The star topology has a central hub or switch that connects directly to each node (like computers or printers). It's like having one referee who everyone communicates with directly in a football match. + +```mermaid +graph LR + Hub(Hub/Switch) --> Node1(Node 1) + Hub --> Node2(Node 2) + Hub --> Node3(Node 3) + Hub --> Node4(Node 4) +``` + +### Pros + +- **Easy Management**: Adding or removing devices is straightforward. + +- **Fault Isolation**: If one cable fails, only its connected device goes offline—a bit like losing one player to a red card but still being able to play the match. + +### Cons + +- **Single Point of Failure**: If the hub fails, everything grinds to a halt—akin to losing the referee and nobody knowing what to do next. + +### Use Cases + +- **Home Networks**: Most home routers use this setup. + +- **Office LANs**: Perfect for small to medium-sized businesses where simplicity and ease of management are key. + +## Bus Topology + +### Description + +Bus topology uses a single central cable (backbone) that all devices tap into—similar to neighbours sharing one water pipeline. + +```mermaid +graph LR + Backbone(Backbone Cable) --> Node1(Node 1) + Backbone --> Node2(Node 2) + Backbone --> Node3(Node 3) +``` + +### Pros + +- **Simple and Cheap**: Less cabling means lower costs. + +- **Ease of Installation**: Quick setup compared to more complex systems. + +### Cons + +- **Scalability Limits**: More devices mean more collisions—like too many cars on a single-lane road. + +- **Fault Vulnerability**: If the main cable breaks, everything stops—a bit like cutting that shared water pipe. + +### Historical Use Cases + +In days gone by, bus topology was common in smaller networks because of its simplicity and cost-effectiveness. However, it's largely been phased out in favour of more robust systems. + +## Mesh Topology + +### Description + +In mesh topology, every node connects directly to every other node. Imagine it as a robust web where every spider has its own silk line to each other spider! + +```mermaid +graph LR + Node1(Node 1) -- Link --> Node2(Node 2) + Node1 -- Link --> Node3(Node 3) + Node1 -- Link --> Node4(Node 4) + Node2 -- Link --> Node3 + Node2 -- Link --> Node4 + Node3 -- Link --> Node4 +``` + +### Pros + +- **High Redundancy and Reliability**: Loads of alternative paths if one goes down. + +- **Robust Performance**: Excellent for high-demand or critical systems. + +### Cons + +- **Complexity and Cost**: All those connections require lots of cabling and configuration—like building an elaborate model railway set. + +### Use Cases + +- **Critical Systems**: Often used in military communications or financial industries where uptime is crucial. + +- **Some Wireless Setups**: Mesh Wi-Fi networks use this principle for better coverage and reliability at home. + +## Other Topologies (Briefly) + +Let's take a quick look at some other topologies: + +- **Ring Topology**: Nodes form a circle. Data travels in one direction until it reaches its destination. Useful for some types of local area networks (LANs), but less popular now due to potential failure points. + +- **Hybrid Topology**: Combines elements of different topologies (e.g., star-bus). Offers flexibility but can be complex. + +- **Tree Topology**: Hierarchical structure resembling an actual tree (rooted at one point). Commonly used in corporate networks with multiple branches or departments. + +## Comparison Table + +Here's a handy table comparing these topologies: + +| Topology | Pros | Cons | +|---------------|----------------------------------------|-----------------------------------------| +| Star | Easy management, fault isolation | Single point of failure | +| Bus | Simple, cheap | Scalability limits, fault vulnerability | +| Mesh | High redundancy, reliability | Complex, expensive | +| Ring | Predictable performance | One break affects entire network | +| Hybrid | Flexible | Complex | +| Tree | Scalable | Complex configuration | + +## Conclusion + +When choosing a network topology: + +- Consider your specific needs—performance versus cost versus reliability. + +- Smaller setups might favour star or hybrid designs due to their simplicity and ease of expansion. + +- Larger or mission-critical environments might lean towards mesh for its reliability despite higher costs. + +Ultimately, just like deciding whether your Sunday League team should play defensively or go all-out attack, there's no one-size-fits-all answer. It's all about what's best suited for your needs and resources. + +--- + +## TL;DR / Takeaway Summary + +Network topologies define how devices connect within a network. Key types include: + +1. **Star Topology** - Centralised control but vulnerable if the hub fails. + +2. **Bus Topology** - Simple but not scalable. + +3. **Mesh Topology** - Extremely reliable but costly and complex. + +Each has its strengths and weaknesses, just like choosing between pie flavours at your local chippy—it's all about what suits your taste! diff --git a/site/docs/blog/posts/2025-04-25-networking-and-home-lab-series.md b/site/docs/blog/posts/2025-04-25-networking-and-home-lab-series.md new file mode 100644 index 0000000..69d389a --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-networking-and-home-lab-series.md @@ -0,0 +1,50 @@ +--- +date: 2025-04-25T09:00:00 +authors: + - markbac +categories: + - Networking + - Home lab +slug: networking-and-home-lab-series +--- + +# Networking and Home Lab: the Series + +A series on how networks work, from the basics to running a home lab on a NAS with Docker. The early posts explain the building blocks; the later ones apply them to a real home network. + + + +| Post | What it covers | Part of | +| --- | --- | --- | +| [What Is a Network? (LAN, WAN, PAN Explained Simply)](2025-04-25-what-is-a-network-lan-wan-pan-explained-simply.md) | This post introduces the basic concept of networks using relatable scenarios. It explains the differences between LAN, WAN, and PAN, with real-world use cases like home internet, office networks, and Bluetooth devices. The goal is to ground the reader in what networks are and why they matter. | Networking | +| [Network Topologies: Star, Bus, Mesh and More](2025-04-25-network-topologies-star-bus-mesh-and-more.md) | This article explores different network topologies, such as star, bus, and mesh. It breaks down the pros and cons of each, where they're commonly used, and their impact on reliability and performance. A great primer before diving into how data actually moves through these structures. | Networking | +| Understanding the 4-Layer TCP/IP Model | This post demystifies the 4-layer model that underpins modern networking. Each layer is explained with examples of protocols and real-world applications, using metaphors and visuals to keep it grounded and digestible. Essential foundation for understanding more advanced network behaviour. | Not written yet | +| [Switches and Routers: What’s the Difference and Why It Matters](2025-04-25-switches-and-routers-whats-the-difference-and-why-it-matters.md) | Here we define what switches and routers are, how they differ, and why they matter in your home or enterprise network. We explore how switches use MAC addresses to forward data and how routers manage traffic between networks using IPs. NAT and basic routing are introduced. | Networking | +| [Wired vs Wireless: Ethernet and Wi-Fi Compared](2025-04-25-wired-vs-wireless-ethernet-and-wi-fi-compared.md) | A comparison of Ethernet and Wi-Fi, including use-cases, pros and cons, and when to choose one over the other. It covers cabling standards, signal interference, and introduces concepts like Wi-Fi mesh networking. Ideal for home users deciding how to set up their networks. | Networking | +| [IP Addresses and Subnetting Demystified](2025-04-25-ip-addresses-and-subnetting-demystified.md) | This article breaks down the often-intimidating subject of IP addresses and subnetting. It uses analogies and diagrams to explain how IPs are structured, how subnets work, and why they matter in network planning. Includes simple CIDR examples for clarity. | Networking | +| [MAC Addresses and ARP: Finding Devices on a Network](2025-04-25-mac-addresses-and-arp-finding-devices-on-a-network.md) | Focuses on how MAC addresses identify devices and how ARP resolves IP to MAC. Introduces basic Layer 2 networking and the concept of spoofing. Ties into switch functionality and potential network attacks like ARP spoofing. | Networking | +| [How Devices Join the Network: DHCP Explained](2025-04-25-how-devices-join-the-network-dhcp-explained.md) | A detailed look at how DHCP works to assign IP addresses. Explains what leases are, how reservations function, and how static IPs differ. This is foundational for understanding network configuration. | Networking | +| [NAT and Port Forwarding: Talking to the Outside World](2025-04-25-nat-and-port-forwarding-talking-to-the-outside-world.md) | Explains the concepts of Network Address Translation (NAT) and port forwarding using real-world home networking setups. It clarifies how internal devices access the internet and how incoming traffic is routed correctly. Covers issues like double NAT and gaming/router compatibility. | Networking | +| [TCP vs UDP: Choosing the Right Transport Protocol](2025-04-25-tcp-vs-udp-choosing-the-right-transport-protocol.md) | Compares TCP and UDP, detailing how each protocol handles data transmission. Describes scenarios where each is preferable, such as TCP for file transfers and UDP for streaming or gaming. Includes packet structure and connection flow visuals. | Networking | +| [DNS: The Internet's Directory Service](2025-04-25-dns-the-internets-directory-service.md) | Provides a comprehensive look at how DNS works, including query resolution, caching, and modern security enhancements like DNSSEC and DoH. Practical uses like ad-blocking via Pi-hole are included to anchor theory in reality. | Networking | +| [Common Application Protocols: HTTP, HTTPS, FTP, SMTP, IMAP](2025-04-25-common-application-protocols.md) | Overviews popular application-layer protocols, explaining what they do and where they are used. The post touches on security considerations and includes a summary table of default ports and functions. | Networking | +| SSL/TLS: The Technology Behind HTTPS | Demystifies how SSL/TLS provides secure communications over the web. It walks through certificate chains, how HTTPS works, and common issues like expired certificates. Practical tips on checking and managing SSL for websites are included. | Not written yet | +| [Firewalls: What They Are and Why You Need One](2025-04-25-firewalls-what-they-are-and-why-you-need-one.md) | Explains how firewalls protect networks by filtering traffic. Covers basic concepts like packet filtering, stateful inspection, and application-aware firewalls. Real examples from home and enterprise use cases provided. | Networking | +| [Authentication, Authorisation, Accounting (AAA)](2025-04-25-authentication-authorisation-accounting-aaa.md) | Clarifies the distinct but related concepts of AAA. Shows how they're implemented in enterprise and home environments, particularly with Wi-Fi networks, VPNs, and logging systems. | Networking | +| [Encryption Basics: Protecting Data in Transit and at Rest](2025-04-25-encryption-basics-protecting-data-in-transit-and-at-rest.md) | Introduces the core principles of encryption, from symmetric key methods to public key infrastructure. Includes examples of when and where different types are used — from HTTPS to full-disk encryption. | Networking | +| [Virtual LANs (VLANs): Network Segmentation 101](2025-04-25-virtual-lans-vlans-network-segmentation-101.md) | Outlines how VLANs are used to logically segment networks without extra hardware. Explains tagging, trunk ports, and gives a practical home setup using managed switches. | Networking | +| [VPNs: Secure Tunnels for Remote Access and Privacy](2025-04-25-vpns-secure-tunnels-for-remote-access-and-privacy.md) | Details how VPNs work and the differences between types. Practical uses in remote work, privacy, and securing IoT setups. Discusses key protocols like IPsec and OpenVPN. | Networking | +| [How the Internet Actually Works (ISPs, IXPs, Routing)](2025-04-25-how-the-internet-actually-works-isps-ixps-routing.md) | Describes the global architecture of the internet, including ISPs, peering points, and routing protocols like BGP. Explains how data traverses networks and what happens when it goes wrong. | Networking | +| [QoS: Prioritising Traffic for Smooth Performance](2025-04-25-qos-prioritising-traffic-for-smooth-performance.md) | Explains how Quality of Service works to prioritise critical network traffic. Covers DSCP tags, traffic shaping, and where it's useful (like VoIP or streaming). | Networking | +| [Home Network Design: Tips from a Real Setup](2025-04-25-home-network-design-tips-from-a-real-setup.md) | Gives a real-world walkthrough of a modern home network with advanced features like VLANs, ad-blocking, and local DNS. Explains the rationale behind architecture choices and offers setup tips. | Home lab | +| [NAS: What It Is and Why You Might Want One](2025-04-25-nas-what-it-is-and-why-you-might-want-one.md) | Describes what a NAS does and how it's useful in homes and small offices. Looks at common protocols, media streaming, and backup setups. Includes real deployment options and pitfalls. | Home lab | +| [IoT Networking: Smart Devices and Their Challenges](2025-04-25-iot-networking-smart-devices-and-their-challenges.md) | Looks at how IoT devices connect to networks and the challenges they pose. Covers constrained protocols like MQTT and CoAP, IPv6 implications, and best practices for securing IoT environments. | Home lab | +| [ARP Spoofing, MAC Filtering and Network Fingerprinting](2025-04-25-arp-spoofing-mac-filtering-and-network-fingerprinting.md) | Explores advanced Layer 2 attacks and how to defend against them. Discusses ARP spoofing, MAC address filtering, and passive network fingerprinting. | Home lab | +| [Using VLANs and Subnets for Smart Home Security](2025-04-25-using-vlans-and-subnets-for-smart-home-security.md) | Practical guide to using VLANs and subnetting to improve smart home network security. Offers real configuration tips, use cases, and security rationales. | Home lab | +| [Running a Local DNS and DHCP Server for Your Home Lab](2025-04-25-running-a-local-dns-and-dhcp-server-for-your-home-lab.md) | Shows how and why to run your own DNS/DHCP for a home lab or advanced home setup. Explains the tools, benefits like faster lookups, and additional features like conditional forwarding. | Home lab | +| [Powering a Home Lab with Docker and Synology NAS](2025-04-25-powering-a-home-lab-with-docker-and-synology-nas.md) | This post introduces Docker as the engine behind your modular, self-hosted infrastructure. It explains how to use Docker on a Synology NAS with Portainer as a GUI to manage containers. It also introduces your key use-cases: dashboards, observability, developer tools, and monitoring. | Home lab | +| [Observability and Monitoring in a Home Network](2025-04-25-observability-and-monitoring-in-a-home-network.md) | A practical guide on how to monitor your home network, device health, and internet uptime using Dockerised observability tools. Describes how each tool contributes to visibility: from container stats to uptime checks and speed tests. | Home lab | +| [Essential Dashboards for Home Labs: Heimdall, Homarr, Flame](2025-04-25-essential-dashboards-for-home-labs-heimdall-homarr-flame.md) | Showcases dashboard and launchpad tools that provide clean, user-friendly frontends for managing services. Explains how they integrate with your containers, how to configure them, and their strengths. | Home lab | +| [Development and Documentation Tools in Containers](2025-04-25-development-and-documentation-tools-in-containers.md) | Covers self-hosted developer productivity tools running in Docker. From editing code remotely to sketching diagrams and hosting a knowledge wiki, these tools provide a powerful dev environment right on your NAS. | Home lab | +| [Network Tools and Automation in Docker](2025-04-25-network-tools-and-automation-in-docker.md) | Highlights how to automate your smart home, document architectures, and track IP/network changes using Dockerised tools. Great for tinkerers, home automation fans, and tech organisers. | Home lab | +| [Keeping Docker in Check with Watchtower](2025-04-25-keeping-docker-in-check-with-watchtower.md) | Shows how Watchtower automatically updates your containers, the pros and cons of automated updates, and when to pin versions. It’s your CI-lite in a home lab. | Home lab | diff --git a/site/docs/blog/posts/2025-04-25-observability-and-monitoring-in-a-home-network.md b/site/docs/blog/posts/2025-04-25-observability-and-monitoring-in-a-home-network.md new file mode 100644 index 0000000..eb2d9a0 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-observability-and-monitoring-in-a-home-network.md @@ -0,0 +1,171 @@ +--- +date: 2025-04-25T09:26:00 +authors: + - markbac +categories: + - Home lab +slug: observability-and-monitoring-in-a-home-network +--- + +# Observability and Monitoring in a Home Network + +*Part 26 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Have you ever found yourself cursing at the router when your Netflix buffers right at the climax of your favourite series? Or scratching your head wondering why that smart thermostat isn't quite as "smart" today? Welcome to the world of home network observability and monitoring—a vital practice for keeping your digital household running smoothly. + + + +Just like a seasoned football manager keeps an eye on his players’ fitness levels, as a home network steward, it's crucial to keep tabs on your devices, internet connectivity, and service uptime. With a few Dockerised tools, you can transform into a tech wizard with insights at your fingertips. + +## Monitoring Goals + +Before diving into the nuts and bolts of setting up monitoring, let’s clarify what we aim to achieve: + +- **Network health**: Ensure all connected devices communicate efficiently without any bottlenecks. +- **Device performance**: Monitor CPU usage, memory load, and overall device health. +- **Service uptime**: Track whether key services (like your media server) are up and running. +- **Internet connectivity**: Keep an eye on your internet speed and availability—after all, it’s the backbone of your home lab. + +## Observability Tools (Dockerised) + +To turn these goals into reality, let's explore some Dockerised tools that will be your allies in this mission: + +### cAdvisor + +- **Purpose**: Provides insights into container statistics. +- **Key Features**: + - Real-time monitoring of resource usage by containers. + - Visualisation of historical data for analysis. + +### Glances + +- **Purpose**: Offers a comprehensive view of system resources. +- **Key Features**: + - Monitors CPU load, memory usage, disk I/O, and more. + - Supports plugins to extend functionality. + +### Uptime Kuma + +- **Purpose**: Uptime monitoring for services. +- **Key Features**: + - Tracks uptime for websites and services. + - Sends alerts based on configured thresholds. + +### Speedtest-cli + +- **Purpose**: Measures internet speed. +- **Key Features**: + - Conducts regular speed tests to monitor ISP performance. + - Automates testing using cron jobs or similar scheduling tools. + +### Fing (Network scanning) + +- **Purpose**: Network scanning to identify connected devices. +- **Key Features**: + - Detects new or unknown devices on your network. + - Provides information about device type and manufacturer. + +## Setup and Configuration + +Now let’s set up a Docker Compose file for our monitoring stack. This will make spinning up our tools as easy as pie: + +```yaml +version: '3' +services: + cadvisor: + image: google/cadvisor + ports: + - "8080:8080" + volumes: + - /:/rootfs:ro + - /var/run:/var/run:rw + - /sys:/sys:ro + - /var/lib/docker/:/var/lib/docker:ro + + glances: + image: nicolargo/glances + ports: + - "61208:61208" + command: "glances -w" + + uptime-kuma: + image: louislam/uptime-kuma + ports: + - "3001:3001" + + speedtest: + image: henrywhitaker3/speedtest-tracker + ports: + - "8765:8765" +``` + +### Configuring Alerts and Notifications + +With great power comes great responsibility—or rather, with great monitoring comes the need for timely notifications: + +- Set up email alerts or push notifications for critical events (e.g., service downtime). +- Consider using alert management platforms like Alertmanager if you’re feeling adventurous. + +## Dashboards and Visualization + +Visualising data is like getting match stats after a football game—it helps you understand the play better. + +### Grafana (Optional) + +For those who fancy advanced dashboards: + +- Integrate Grafana with your monitoring stack to create custom dashboards. +- Pull data from cAdvisor and Glances to visualise trends over time. + +### Displaying Monitoring Data + +Consider using Heimdall or similar tools to create a visually appealing homepage displaying all critical metrics at a glance. + +## Analyzing Monitoring Data + +Once you've got data flowing in, it's time for analysis: + +- **Identifying Bottlenecks**: Look for patterns indicating resource strain (e.g., constant high CPU usage). + +- **Troubleshooting Issues**: Use historical data to diagnose recurring problems. If your internet drops every evening at six o’clock, maybe it’s time to check the kids' gaming habits! + +- **Capacity Planning**: Plan upgrades based on trends (e.g., adding more RAM if memory usage consistently peaks). + +## Best Practices + +To ensure effective monitoring without being overwhelmed: + +### Centralized Logging + +Store logs centrally using tools like ELK Stack for easy access and analysis. + +### Alert Fatigue Management + +Avoid being bombarded by alerts: + +- Set appropriate thresholds for alerts based on normal usage patterns. +- Group related alerts together to avoid notification spam. + +### Regular Monitoring Reviews + +Schedule periodic reviews of your monitoring setup: + +- Adjust thresholds as needed based on changing usage patterns. +- Evaluate new tools or updates that might offer better insights. + +## Conclusion + +By implementing observability and monitoring in your home network, you’ll not only prevent those dreaded buffer circles but also gain peace of mind knowing everything is shipshape. Just like a well-oiled machine needs regular maintenance, so does your home lab. So grab a brew, get those Docker containers up and running, and keep an eye on things—you’ll thank yourself later! + +--- + +### TL;DR Summary + +To maintain a reliable home lab: + +1. Set clear monitoring goals (network health, device performance, etc.). +2. Use Dockerised tools like cAdvisor, Glances, Uptime Kuma, Speedtest-cli, and Fing. +3. Set up with Docker Compose; configure alerts via email or push notifications. +4. Visualise data using Grafana or Heimdall for easier interpretation. +5. Regularly analyse data for bottlenecks or issues; plan capacity accordingly. +6. Follow best practices including centralized logging and managing alert fatigue effectively. diff --git a/site/docs/blog/posts/2025-04-25-powering-a-home-lab-with-docker-and-synology-nas.md b/site/docs/blog/posts/2025-04-25-powering-a-home-lab-with-docker-and-synology-nas.md new file mode 100644 index 0000000..6bb65ad --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-powering-a-home-lab-with-docker-and-synology-nas.md @@ -0,0 +1,202 @@ +--- +date: 2025-04-25T09:25:00 +authors: + - markbac +categories: + - Home lab +slug: powering-a-home-lab-with-docker-and-synology-nas +--- + +# Powering a Home Lab with Docker and Synology NAS + +*Part 25 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Alright, folks, let's talk about harnessing the power of Docker in your home lab. Imagine your home lab as a bustling city, and Docker is the efficient public transport system that keeps everything moving smoothly. It allows you to run applications as lightweight containers, making it easier to build, ship, and deploy across various platforms. + + + +Now, if Docker is our transport system, then the Synology NAS (like my trusty DS723+) is the central hub or station. It's the platform where all your data and services converge. With a Synology NAS, you're not just storing files; you're opening up a world of possibilities for running applications and services. + +So, grab a brew and let's dive into how you can use Docker on your Synology NAS, with Portainer as your GUI conductor, to manage everything from dashboards to developer tools. + +## Docker Basics + +Before we get ahead of ourselves, let's cover some basics. + +### Containers vs. VMs + +- **Containers**: Think of them as lightweight, portable operating environments. They package up software and its dependencies so it runs reliably across different computing environments. +- **Virtual Machines (VMs)**: These are like full-blown computers running inside your computer. They simulate hardware and are generally heavier on resources. + +In simple terms, if VMs are houses with their own land (OS), containers are flats in a block sharing the same foundation (kernel). + +### Docker Images and Repositories + +- **Docker Images**: The blueprint for your containers. Once you've got an image, you can spin up multiple containers from it. +- **Docker Repositories**: Where images live. Docker Hub is a popular public repository where you can find thousands of images. + +## Docker on Synology NAS + +Now, onto the star of our show—the Synology NAS. + +### Installing Docker + +1. Head over to the Package Centre on your Synology DSM. +2. Search for "Docker" and hit install. + +That’s it—Docker will be ready to roll! + +### Configuring Docker Settings + +Once installed: + +- Go into Docker > Settings. +- Configure network settings like bridge or host mode depending on your needs. + +A bit like setting up lanes on a motorway—choose how traffic flows between containers and outside world. + +## Portainer + +Managing Docker from the command line is powerful but sometimes you just want a nice dashboard. Enter Portainer. + +### Why Use Portainer? + +- **GUI Management**: Simplifies container management with an intuitive web interface. +- **Features**: View logs, manage images and networks, monitor resource usage—all without touching the CLI. + +### Portainer Setup and Features + +1. Pull the Portainer image from Docker Hub: + ```bash + docker pull portainer/portainer-ce + ``` +2. Run Portainer in a container: + ```bash + docker run -d -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce + ``` + +Once it's running: +- Access Portainer via `http://:9000`. +- Follow setup wizard to link it with your Docker environment. + +![Portainer Dashboard](https://via.placeholder.com/150) + +*Placeholder for an image showing Portainer's dashboard.* + +## Home Lab Use Cases + +Now that we're set up, let’s look at some practical uses for Docker on your Synology NAS. + +### Dashboards + +Tools like Heimdall, Homarr, or Flame act as front doors to your services: + +```yaml +services: + heimdall: + image: linuxserver/heimdall + ports: + - 80:80 + volumes: + - /path/to/config:/config +``` + +These dashboards give you quick access to all your apps—a bit like having bookmarks on steroids. + +### Observability and Monitoring + +Keep an eye on what’s happening under the hood with: + +- **cAdvisor**: Monitor container resource usage. +- **Glances**: A cross-platform monitoring tool. +- **Uptime Kuma**: Check if services are up and running. + +### Development Tools + +Spin up dev environments quickly with something like `code-server`: + +```yaml +services: + code-server: + image: codercom/code-server + ports: + - 8443:8080 + volumes: + - /path/to/projects:/home/coder/project +``` + +This turns any device into a powerful coding space—perfect for tinkering without messing up local setups. + +### Network Tools + +Automate workflows or monitor network traffic with: + +- **Node-RED**: Great for wiring together APIs and devices. + +## Docker Compose + +When you're dealing with multiple interconnected containers, Docker Compose comes to rescue. + +### Defining Multi-container Applications + +Create a `docker-compose.yml` file: + +```yaml +version: '3' +services: + app1: + image: app/image1 + app2: + image: app/image2 +``` + +Run everything defined in the file with one command: + +```bash +docker-compose up -d +``` + +It’s like handing over blueprints to builders—they know exactly what goes where. + +## Best Practices + +A few tips to keep things running smoother than a well-oiled machine: + +### Managing Docker Data + +- Use named volumes for persistent data. + +### Security Considerations + +- Keep containers isolated unless they need to communicate. + +### Updates and Maintenance + +Automate updates using Watchtower: + +```yaml +watchtower: + image: containrrr/watchtower + volumes: + - /var/run/docker.sock:/var/run/docker.sock +``` + +This keeps containers updated without lifting a finger—like having an automated pit crew for your race car. + +## Conclusion + +Unleashing the potential of Docker in your home lab can transform how you manage services. With a Synology NAS as your platform and tools like Portainer guiding you through container management, you're set to create a resilient and flexible self-hosted environment. + +So whether you're spinning up dashboards or monitoring systems like a pro football manager keeping tabs on their team—Docker makes it all possible. Happy tinkering! + +## Takeaway Summary + +Here's what we've covered: + +- **Docker Basics**: Containers vs VMs; images & repositories. +- **Synology NAS**: Installation & configuration for Docker. +- **Portainer**: GUI management simplicity. +- **Use Cases**: Dashboards, observability tools, dev setups. +- **Best Practices**: Data management, security tips. + +Remember—Docker is powerful but only as good as how you manage it! diff --git a/site/docs/blog/posts/2025-04-25-qos-prioritising-traffic-for-smooth-performance.md b/site/docs/blog/posts/2025-04-25-qos-prioritising-traffic-for-smooth-performance.md new file mode 100644 index 0000000..78bce9d --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-qos-prioritising-traffic-for-smooth-performance.md @@ -0,0 +1,123 @@ +--- +date: 2025-04-25T09:18:00 +authors: + - markbac +categories: + - Networking +slug: qos-prioritising-traffic-for-smooth-performance +--- + +# QoS: Prioritising Traffic for Smooth Performance + +*Part 18 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Picture this: you're on an important video call, and suddenly, your screen freezes. Or maybe you're in the middle of a heated online game, and your character lags just as you're about to secure victory. Annoying, right? This is where Quality of Service (QoS) comes into play. It's the unsung hero working behind the scenes to ensure your critical network traffic gets the red-carpet treatment it deserves. + + + +## What is QoS? + +Quality of Service, or QoS, is a set of technologies used to manage network resources by prioritising certain types of data traffic. Think of it like a motorway with dedicated lanes for emergency vehicles — it ensures that vital services can always get through, even when things are busy. + +### Why is QoS Important? + +In today's world, many applications rely on real-time data transmission: + +- **VoIP (Voice over IP):** Needs consistent connectivity for clear conversations. +- **Video Conferencing:** Requires stable bandwidth to avoid awkward freeze frames. +- **Online Gaming:** Demands low latency to keep gameplay smooth. +- **Streaming Media:** Needs continuous data flow for uninterrupted viewing. + +Without QoS, these applications might suffer from delays and interruptions due to competing traffic demands. + +## QoS Mechanisms + +Implementing QoS involves several mechanisms that work together like a well-oiled machine: + +### Prioritisation + +This is about deciding which packets get sent first. Imagine being at a theme park where some folks have fast passes — they skip the queue while others wait. Similarly, QoS prioritises data packets based on their importance. + +### Bandwidth Allocation + +Here we divide the available bandwidth among different services. It's like splitting a pie fairly among your mates — everyone gets what they need without anyone going hungry. + +### Traffic Shaping + +Traffic shaping smooths out the data flow across the network. It’s akin to pacing yourself during a long-distance run — you don’t sprint at the start only to run out of steam halfway through. + +### Queuing + +Packets are placed in queues based on priority levels. High-priority packets get served first, similar to how emergency patients are treated before less critical cases in A&E. + +## DSCP (Differentiated Services Code Point) + +To effectively prioritise traffic, you need a way to mark it. Enter DSCP — think of it as putting labels on parcels to indicate which ones are fragile and need careful handling. + +### Marking Traffic for Priority + +DSCP tags each packet with a value indicating its priority level. Network devices then use these values to decide how to handle each packet. + +### DSCP Values + +Different DSCP values correspond to various levels of service: + +- **EF (Expedited Forwarding):** Top priority for critical traffic like VoIP. +- **AF (Assured Forwarding):** For important but less time-sensitive data. +- **BE (Best Effort):** Standard service for regular traffic. + +```mermaid +graph TD; + A[Data Packet] -->|Label with DSCP| B[Network Device]; + B -->|Prioritise Based on DSCP| C[Traffic Handling]; +``` + +## QoS Applications + +Let's look at some real-world scenarios where QoS makes a difference: + +### VoIP (Voice over IP) + +QoS ensures that voice packets are prioritised so that your calls remain clear and free from jitter or delay. + +### Video Conferencing + +By allocating sufficient bandwidth and prioritising video streams, QoS helps prevent those embarrassing "Sorry, I lost you there" moments. + +### Online Gaming + +Low latency is crucial here; with QoS, you minimise lag and maximise your chances of victory. + +### Streaming Media + +QoS ensures that streaming services deliver content smoothly without buffering interruptions. + +## QoS Configuration + +Getting QoS set up correctly is crucial. Here's how you can configure it: + +### Router Settings + +Most modern routers come with built-in QoS settings. Look for options to enable QoS and set priorities for different types of traffic. + +### Network Devices + +Ensure all network devices support and respect DSCP values. This might involve updating firmware or configuring settings manually. + +## QoS Challenges + +While QoS offers significant benefits, it's not without its challenges: + +### Complexity + +Configuring QoS can be complex and requires a good understanding of networking principles. It's not just about flipping a switch — it’s more like tuning an orchestra for harmony. + +### Over-Provisioning vs. QoS + +Some might argue that simply increasing bandwidth (over-provisioning) could solve issues without implementing QoS. However, this isn’t always cost-effective or practical and doesn't address prioritisation needs during peak times. + +## Conclusion + +Implementing Quality of Service in your network can significantly enhance performance for critical applications by ensuring they receive the necessary bandwidth and priority. While setting up QoS might require some initial effort and understanding, the long-term benefits make it worthwhile — smoother video calls, seamless gaming experiences, and uninterrupted streaming sessions await those who embrace it. + +> **Takeaway:** If you rely heavily on real-time applications, consider implementing QoS in your network setup. It could be the difference between frustration and flawless performance. diff --git a/site/docs/blog/posts/2025-04-25-running-a-local-dns-and-dhcp-server-for-your-home-lab.md b/site/docs/blog/posts/2025-04-25-running-a-local-dns-and-dhcp-server-for-your-home-lab.md new file mode 100644 index 0000000..315464b --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-running-a-local-dns-and-dhcp-server-for-your-home-lab.md @@ -0,0 +1,156 @@ +--- +date: 2025-04-25T09:24:00 +authors: + - markbac +categories: + - Home lab +slug: running-a-local-dns-and-dhcp-server-for-your-home-lab +--- + +# Running a Local DNS and DHCP Server for Your Home Lab + +*Part 24 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + + +## Introduction + +If you've ever dreamt of having your own little digital kingdom at home, complete with faster internet lookups and customised network configurations, then setting up a local DNS and DHCP server might just be the answer. It's like having your own football pitch where you decide who plays, when they play, and what the rules are. In this post, we'll explore how to harness the power of DNS and DHCP for your home lab or advanced home setup, giving you control, speed, and a sprinkle of geeky magic. + + + +### Benefits of Running a Local DNS and DHCP Server + +- **Control**: You're the gaffer here. Decide which devices connect and how they're identified. +- **Speed**: Local lookups can be much quicker than relying on external servers. +- **Customisation**: Tailor your network settings to fit your unique needs. + +So, let's dive in and take a closer look at how to set this up. + +## DNS and DHCP Basics + +Before we start fiddling with cables and software installs, let's recap the basics. + +### Recap of DNS and DHCP Functions + +- **DNS (Domain Name System)**: Translates human-friendly domain names (like `www.example.com`) into IP addresses that computers understand. +- **DHCP (Dynamic Host Configuration Protocol)**: Automatically assigns IP addresses to devices on your network, ensuring they can communicate with each other without you playing referee every time one connects. + +### Standard Network Configuration + +Imagine your network as a well-organised post-office system: +- **DNS** is like the address book used by postmen to deliver letters. +- **DHCP** is akin to assigning pigeonholes for new employees in an office. + +## Tools for Local DNS and DHCP + +When it comes to picking your tools, you've got some excellent options that won't break the bank. Two popular choices are Dnsmasq and Pi-hole. + +### Dnsmasq + +Dnsmasq is lightweight yet powerful. It's like the Swiss Army knife of network services: +- Provides both DNS caching and DHCP services +- Perfect for small networks +- Easy to configure with minimal overhead + +### Pi-hole (with DNS/DHCP Features) + +Pi-hole is originally designed as an ad-blocker but comes with built-in DNS/DHCP capabilities: +- Blocks unwanted ads across all devices +- Offers an easy-to-use web interface +- Can replace your router's own DNS/DHCP services + +## Setup and Configuration + +Let's get our hands dirty now. We'll walk through setting up either Dnsmasq or Pi-hole. + +### Installing and Configuring Dnsmasq + +Here’s how you can set up Dnsmasq: + +1. **Install Dnsmasq**: + ```bash + sudo apt-get install dnsmasq + ``` + +2. **Configure `/etc/dnsmasq.conf`**: + - Define DHCP range: + ``` + dhcp-range=192.168.1.50,192.168.1.150,12h + ``` + +3. **Set Up Local DNS Records**: + - Add entries like: + ``` + address=/myserver.local/192.168.1.10 + ``` + +4. **Restart Dnsmasq**: + ```bash + sudo systemctl restart dnsmasq + ``` + +## Setting Up Pi-hole + +For those using Pi-hole: + +1. **Installation**: Follow the [official guide](https://pi-hole.net). +2. **Configure DHCP**: Enable DHCP from the Pi-hole admin interface. +3. **Local DNS Records**: Add custom entries via the admin panel. + +## Advanced Features + +Once you're up and running, there's more magic to explore. + +### Conditional Forwarding + +Want specific queries to go elsewhere? Use conditional forwarding to send requests to different upstream servers based on domain names. + +### DNS Caching + +Speed up repeated lookups by caching previous responses locally, reducing latency significantly. + +### Local Domain Management + +Manage local domains without needing an internet connection—a boon if you're testing IoT setups or developing web apps in-house. + +## Home Lab Examples + +You might wonder how these tools fit into real-world scenarios—let's tackle that next. + +### Integrating with Docker Containers + +Route container traffic through your local DNS for consistent naming conventions across services. + +### Using with a Synology NAS + +Enhance your NAS capabilities by integrating it with your local DNS setup for better file sharing and application hosting. + +### Pi-hole on a Raspberry Pi (Given Your Setup!) + +A Raspberry Pi running Pi-hole is like having a mini server that keeps everything tidy while sipping tea quietly in the corner. + +```mermaid +graph TD; + A[User Device] -->|DNS Request| B[Pi-hole]; + B -->|Query| C[Internet DNS Servers]; + C -->|Response| B; + B -->|Filtered Response| A; +``` + +## Troubleshooting + +Even the best-laid plans can hit a snag or two—here are common issues: + +- **No Internet? Double-check** if another device is acting as a DHCP server. +- **Slow Lookups? Ensure caching** is enabled and properly configured. +- **Devices Not Receiving IPs? Revisit** your DHCP range settings. + +## Conclusion + +By setting up a local DNS and DHCP server, you're essentially empowering yourself with greater control over your network environment—faster lookups, customised configurations, and enhanced privacy are just some of the benefits you'll enjoy. + +--- + +## Takeaway Summary + +Running a local DNS and DHCP server isn't just for techies tinkering in their lofts; it's about making technology work smarter for you at home. Whether you're blocking ads across devices or managing multiple gadgets seamlessly, this setup brings order to the chaos of modern connectivity—just like refereeing a Sunday league match where everyone knows their place on the field. diff --git a/site/docs/blog/posts/2025-04-25-switches-and-routers-whats-the-difference-and-why-it-matters.md b/site/docs/blog/posts/2025-04-25-switches-and-routers-whats-the-difference-and-why-it-matters.md new file mode 100644 index 0000000..dd707bd --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-switches-and-routers-whats-the-difference-and-why-it-matters.md @@ -0,0 +1,141 @@ +--- +date: 2025-04-25T09:03:00 +authors: + - markbac +categories: + - Networking +slug: switches-and-routers-whats-the-difference-and-why-it-matters +--- + +# Switches and Routers: What’s the Difference and Why It Matters + +*Part 3 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Welcome to the world of networking—a bit like plumbing, but for data. Whether you're setting up a home network or managing a corporate behemoth, understanding the roles of switches and routers is crucial. Much like how a football team needs both defenders and forwards to function correctly, your network depends on these trusty devices to keep data flowing smoothly. + + + +## Introduction + +In this post, we'll explore: + +- **Switches:** The backbone of local area networks (LANs), handling data traffic within your network. +- **Routers:** The gateway between different networks, routing data from one network to another. +- **Key Differences:** Why it's essential to know which device does what. +- **NAT (Network Address Translation):** A little trick routers play to keep everything running smoothly. + +Let's dive into each component and see why they matter in your setup. + +## Switches + +A switch is your local postman—delivering data packets precisely where they need to go within a network. They're all about efficiency and speed. + +### Definition and Function (Layer 2) + +- **Layer 2 Device:** Operates on the data link layer according to the OSI model. This means it deals with MAC addresses, not IP addresses. +- **Purpose:** To connect devices within the same network, enabling them to communicate effectively. + +### MAC Addresses and Forwarding + +- **MAC Address:** A unique identifier assigned to network interfaces for communications on the physical network segment. +- **Forwarding:** Switches use MAC addresses to determine where to send data packets. They learn which devices are connected to which ports and build a MAC address table. + +### LAN Operations + +Imagine a village where everyone knows everyone else's house by sight—that's your LAN. Switches ensure that when one villager wants to chat with another, the message goes directly there without bothering the whole street. + +### Managed vs. Unmanaged Switches + +- **Managed Switches:** Offer control over network traffic with features like VLANs, Quality of Service (QoS), and monitoring tools. +- **Unmanaged Switches:** Simpler devices that require no configuration—plug and play. Perfect for straightforward setups like home networks. + +### Mermaid Diagram: Simple Network with Switch +```mermaid +graph LR + A[Computer 1] -- MAC Address --> S[Switch] + B[Computer 2] -- MAC Address --> S + C[Printer] -- MAC Address --> S +``` + +## Routers + +While switches are busy handling traffic within your local village, routers are like border control—ensuring that messages heading out or coming in are routed correctly between different networks. + +### Definition and Function (Layer 3) + +- **Layer 3 Device:** Operates on the network layer using IP addresses. +- **Purpose:** Connects multiple networks together, directing data from source to destination across network boundaries. + +### IP Addresses and Routing + +- **IP Address:** A unique identifier for a device on a TCP/IP network. +- **Routing:** Routers determine the best path for data packets based on IP addresses, akin to finding the quickest route through city streets. + +### Connecting Different Networks (LANs, WANs) + +Routers make sure that when you want to send data from your home office in Manchester to a server in London, it takes the most efficient route possible over various interconnected networks (the internet). + +### Home Router Functions + +Your home router is a multi-talented beast: + +- Acts as a router by connecting your home LAN to your Internet Service Provider's (ISP) network. +- Often includes built-in switches for connecting multiple wired devices. +- Provides Wi-Fi access for wireless devices. + +## Key Differences + +Understanding these differences helps when choosing equipment: + +### Layer 2 vs. Layer 3 + +- **Switches (Layer 2):** Handle local traffic within a single network using MAC addresses. +- **Routers (Layer 3):** Route traffic between different networks using IP addresses. + +### MAC vs. IP Addressing + +- **Switches:** Use MAC addresses to forward packets internally. +- **Routers:** Use IP addresses for directing packets externally. + +### LAN vs. WAN Focus + +- **Switches:** Keep it local—focused entirely on LAN operations. +- **Routers:** Bridge the gap between LANs and WANs, managing internet connectivity. + +## NAT (Network Address Translation) + +NAT is like having one postal address for an entire apartment block but ensuring each flat gets its letters. + +### Why NAT is Needed + +With more devices than available public IP addresses, NAT allows multiple devices on a private network to share a single public IP address for accessing external networks. + +### Basic Explanation + +1. Devices within your LAN use private IPs. +2. When accessing external networks (like the internet), NAT translates these private IPs into one public IP address. +3. Replies from external servers are then translated back by NAT into the corresponding private IPs inside your LAN. + +## Conclusion + +When setting up or managing networks: + +- **Use Switches** for internal communications within a LAN. They're essential for expanding your wired connections without fuss. + +- **Deploy Routers** when connecting different networks or needing internet access. They manage traffic efficiently, ensuring data reaches its correct destination across various paths. + +Both devices play vital roles in any well-oiled network setup, whether it's at home or in an enterprise environment. Understanding their functions ensures you can tailor your network infrastructure effectively—keeping everything running as smoothly as a well-executed football strategy. + +--- + +### Takeaway Summary + +In essence: + +- **Switches**: Local traffic controllers using MAC addresses; perfect for internal communication. + +- **Routers**: Network connectors using IP addresses; essential for external communications and internet access. + +And remember, NAT keeps everything tidy by sharing one public face for many private ones—a real team player! + +That's all you need from me today—now go forth and build those networks! diff --git a/site/docs/blog/posts/2025-04-25-tcp-vs-udp-choosing-the-right-transport-protocol.md b/site/docs/blog/posts/2025-04-25-tcp-vs-udp-choosing-the-right-transport-protocol.md new file mode 100644 index 0000000..66d1077 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-tcp-vs-udp-choosing-the-right-transport-protocol.md @@ -0,0 +1,129 @@ +--- +date: 2025-04-25T09:09:00 +authors: + - markbac +categories: + - Networking +slug: tcp-vs-udp-choosing-the-right-transport-protocol +--- + +# TCP vs UDP: Choosing the Right Transport Protocol + +*Part 9 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +In the world of networking, choosing the right transport protocol is a bit like picking the ideal vehicle for your journey. Need to transport delicate goods across town? Perhaps a well-sprung van would do. Racing against time? A nimble motorbike might be your best bet. Similarly, when it comes to data transmission over networks, TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are our two trusty steeds. + + + +Understanding these protocols is essential for anyone dealing with networked applications—whether you’re streaming yesterday’s match highlights or transferring important documents. Let’s dive into each protocol, explore their strengths and weaknesses, and see how they stack up against each other. + +## TCP (Transmission Control Protocol) + +### Connection-Oriented + +TCP is akin to having a dedicated phone line where both parties ensure they're listening before chatting away—think of it as making sure you’ve got someone’s attention before spilling the beans. + +- **Handshake**: Establishes a connection using a three-way handshake (SYN, SYN-ACK, ACK). +- **Dedicated Channel**: Guarantees that both sender and receiver are ready to communicate. + +```mermaid +sequenceDiagram + participant Client + participant Server + Client->>Server: SYN + Server-->>Client: SYN-ACK + Client->>Server: ACK +``` + +### Reliable Data Transfer + +TCP prides itself on reliability. It’s a bit like hiring a courier who confirms receipt with every delivery. + +- **Acknowledgments**: Every packet sent is acknowledged. +- **Retransmissions**: Lost packets are resent until acknowledged. +- **Error Checking**: Ensures that data arrives intact. + +### Sequencing and Flow Control + +TCP doesn’t just dump data at the receiver's door; it ensures everything arrives in order and at a manageable pace. + +- **Sequencing**: Packets are numbered so they can be reordered if they arrive out of sequence. +- **Flow Control**: Adjusts data flow based on network congestion and receiver capacity. + +### Overhead and Acknowledgments + +All this reliability comes at a cost—overhead. TCP's checks and balances mean more data overhead, making it less speedy than its counterpart. + +### Use Cases + +Ideal for scenarios where accuracy matters more than speed: + +- Web browsing +- Email +- File transfers + +## UDP (User Datagram Protocol) + +### Connectionless + +UDP shrugs off the formalities—no handshakes here. It sends packets as if tossing letters into postboxes without checking if anyone’s home. + +### Unreliable Data Transfer + +UDP offers no guarantees. If a packet goes missing, tough luck—it won’t come knocking again. + +### Low Overhead and Speed + +UDP travels light and fast, perfect for when timing trumps precision: + +- **No Acknowledgments**: Eliminates need for confirmations. +- **Minimal Error Checking**: Less overhead means faster delivery. + +### Use Cases + +When speed is king and minor losses are tolerable: + +- Streaming media +- Online gaming +- Voice over IP (VoIP) + +## Comparison + +Choosing between TCP and UDP boils down to balancing reliability against speed. Here’s how they compare: + +| Feature | TCP | UDP | +|--------------------|---------------------------------------------|------------------------------------| +| Connection | Connection-oriented | Connectionless | +| Reliability | High | Low | +| Overhead | High | Low | +| Order | Guarantees order | No order guarantees | +| Use Cases | Web browsing, email, file transfer | Streaming, gaming, VoIP | + +### When to Use Each Protocol + +Here’s a quick guide: + +- **Choose TCP** if you need guaranteed delivery and data integrity. Think about sending invoice files or accessing websites. + +- **Opt for UDP** when speed is critical and occasional losses are acceptable. Streaming your favourite show or battling in an online game are prime examples. + +## Conclusion + +Selecting the right transport protocol isn’t just about understanding technical differences; it’s about aligning those differences with your application’s needs. Much like deciding whether to take the scenic route or motorway on a road trip, it’s all about priorities—reliability or speed? + +Remember: + +- Use TCP for accuracy, order, and reliability. +- Choose UDP for speed and efficiency where minor losses don’t matter. + +With these insights under your belt, picking the right protocol should feel less like rocket science and more like choosing between tea or coffee—it all depends on what suits your needs best! + +### Takeaway Summary + +In essence: + +- TCP ensures reliable communication with more overhead. +- UDP prioritises speed with less assurance. +- Match your choice to the application's requirements—precision or performance? + +Don’t forget to consider network conditions and end-user experience when deploying either protocol in real-world scenarios. Happy networking! diff --git a/site/docs/blog/posts/2025-04-25-using-vlans-and-subnets-for-smart-home-security.md b/site/docs/blog/posts/2025-04-25-using-vlans-and-subnets-for-smart-home-security.md new file mode 100644 index 0000000..4efbab1 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-using-vlans-and-subnets-for-smart-home-security.md @@ -0,0 +1,167 @@ +--- +date: 2025-04-25T09:23:00 +authors: + - markbac +categories: + - Home lab +slug: using-vlans-and-subnets-for-smart-home-security +--- + +# Using VLANs and Subnets for Smart Home Security + +*Part 23 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Alright, folks, let's have a natter about your smart home network security. Picture your smart home as a bustling football match. You've got players (devices) all over the pitch (network), and without proper strategy (security measures), it's utter chaos. Imagine the goalkeeper (your data) unprotected – not ideal, right? Today, we'll explore how VLANs and subnets can be your network's best defence, keeping your IoT gadgets in line and your data safe. + + + +## Security Challenges in Smart Home Networks + +Smart homes are fantastic, aren't they? You can control everything from lights to the kettle with a tap on your phone. However, these conveniences come with their own set of challenges: + +- **Numerous Devices**: From smart fridges to security cameras, you might have dozens of devices connected. +- **Vulnerabilities**: Each device could be a potential entry point for hackers. +- **Data Privacy**: Sensitive information could be exposed if not properly secured. + +## Importance of Segmentation + +Just like a good pie has distinct layers, so should your network. Segmenting your network through VLANs and subnets isolates devices, minimising risk. Imagine it as having separate sheds for different tools rather than a cluttered garage – much easier to manage. + +## VLANs for Smart Home Security + +Virtual LANs (VLANs) allow you to partition your network logically rather than physically. It's like having invisible walls between rooms in your house. + +### Isolating IoT Devices + +- **Why Isolate?**: Many IoT devices have limited security features, making them easy targets. +- **How to Isolate**: Use VLANs to separate these devices from more secure ones like personal computers or NAS units. + +### Creating Guest Networks + +Ever had guests over who need Wi-Fi access? Instead of offering them the keys to your digital kingdom, create a guest network: + +- **Guest VLAN**: Set up a separate VLAN for guests to keep their traffic away from your main network. +- **Access Control**: Limit what guests can access; they shouldn't be able to snoop around your IoT devices. + +### Segmenting Cameras and Other Sensitive Devices + +Cameras and other sensitive devices should have their own VLAN. This is akin to putting them in Fort Knox compared to the general house: + +- **Dedicated VLAN**: Keep these high-risk devices isolated. +- **Limited Access**: Only allow necessary communication paths. + +### Example - VLAN Configuration on an Asus Router for a Smart Home + +Here's how you could configure VLANs on an Asus router: + +1. **Access Your Router's UI**: Typically via a web browser pointing at its IP address. +2. **Navigate to LAN Settings**: Look for VLAN configuration options. +3. **Create New VLANs**: + - **VLAN 10**: IoT Devices + - **VLAN 20**: Cameras + - **VLAN 30**: Guest Network +4. **Assign Ports/SSIDs**: Map specific ports or wireless SSIDs to each VLAN. + +```mermaid +graph TD; + A[Internet] -->|Traffic| B[Router]; + B --> C(VLAN 10: IoT); + B --> D(VLAN 20: Cameras); + B --> E(VLAN 30: Guest); +``` + +## Subnetting for Smart Home Security + +Subnets allow even finer control within VLANs by dividing them into smaller networks. + +### Further Segmentation within VLANs + +- **Micro-segmentation**: Even within a single VLAN, you can use subnets to create further divisions. +- **Example Setup**: + - Subnet 192.168.10.0/24 for IoT. + - Subnet 192.168.20.0/24 for cameras. + +### IP Address Management + +Managing IP addresses becomes more efficient with subnets: + +- **Static vs Dynamic IPs**: Assign static IP addresses to critical devices for consistent access. +- **DHCP Configuration**: Ensure the DHCP server assigns addresses within the correct subnet range. + +### Firewall Rules and Subnet-Based Policies + +With subnets in place, you can craft precise firewall rules: + +- **Allow Specific Traffic**: Permit only essential communication between subnets. +- **Block Unnecessary Communication**: Prevent inter-subnet chatter that could lead to vulnerabilities. + +## Firewall Rules and Policies + +A firewall is like a bouncer at your local pub – deciding who gets in and who doesn't. + +### Allowing Necessary Traffic + +- Only enable communication that's absolutely essential between different VLANs/subnets. + +```plaintext +Allow TCP 192.168.10.0/24 -> 192.168.20.0/24 port 80 +``` + +### Blocking Unnecessary Communication + +- Block anything that looks dodgy or isn't needed. + +```plaintext +Deny all ICMP from VLAN 30 +``` + +### Monitoring and Logging + +Keep an eye on things: + +- Regularly review logs. +- Look out for unusual traffic patterns – like spotting an unexpected player on the field. + +## Practical Configuration Examples + +Let's get our hands dirty with some configurations: + +### VLAN Setup on a Managed Switch + +1. **Access Switch Management Console** +2. **Create New VLANs** +3. **Assign Ports to Relevant VLANs** + +### Subnet Configuration + +1. **Configure DHCP Server** +2. **Define Subnet Ranges** + +### Firewall Rules on a Home Router + +1. **Access Router Firewall Settings** +2. **Create Rules Based on Subnet/VLAN Needs** + +## Security Rationale + +Why go through all this effort? Let's explore: + +### Defence in Depth + +Like building layers of stone walls around a castle – each layer offers additional protection. + +### Reducing the Attack Surface + +Fewer entry points mean fewer opportunities for attackers – much like securing individual barn doors rather than just the main gate. + +### Containing Breaches + +If one device gets compromised, segmentation ensures it doesn't spread across your entire network. + +## Conclusion + +In summary, using VLANs and subnets enhances smart home security significantly by isolating devices and controlling traffic flow meticulously. It’s akin to managing different livestock fields separately; each has its space and purpose, reducing chaos and enhancing safety. + +By implementing these strategies, you're not just putting up barriers but creating an organised system that's robust against threats – much like organising players effectively in football. It may seem complex initially, but once set up, it’s mostly maintenance work with occasional tweaks as new devices join or requirements change. + +So there you have it – grab yourself another cuppa and start planning that secure smart home network today! Cheers! diff --git a/site/docs/blog/posts/2025-04-25-virtual-lans-vlans-network-segmentation-101.md b/site/docs/blog/posts/2025-04-25-virtual-lans-vlans-network-segmentation-101.md new file mode 100644 index 0000000..5007db4 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-virtual-lans-vlans-network-segmentation-101.md @@ -0,0 +1,131 @@ +--- +date: 2025-04-25T09:15:00 +authors: + - markbac +categories: + - Networking +slug: virtual-lans-vlans-network-segmentation-101 +--- + +# Virtual LANs (VLANs): Network Segmentation 101 + +*Part 15 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Ever found yourself buried under a pile of network cables, wondering if there's an easier way to manage your growing home network or small business setup? Well, you're in luck. Virtual LANs (VLANs) are like the Swiss Army knife for network segmentation, allowing you to slice and dice your network logically without splurging on extra hardware. This post will walk you through the basics of VLANs, why they matter, and how you can deploy them effectively in both home and business environments. + + + +## What Are VLANs? + +### Why Segment Networks? + +Think of a network as a football pitch. Without any lines, it's chaos—players running everywhere with no structure. VLANs are like drawing those white chalk lines, creating logical boundaries so each team knows its space and role. + +- **Organisation**: By segmenting networks, you can organise devices into logical groups. +- **Security**: Reduce the risk of sensitive data being intercepted by isolating different types of traffic. +- **Efficiency**: Limit broadcast traffic to specific segments, improving overall network performance. + +## VLAN Basics + +### Logical Separation of Networks + +VLANs allow you to create separate networks on the same physical switch. Instead of buying multiple switches to segment traffic physically, VLANs do it logically. + +- **VLAN IDs and Tagging**: Each VLAN is assigned an ID (a number) that is used to tag packets. This tagging ensures that data travels only within its designated virtual lane. +- **Broadcast Domains**: A VLAN confines broadcast traffic to its own domain, preventing unnecessary data from bogging down other parts of the network. + +```mermaid +graph TD; + A[Device 1 - VLAN 10] -->|Tagged| B(Switch) + C[Device 2 - VLAN 20] -->|Tagged| B + D[Device 3 - VLAN 10] -->|Tagged| B + A --> D +``` + +## VLAN Types + +### Port-based VLANs + +This method assigns specific switch ports to a VLAN. Simple but less flexible if devices move around frequently. + +- **Static association**: Ports are manually assigned. +- **Usage**: Best in environments where devices rarely change locations. + +### Tag-based VLANs + +Also known as IEEE 802.1Q, this is more dynamic. Tags are added to Ethernet frames to identify which VLAN they belong to. + +- **Dynamic association**: Easier to manage when devices need to move between ports. +- **Common use case**: Ideal for larger environments where flexibility is key. + +## VLAN Configuration + +### Managed Switches + +To set up VLANs, you'll need managed switches capable of handling tagging. + +- **Configuration Access**: Often done via a web interface or command line. +- **Flexibility**: Allows for complex setups and fine-tuning of network policies. + +### Trunk Ports + +Trunk ports carry traffic for multiple VLANs over a single physical link. Imagine them as motorways for your network packets. + +- **Tagging Required**: Tags ensure packets reach their correct destination. +- **Use Case**: Essential for connecting multiple switches or integrating with routers. + +### Access Ports + +Access ports connect end devices to the switch and belong to a single VLAN. Think of these as side streets leading off the motorway (trunk port). + +- **Untagged Traffic**: Only carry untagged traffic specific to their assigned VLAN. +- **Best Practice**: Use for endpoint connections like computers or printers. + +## Home and Small Business Use Cases + +VLANs aren't just for big corporate data centres. They're handy in homes and small offices too! + +### Guest Networks + +Create a separate guest network so your visitors can't access your main devices. It's like giving guests their own room without keys to the rest of your house. + +### IoT Device Isolation + +Keep your smart fridge from chatting up your work computer by isolating IoT devices on their own VLAN. + +### Network Segmentation for Security + +Separate critical business systems from general employee access networks, akin to having a locked filing cabinet for sensitive documents. + +### Example - Using VLANs on an Asus Router + +Some home routers offer basic VLAN support: + +1. Log into the router's admin panel. +2. Navigate to the "LAN" settings. +3. Configure different ports with unique VLAN IDs. +4. Ensure trunking is enabled if connecting additional switches. + +## Benefits of VLANs + +Why go through all this effort? Here’s why: + +### Security + +By isolating sensitive parts of your network, you minimise potential attack vectors—like keeping valuable livestock fenced away from roaming predators. + +### Performance + +Limiting broadcast domains means less traffic congestion, akin to reducing rush-hour jams by adding more lanes on a road. + +### Management + +Logical segmentation simplifies administration tasks—from troubleshooting issues to applying policies—just like managing smaller fields rather than one massive plot of land. + +## Conclusion + +In summary, VLANs offer a powerful tool for organising and optimising networks both at home and in small businesses. They provide enhanced security, improved performance, and simpler management without requiring additional physical infrastructure. So next time you're faced with tangled cables and sluggish connections, consider deploying VLANs—it might just be the solution you're looking for! + +--- + +> **Takeaway**: Think of VLANs as logical fences within your digital landscape—providing structure, security, and efficiency without needing more hardware cluttering up your space. Whether it's enhancing security or boosting performance, they're an indispensable part of modern networking strategy. diff --git a/site/docs/blog/posts/2025-04-25-vpns-secure-tunnels-for-remote-access-and-privacy.md b/site/docs/blog/posts/2025-04-25-vpns-secure-tunnels-for-remote-access-and-privacy.md new file mode 100644 index 0000000..70d3915 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-vpns-secure-tunnels-for-remote-access-and-privacy.md @@ -0,0 +1,127 @@ +--- +date: 2025-04-25T09:16:00 +authors: + - markbac +categories: + - Networking +slug: vpns-secure-tunnels-for-remote-access-and-privacy +--- + +# VPNs: Secure Tunnels for Remote Access and Privacy + +*Part 16 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Ah, VPNs—those mysterious things that techies rave about and adverts promise will make you invisible online. But what are they really? Let's dive into the world of Virtual Private Networks (VPNs) and see how they work, why they matter, and how you can use them to boost your security and privacy. + + + +## Introduction - What is a VPN? + +A Virtual Private Network (VPN) is like a secret tunnel through the chaotic world of the internet. Imagine driving through a city in an armoured van with tinted windows. No one can see inside, but you can travel from point A to B safely. A VPN creates a secure and encrypted connection between your device and a remote server. This protects your data from prying eyes and allows you to surf the web more privately. + +## Why Use a VPN? + +### Remote Access + +- **Working From Home**: In the age of remote work, accessing company networks securely is crucial. A VPN lets employees connect to office resources as if they're physically there. +- **Travel**: Whether you're at a café or on a train, using public Wi-Fi can be risky. A VPN encrypts your data, keeping it safe from snoopers. + +### Privacy + +- **Online Anonymity**: Your internet service provider (ISP) or government may track your online activity. A VPN hides your IP address and encrypts your traffic. +- **Bypass Geographical Restrictions**: Want to watch content not available in your region? A VPN allows you to appear as if you're browsing from another location. + +## VPN Types + +### Remote Access VPNs + +These are for individuals who need to connect to a private network from a remote location. It's like having a key to enter the office network from wherever you are. + +### Site-to-Site VPNs + +Used by businesses with multiple branches. It's like connecting different offices with a dedicated, secure line over the internet. + +## VPN Technologies + +### VPN Protocols + +Understanding protocols can be as dry as watching paint dry, but bear with me—they're important. + +- **IPsec (Internet Protocol Security)**: Often used in site-to-site VPNs, it provides high security at the network layer. +- **OpenVPN**: An open-source protocol that is highly configurable for both remote access and site-to-site connections. +- **WireGuard**: The new kid on the block—it's faster and simpler than others but still offers robust security. + +```mermaid +graph LR + A[Client] -->|Encrypted Tunnel| B[VPN Server] + B --> C[Internet] +``` + +### Encryption and Tunneling + +Encryption is like turning your data into gibberish that only authorised parties can read. Tunneling encapsulates this data in a secure 'tunnel', protecting it during its journey across the internet. + +## VPN Use Cases + +### Remote Work and Access to Company Resources + +Imagine being able to access files and printers at work while sipping coffee at home. That's remote access via VPN—keeping productivity high without compromising security. + +### Protecting Privacy Online + +A bit like wearing a cloak of invisibility, a VPN shields your online identity from trackers, advertisers, and nosy governments. + +### Securing Home Networks and IoT Devices + +Your smart fridge might be handy, but it's also vulnerable. By securing your home network with a VPN, you add an extra layer of protection against cyber threats. + +## Setting up a VPN + +### VPN Servers and Clients + +Setting up involves configuring both ends—the server (endpoint receiving requests) and clients (devices making requests). + +- **VPN Server**: Can be hosted on-premise or in the cloud. +- **Clients**: Laptops, phones, tablets—all need client software for connection. + +### Router-based VPNs + +Some routers support built-in VPN capabilities. Configuring these can secure all devices on your home network with minimal fuss. + +### VPNs on NAS (e.g., Synology) + +Network Attached Storage (NAS) devices like Synology can act as personal cloud servers. Set up a VPN on these for secure file access anywhere. + +## VPN Considerations + +Like choosing between fish & chips or pie & peas, picking the right VPN requires some thought: + +### Performance + +- **Speed**: Encryption can slow down connections; choose protocols wisely. + +### Security + +- **Encryption Strength**: Look for AES-256 encryption for robust security. + +### Trust and Provider Selection + +- **No-log Policies**: Ensure providers don’t log user activities. + +> **Note:** Free services might sell your data; it's worth investing in reputable ones. + +## Conclusion - Benefits and Importance of VPNs in Modern Networking + +VPNs have become essential tools in our digital toolkit. Whether it's securing remote work connections or safeguarding personal privacy online, their importance can't be overstated. Just like you'd lock your front door at night, consider using a VPN to lock down your internet traffic securely. + +## TL;DR - Key Takeaways + +- **What is a VPN?** A secure tunnel for internet traffic ensuring privacy. +- **Why use one?** For secure remote access and enhanced online privacy. +- **Types:** Remote access and site-to-site. +- **Protocols:** IPsec, OpenVPN, WireGuard offer various features. +- **Use Cases:** Essential for remote work, privacy protection, IoT security. +- **Setup:** Requires server-client configuration; consider router or NAS options. +- **Considerations:** Balance performance with security; choose trusted providers. + +In summary, whether you're navigating corporate networks or just want peace of mind at home, integrating a VPN into your routine could be the best decision since sliced bread. diff --git a/site/docs/blog/posts/2025-04-25-what-is-a-network-lan-wan-pan-explained-simply.md b/site/docs/blog/posts/2025-04-25-what-is-a-network-lan-wan-pan-explained-simply.md new file mode 100644 index 0000000..634ef80 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-what-is-a-network-lan-wan-pan-explained-simply.md @@ -0,0 +1,134 @@ +--- +date: 2025-04-25T09:01:00 +authors: + - markbac +categories: + - Networking +slug: what-is-a-network-lan-wan-pan-explained-simply +--- + +# What Is a Network? (LAN, WAN, PAN Explained Simply) + +*Part 1 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +Pull up a chair, grab a brew, and let's have a chat about something that's basically the backbone of our modern world: networks. No, not the social media kind; I'm talking about the unseen web of connections that lets your computer talk to your printer, or your phone stream football results without missing a beat. + + + +## What Is a Network? (Layman's Terms) + +Think of a network like a good old-fashioned postal system. You've got different post offices (devices) sending letters (data) along various routes (connections) to reach their destination. Simple enough, right? + +## Why Are Networks Important in Daily Life? + +Networks are the unsung heroes in our daily lives. They: + +- **Connect us**: Whether you're video calling your nan or emailing your boss. +- **Entertain us**: Streaming the latest series or playing games online. +- **Enable work**: From accessing company servers to collaborating with colleagues. +- **Automate homes**: With smart devices and IoT gadgets making life easier. + +Now, let's break down the different types of networks you'll come across in both home and office environments. + +## LAN (Local Area Network) + +### Definition and Characteristics + +A **Local Area Network (LAN)** is like a small village — everything's close by. It's a network covering a limited area such as a home, school, or office building. + +- **Geographical scope**: Limited to a single location. +- **Speed**: Typically faster due to short distances. +- **Ownership**: Usually owned and managed by a single organisation or individual. + +### Home Network Examples + +In most homes, a LAN is set up using: + +- **Router**: Acts like the village post office, directing traffic between devices. +- **Devices**: Computers, smartphones, tablets, smart TVs all connected to the router. +- **Wired/Wireless connections**: Ethernet cables for stability or Wi-Fi for convenience. + +### Small Office LAN Scenarios + +Imagine an office where: + +- Printers are shared across multiple computers. +- Files are stored on a central server accessible by everyone. +- Employees connect their laptops both wirelessly and through docking stations. + +Here, everything operates within this local network bubble. + +## WAN (Wide Area Network) + +### Definition and Scale + +A **Wide Area Network (WAN)** is more like an entire country with cities and towns connected over long distances. It uses telecommunications links for its connections. + +- **Geographical scope**: Spans large distances, even globally. +- **Speed**: Generally slower due to distance but has high capacity. +- **Ownership**: May involve multiple organisations. + +### Internet as the Largest WAN + +The Internet itself is the ultimate WAN — it's basically every network in the world talking to each other. Your home LAN connects to it via your Internet Service Provider (ISP). + +### Connecting Remote Offices + +For businesses with multiple locations: + +- A WAN can link these remote offices into one cohesive network. +- Often involves leased lines or VPNs for secure communication. + +## PAN (Personal Area Network) + +### Definition and Short-range Focus + +A **Personal Area Network (PAN)** is like your personal bubble — just you and your immediate space. It covers very short distances — think centimetres to metres. + +- **Geographical scope**: Very limited range around an individual. +- **Speed**: Fast due to proximity but limited by technology used. +- **Ownership**: Personal and typically managed by the user. + +### Bluetooth Examples + +Bluetooth creates PANs with devices like: + +- Headphones paired with your smartphone. +- Wireless keyboards connected to your computer. + +### Wi-Fi Direct + +Wi-Fi Direct lets devices communicate directly without needing a router — handy for quick file transfers between phones or connecting printers without cables. + +## Comparison Table (LAN, WAN, PAN) + +Here's how these networks stack up against each other: + +```mermaid +graph TD; + A[LAN] -->|Local| B[Home/Office] + A -->|Fast| C[High Speed] + A -->|Single Owner| D[Private Control] + + E[WAN] -->|Global| F[Internet] + E -->|Varied Speed| G[Variable Capacity] + E -->|Multiple Owners| H[Shared Control] + + I[PAN] -->|Personal| J[Individual Devices] + I -->|Short Range| K[Direct Connections] + I -->|User Owned| L[Personal Control] +``` + +## Conclusion + +### Recap of Network Types + +To sum it all up: + +- **LANs** are your local networks at home or office — fast and private. +- **WANs** stretch across countries or continents — slower but vast. +- **PANs** are intimate setups around you — personal and direct. + +### Importance of Understanding Networks + +Understanding these basics helps demystify how your devices connect and interact. It's not just geeky trivia; it's about knowing the tech that keeps us all connected in our ever-digital world. So next time you hit "send" on an email or stream that new show, you'll know exactly what goes on behind the scenes. Cheers! diff --git a/site/docs/blog/posts/2025-04-25-wired-vs-wireless-ethernet-and-wi-fi-compared.md b/site/docs/blog/posts/2025-04-25-wired-vs-wireless-ethernet-and-wi-fi-compared.md new file mode 100644 index 0000000..82283c1 --- /dev/null +++ b/site/docs/blog/posts/2025-04-25-wired-vs-wireless-ethernet-and-wi-fi-compared.md @@ -0,0 +1,142 @@ +--- +date: 2025-04-25T09:04:00 +authors: + - markbac +categories: + - Networking +slug: wired-vs-wireless-ethernet-and-wi-fi-compared +--- + +# Wired vs Wireless: Ethernet and Wi-Fi Compared + +*Part 4 of the [networking and home lab series](2025-04-25-networking-and-home-lab-series.md).* + +## Introduction + +Ah, network connectivity—a bit like the plumbing in your house. You don't really think about it until something goes wrong, but when it does, you're knee-deep in water (or in this case, connection issues). Whether you’re binge-watching the latest series on Netflix or working from home, a solid network is crucial. In this post, we'll have a natter about two common networking options: **Ethernet** and **Wi-Fi**. + + + +We’ll explore the ins and outs of these technologies, looking at their pros and cons, suitable use-cases, and how to choose between them for your specific needs. Spoiler alert: there’s no one-size-fits-all answer, just like there’s no one brew that suits everyone. So let’s dive in. + +## Ethernet + +### Cabling Standards + +The backbone of wired networks is made up of cables—no surprises there. Let’s break down the common ones: + +- **Cat5e**: Supports gigabit speeds (up to 1 Gbps) over short distances. It's like your reliable old workhorse tractor; nothing fancy but gets the job done. +- **Cat6**: Handles up to 10 Gbps over 55 metres. It's faster than Cat5e, akin to upgrading from a regular drill to an impact driver. +- **Cat6a**: This one can stretch 10 Gbps speeds over the full 100 metres. Think of it as a sports car—fast and capable. +- **Cat7/Cat8**: More suited for data centres with very high speeds and shielding to reduce interference. These are your Formula 1 cars. + +### Speeds and Bandwidth + +Ethernet generally offers higher and more consistent speeds than Wi-Fi: + +- Steady speeds up to 1 Gbps or more +- Minimal latency—great for gaming or VoIP calls +- Less prone to interference from other devices + +### Reliability and Stability + +In terms of reliability, Ethernet is like that mate who always turns up on time—dependable: + +- Consistent performance unaffected by physical obstacles +- Ideal for environments where stability is key (like offices or server rooms) + +### Use Cases + +Ethernet shines in scenarios where reliability is non-negotiable: + +- **Desktops**: For those who need uninterrupted internet access. +- **Servers**: Data centres rely heavily on wired connections. +- **Gaming Consoles**: Keeps latency low for that competitive edge. + +## Wi-Fi + +### Wi-Fi Standards + +Wi-Fi has its own bunch of standards. It’s like football leagues; different levels for different needs: + +- **802.11a/b/g/n/ac/ax (Wi-Fi 6)**: The alphabets represent various versions with increasing speed and efficiency. +- Wi-Fi 6 (802.11ax) is the latest, offering better performance in crowded environments—like playing five-a-side on a crowded pitch. + +### Frequency Bands + +Wi-Fi operates over two primary frequency bands: + +- **2.4 GHz**: Longer range but lower speed, akin to a long-distance runner. +- **5 GHz**: Faster but shorter range—think of it as a sprinter. + +### Range and Obstacles + +Wi-Fi signals are like sound waves—they get blocked by walls and furniture: + +- Range varies based on obstacles; open spaces are ideal +- Walls, metal objects, and even fish tanks can cause signal degradation + +### Interference and Security + +Wi-Fi isn't immune to interference: + +- Competes with other wireless devices (microwaves, Bluetooth) +- Security is paramount; always use WPA3 if possible to avoid being the open goal of cyber threats. + +### Wi-Fi Mesh Networks + +A modern solution for large homes or complex layouts: + +```mermaid +graph TD; + A[Internet] -->|Router| B[Mesh Node 1]; + B --> C[Mesh Node 2]; + B --> D[Mesh Node 3]; +``` + +Mesh networks spread coverage evenly across an area, reducing dead zones. It’s like having multiple midfielders covering every inch of the pitch. + +## Comparison + +Here's how Ethernet and Wi-Fi stack up against each other: + +| Feature | Ethernet | Wi-Fi | +|---------------|-------------------------|---------------------------| +| Speed | Up to 10 Gbps | Up to 9.6 Gbps (Wi-Fi 6) | +| Reliability | High | Variable | +| Cost | Generally cheaper | Equipment can be pricey | +| Mobility | Limited (tethered) | Excellent | + +## Choosing the Right Technology + +Deciding between Ethernet and Wi-Fi depends largely on your environment and requirements: + +### Home Scenarios + +For a typical home setup: + +- **Use Ethernet** if you have devices that stay in one place—like desktop PCs or smart TVs. +- **Opt for Wi-Fi** if you need flexibility—tablets, smartphones, or laptops moving around the house. + +### Office Scenarios + +In an office setting: + +- A hybrid approach often works best: + - **Ethernet** for stationary equipment such as PCs and printers. + - **Wi-Fi** for mobile devices or areas where cabling isn’t feasible. + +## Conclusion + +The world of connectivity is ever-evolving. Much like how VAR has changed football strategies (for better or worse), new developments in networking technology continuously alter our approach to home and office setups. + +Ultimately, both Ethernet and Wi-Fi have their place in our digital lives; understanding your specific needs will help you make the best choice. Whether it’s the steadfast reliability of Ethernet or the flexible convenience of Wi-Fi—or indeed a combination of both—there’s a solution that fits every scenario. + +## Key Takeaways + +- Ethernet offers unmatched reliability and speed; it's perfect if stability is essential. +- Wi-Fi provides flexibility and ease of installation but can suffer from interference. +- Consider your specific use-case: stationary vs. mobile devices. +- A hybrid approach often yields the best results in mixed environments. + +In the end, it all comes down to what fits your needs best—and remember, just like tea preferences, it's all about personal taste! diff --git a/site/docs/blog/posts/2025-05-15-sds-docs-as-code.md b/site/docs/blog/posts/2025-05-15-sds-docs-as-code.md index c85b65b..b1d7af4 100644 --- a/site/docs/blog/posts/2025-05-15-sds-docs-as-code.md +++ b/site/docs/blog/posts/2025-05-15-sds-docs-as-code.md @@ -7,281 +7,1125 @@ categories: slug: sds-docs-as-code --- -# Writing a Software Design Specification (SDS) Using Docs-as-Code +# Writing a Software Design Specification Using Docs-as-Code -A Software Design Specification (SDS) should be the living, central document that expresses how your software system works — from architecture to scheduling, interfaces to timing, limitations to traceability. It’s the backbone of software quality, design reasoning, and cross-team understanding. +A Software Design Specification (SDS) is one of the most important documents produced during a software project. It serves as the definitive technical description of how a system is intended to work, why certain decisions were made, and how various components interact with each other. When maintained well, it forms the foundation for development, testing, onboarding, auditing, and future evolution of the system. -But too often, SDSs end up as dead PDFs written once before code begins and never updated again. That's a mistake. Whether you're working on a distributed platform, a real-time embedded system, or something in between — your SDS should grow with the system it describes. +However, in traditional workflows, SDS documents are often created as Word files or PDFs during early project phases, and then left to gather dust. They become obsolete, disconnected from reality, and difficult to maintain — leading to inconsistencies across design, code, and testing. + +A Docs-as-Code approach fixes this. Instead of static documents, we use markdown, version control (e.g. Git), CI automation, and modular structure to make the SDS a **living artefact**. It stays with the code, evolves with the system, and integrates with your engineering workflows. + +This guide walks through how to structure, write, maintain, and automate an SDS using Docs-as-Code principles, focusing on practical use in embedded, platform, and full-stack software systems. + +--- + +## Where the SDS Fits in the Engineering Lifecycle + +The SDS bridges the gap between architectural decision-making and concrete implementation. It complements: + +* **RFCs** (Requests for Comment) which explore possible changes and solicit feedback +* **ADRs** (Architecture Decision Records) which capture why decisions were made +* **Interface Specifications**, source code, test plans, and CI pipelines + +The SDS captures how these decisions are realised in a consistent, understandable, and traceable form. It provides clarity not only for developers, but for stakeholders, testers, maintainers, and even future hires. + +A good SDS doesn’t just list features — it tells the story of how and why the system behaves the way it does. + +> 🧠 A good SDS is a narrative, not just a checklist — it explains how the system is built and *why* it’s built that way. -This post explains how to create a modular, version-controlled SDS using a Docs-as-Code approach. It details the folder structure and what each section should contain. It also shows how it ties into the wider engineering workflow including ADRs, RFCs, and interface specs. --- -## Recommended SDS Structure +## Recommended Folder Structure + +Here’s a suggested structure for storing your SDS as part of a wider docs repository: ```plaintext docs/ -├── adr/ -├── rfc/ -├── interfaces/ -│ ├── lwm2m.md -├── diagrams/ -├── scripts/ -├── sds/ +├── adr/ # Architectural decision records +├── rfc/ # Requests for comment +├── sds/ # The software design specification │ ├── index.md │ ├── architecture.md +│ ├── security.md │ ├── task-model.md │ ├── power.md │ ├── timing.md -│ ├── memory.md +│ ├── memory/ +│ │ ├── memory.md # Summary of memory usage: flash, RAM, EEPROM +│ │ ├── nvm-layout.md # Detailed description of config/data NVM │ ├── updates.md -│ ├── interfaces/ -│ │ ├── i2c-protocol.md +│ ├── interfaces/ # Internal interfaces │ │ ├── uart-api.md +│ │ ├── i2c-protocol.md │ │ ├── nvm-layout.md -│ ├── components/ +│ ├── components/ # Software components/modules │ │ ├── rtc.md -│ │ ├── eeprom.md +│ │ ├── lcd.md +│ │ ├── watchdog.md +│ ├── system-modes.md +│ ├── manufacturing.md +│ ├── qa-support.md +│ ├── configuration.md +│ ├── design-patterns.md │ ├── instrumentation.md │ ├── limitations.md -│ ├── build-system.md +│ ├── build-system/ +│ │ ├── build-system.md +│ │ ├── ci-build.md +│ ├── traceability.md +│ ├── glossary.md │ └── metadata.yaml +├── interfaces/ # External/public APIs +├── diagrams/ +├── scripts/ ``` -Each file is atomic, reviewable, and versioned. Now let’s walk through what should go in each one. +Each file is modular, focused, and reviewed independently. This makes it easier to keep the specification updated and correct. --- -## 1. index.md – Introduction and System Overview +## Introduction and Overview (`index.md`) + +This section is the entry point into the SDS and should provide enough background and context to help any reader understand the system being described. It orients new team members, aligns stakeholders, and sets the tone for the level of depth and rigour expected throughout the document. -- Purpose and audience of the SDS -- Brief on the system context (e.g. mobile app, IoT device, backend service) -- Dependencies on hardware, cloud, network, or other services -- Relevant upstream artefacts: requirement docs, ADRs, RFCs -- Scope boundaries: what’s included, what isn’t +Start by introducing the system at a high level. Explain what the system is, what it does, who it's for, and what this document covers. Provide links to related documentation so readers can navigate easily between high-level decisions (ADRs), requirements, and lower-level technical references. + +Consider including: + +* **Purpose and Scope**: What is the system or component? What does the SDS cover (and not cover)? +* **Audience**: Who is this document for? Developers? QA? External auditors? +* **System Context**: Where does this software live — in a product, service, embedded device? +* **References**: Point to upstream ADRs, RFCs, or requirements. Link to downstream interface specs or test plans. +* **System Goals**: High-level objectives (e.g. real-time performance, security posture, battery life targets). + +This section should be written in plain English with clarity in mind. Avoid overwhelming detail — the goal is to provide a map and a purpose, not to dive into architecture just yet. --- -## 2. architecture.md – System or Firmware Architecture +## Architecture (`architecture.md`) + +The architecture section gives a clear view of how the software system is structured and how it behaves at a high level. It provides both a *static* and *dynamic* understanding of the system: + +* Static: the main building blocks, layers, and data flows +* Dynamic: how those components interact over time (e.g. task lifecycles, signal flows) -- High-level diagrams showing modules, services, interfaces -- Layers: app, drivers, middleware, platform APIs -- Patterns used (e.g. layered, hexagonal, CQRS) -- Component responsibilities -- Optional: MPU, container security, or privilege separation +This is often the first place a developer or system integrator will look when trying to understand how things fit together. + +You should include: + +* **Visual diagrams**: Use layered diagrams (e.g. C4, onion, service interaction) to show how software components are related. Sequence diagrams can help explain runtime behaviours. +* **Component layers**: Describe how your application, middleware, driver layers, and hardware interact. For example, where does business logic live? What abstracts hardware? +* **Runtime interactions**: Summarise typical workflows (e.g. "sensor event → processing → transmit via UART"). +* **Platform constraints**: Note any limitations imposed by the target (e.g. limited memory, no MMU, slow flash writes). +* **Security boundaries**: Indicate zones of privilege separation, sandboxing, or trust boundaries. ```mermaid flowchart TD - A[App Layer] --> B[Middleware] - B --> C[Drivers] - C --> D[Hardware or Host Interfaces] + App[Application Logic] --> Services[Service Layer] + Services --> Drivers[Drivers & HAL] + Drivers --> Peripherals[Hardware Interfaces] ``` +> 💡 Treat this section as your system's blueprint — not every detail, but enough for someone to understand the structure and find their way around. + --- -## 3. task-model.md – Threading, Tasks, and Scheduling +## Security and Security Considerations (`security.md`) -- Scheduler model: FreeRTOS, OS scheduler, custom loop -- Task list with purpose, priority, stack usage -- IPC strategy: queues, semaphores, message passing -- Event loops or co-routine flows -- Background and idle handling +This section documents the security posture of the software system — what threats are considered, how they are mitigated, and what design decisions have been made to protect the system and its users. -| Task Name | Priority | Trigger | Comments | -|--------------|----------|-------------|----------| -| Sync Task | High | Timer | Periodic updates | -| UI Task | Low | Button IRQ | User input handling | -| Logging Task | Low | Event Queue | Flush to UART/log buffer | +Security must be baked into the system from the start, not bolted on at the end. A good SDS explicitly documents the threat model, key assumptions, and concrete mechanisms used to mitigate risks. --- -## 4. timing.md – Timing Budgets and Real-Time Behaviour +### Security Objectives -- Real-time constraints -- Deadline enforcement strategies -- Task execution windows -- System tick configuration -- Jitter sensitivity -- Time maintenance mechanisms (e.g. RTC, NTP sync) +- Protect confidential data stored or transmitted by the device +- Ensure software integrity during updates and boot +- Prevent unauthorised access or misuse of interfaces +- Enable recovery from fault or attack +- Support traceability and audit of changes -```yaml -timing-budget: - ui-refresh: 16ms - http-retry-timeout: 3s - i2c-isr-max: 5µs -``` +--- + +### Threat Model + +Describe the assumptions about potential attackers, their capabilities, and vectors. Define what is in-scope vs out-of-scope. + +#### Example Considerations + +* **Physical Access**: Does the attacker have physical access to the device? +* **Network Exposure**: Is the device internet-connected or air-gapped? +* **Tampering Scenarios**: Battery removal, sensor spoofing, UART sniffing? +* **Supply Chain Risks**: Malicious firmware injection at factory? +* **Replay Attacks**: Are command messages signed or timestamped? --- -## 5. power.md – Power Strategy and Constraints +### Secure Boot and Firmware Validation -Even in non-battery-powered systems, power optimisation may matter (e.g. thermals, green compute, power caps). In embedded systems, it's often critical. +Explain how the system ensures that only authorised code is executed: -- Active, idle, sleep, standby modes -- Wakeup sources: timers, GPIO, network events -- Power state transitions and rules -- Peripheral shutdown behaviour -- Boot vs runtime power profiles +- **Bootloader Validation**: CRCs, digital signatures, hash comparison? +- **Key Storage**: Where are root-of-trust keys stored? (OTP, secure flash, external secure element) +- **Rollback Protection**: How are older, vulnerable images prevented from reloading? +- **Integrity Checks**: Are checks done at boot only, or periodically? + +--- + +### Data Protection + +Detail how sensitive data is stored or transmitted: + +- **At Rest**: Is config/NVM data encrypted? How is key material protected? +- **In Transit**: Are communications encrypted (TLS, DTLS, AES)? On which links? +- **Tamper Evident Storage**: Are audit logs or counters protected from erasure? + +> 🧠 Include how secrets are provisioned — at factory, on first boot, or over-the-air. + +--- + +### Interface Hardening + +Document how external and internal interfaces are protected: + +- **Authentication/Authorisation**: Who can call what APIs or send what commands? +- **Rate Limiting**: Are repeated attempts (e.g. password, firmware flashing) throttled? +- **Error Handling**: Are responses generic enough to avoid leaking internal state? +- **Interface Disablement**: Are debug ports disabled/locked after provisioning? + +--- + +### Secure Update Process + +Reiterate or expand on the update mechanism with a focus on security: + +- **Manifest Validation**: Signature verification before applying? +- **Fail-Safe Update**: Can update be rolled back or re-attempted without bricking? +- **Secure Channels**: Is firmware fetched over a trusted path? + +--- + +### Logging and Audit + +Capture how events are monitored or logged for later review: + +- **Security Logs**: Are invalid access attempts or tamper events recorded? +- **Forensics**: Can retained logs aid post-attack analysis? +- **Log Protection**: How are logs protected from tampering? + +--- + +### Mitigations and Controls + +Summarise protections in place against common embedded threats: + +| Threat | Mitigation | +|--------------------------|------------------------------------------| +| Firmware tampering | Secure boot, signed updates | +| UART console hijack | Passwords, lockout, port disable | +| Flash memory extraction | Encryption, obfuscation, secure storage | +| Replay attacks | Nonces, timestamps, rolling counters | +| Debug access | JTAG/SWD lockout after provisioning | + +--- + +### Security Testing + +Describe how the system’s security is validated: + +- Penetration testing (manual or automated) +- Fuzzing tools or static analysis +- Formal review of crypto or secure boot flows + +Mention any compliance standards targeted (e.g. IEC 62443, ENISA guidelines). + +--- + +### Maintenance & Response + +Explain how the system stays secure over time: + +- How are CVEs and third-party vulnerabilities tracked? +- How are keys rotated or revoked? +- What’s the process for patching live devices? + +--- + +### Documentation and Policy Links + +Reference supporting documents such as: + +- Key management policy +- Secure coding guidelines (e.g. MISRA, CERT) +- Incident response plan +- Vulnerability disclosure process + +--- + +> 🔐 Security must be a living concern — this section should be reviewed regularly and updated alongside changes in the threat landscape or architecture. + + +## System Modes (`system-modes.md`) + +Many software systems operate in distinct modes or states, each optimised for a specific set of conditions, use cases, or environmental triggers. Capturing these modes and the rules for transitioning between them is essential for understanding runtime behaviour — especially under error conditions, during startup, or in low-power scenarios. + +This section outlines the operational states of the system and describes how and why transitions occur. It should be used by developers, testers, and system architects to ensure consistent behaviour across all operating conditions. + +Consider including: + +* **Defined Modes**: Common examples include Boot, Active, Sleep, Maintenance, Recovery, Fault, and Shutdown. +* **Triggers and Transitions**: What causes a transition? Examples: GPIO wakeup, watchdog timeout, scheduled maintenance, low battery. +* **Mode-specific Behaviour**: What changes in each mode? Power usage, active tasks, network stack activity, user interface state. +* **State Diagrams**: These are especially helpful for visualising complex transitions. ```mermaid stateDiagram-v2 - [*] --> Init - Init --> Active - Active --> Idle : No activity - Idle --> Sleep : Timeout - Sleep --> Wake : Event (RTC, GPIO) + [*] --> Boot + Boot --> Active : Initialisation complete + Active --> Sleep : Idle timeout + Sleep --> Active : Interrupt (GPIO/RTC) + Active --> Maintenance : Config change + Active --> Fault : Error detected + Fault --> Recovery : Watchdog reset ``` +> 📘 Use this section to provide a robust mental model of how your system behaves over time. + --- + 🏭 Manufacturing Support (`manufacturing.md`) + +This section describes any software features, modes, or behaviours specifically included to support factory testing, provisioning, or calibration. + +## Why It Matters + +These features are often critical for volume deployment but easily forgotten or misunderstood later. If not documented, changes made for manufacturing (e.g. skipping init steps, ignoring CRCs, bypassing security checks) can leak into field code or be misunderstood in audits. + +## Suggested Contents + +### Manufacturing Modes -## 6. interfaces/ +- Does the firmware support a dedicated mode? (e.g. “factory mode”, “golden image boot”) +- How is it triggered? (pin strapping, serial command, fuse setting) -### Each file here defines +### Test Points / Interfaces -- Internal APIs (between modules or layers) -- Hardware bus protocols (I2C, SPI, UART, etc.) -- Service APIs (REST, gRPC, MQTT, WebSocket) -- Message formats and error models -- Configuration schemas (YAML, JSON, binary) +- Special I/O exposed for boundary scan, serial diagnostics, or sensor calibration? +- UART/I2C debug protocols used? -Example: `i2c-protocol.md` includes device address, command set, register layout, and CRC approach. +### Provisioning Workflows + +- How are serial numbers, keys, or config data written? +- Describe use of factory fixtures, scripts, or host tools. + +### Bypass Logic + +- Does manufacturing mode skip watchdogs, boot validation, or timers? +- Any backdoors enabled for engineering use (should be removed or gated in release builds)? + +### Logs / Confirmation Output + +- What signals a “pass” vs “fail”? +- Where are results logged — UART, EEPROM, test jig? + +> ⚠️ Be clear about how manufacturing features are locked out, hidden, or disabled in release builds (e.g. `#ifdef MANUFACTURE_MODE` + no release define) --- -## 7. memory.md – Memory Layout +## QA & Testing Support (`qa-support.md`) -Covers: +This section documents special firmware behaviours or interfaces that help with validation, automated testing, or quality gates. -- Flash/RAM/NVM use -- Emulated EEPROM usage -- Partition maps for multistage bootloaders or A/B firmware -- Memory guard strategies -- Persistent storage and config areas +### Why It’s Important -```plaintext -0x0000 Bootloader (16KB) -0x4000 App Primary (128KB) -0x24000 Config + Logs (64KB) +These are often the hooks that make factory validation or CI testing practical. Without this, firmware teams forget which debug pins or CLI commands were critical to QA and lose velocity on regression testing. + +### Suggested Contents + +#### Debug/Test CLI or Protocols + +- Are there commands to simulate inputs, dump state, or force transitions? +- Accessible via serial, SWD, CLI shell? + +#### Test Hooks / Mock Modes + +- Can you fake sensor inputs, bypass power checks, etc. for bench testing? +- Support for hardware abstraction that allows mocking? + +#### Assertions, Logs, and Diagnostics + +- Are test assertions enabled? +- Logs routed to test tools or available via postmortem dump? + +#### Test Patterns or Modes + +- Is there a loopback, soak, or burn-in test? +- Stress patterns (e.g. rapid wake/sleep) for endurance checks? + +#### API or Register Access + +- Can testers access internal registers, counters, or performance metrics? +- Memory-mapped or via an RPC/debug protocol? + +#### Integration With QA Tools + +- Jigs or scripts used to validate results? +- Compatibility with TestStand, LabView, or custom Python frameworks? + + +--- +## Design Considerations + +This section exists to provide insight into the broader context and trade-offs that shape your system’s design. These considerations often come from business goals, development constraints, or risk management practices — and they help justify why the system is built the way it is. + +Think of it as the foundation layer of your technical rationale. If someone asks "why didn’t you use X?" or "why is it done this way?", this section should help answer that. + +You might want to discuss: + +* **Assumptions**: What has been assumed about the hardware, users, network environment, third-party components? +* **External Dependencies**: Include reliance on SDKs, vendor HALs, cloud services, or middleware. +* **Coding Standards**: Which are followed? (e.g. MISRA C, CERT, AUTOSAR) +* **Security Concerns**: Discuss expected threat models and mitigations — encrypted channels, secure boot, flash protection. +* **Power and Performance Constraints**: e.g. CPU utilisation targets, sleep mode constraints, battery sizing. +* **Legal and Licensing**: GPL risk avoidance, open-source use policies, FIPS compliance. + +> 💡 This is the place to capture what influences your design but doesn’t always show up in code. + +--- + +## Design Patterns and Anti-Patterns (`design-patterns.md`) + +As your system evolves, recurring architectural or coding practices will emerge. Capturing these as standardised patterns (or anti-patterns) helps your team improve consistency, reduce onboarding time, and avoid costly mistakes. + +This section acts like a miniature design playbook. It highlights what has worked well and what should be avoided — especially in areas like tasking, interface boundaries, or message handling. + +### Recommended Patterns + +* **Publish/Subscribe Messaging**: Used to decouple components and allow for asynchronous event handling. Useful for telemetry, sensor updates, command dispatch. +* **State Machines**: Encapsulate mode transitions using modular state patterns, avoiding sprawling switch/case logic. +* **Factory or Registry Patterns**: To dynamically configure drivers or interfaces based on configuration at boot. +* **Command Pattern**: Separates logic from actions, helpful in communication protocols and UI abstraction. + +### Known Anti-Patterns + +* **God Objects**: One module knows and does too much — often seen in monolithic main loops. +* **Tight Coupling Across Layers**: HALs calling business logic directly, breaking abstraction layers. +* **Hidden Global State**: Making testing and debugging unpredictable. +* **Duplicated Logic Across Tasks**: E.g. similar parsing code in both comms and test tasks. + +Where possible, include: + +* Code examples of good vs. bad implementations +* Links to components that exemplify each pattern +* Notes about when a pattern *shouldn’t* be used + +> ⚠️ This section helps encode institutial knowledge, it shoudl evolve over time. + + +--- +## Task Model (`task-model.md`) + +In systems using real-time operating systems (RTOS), cooperative scheduling, or even basic super loops, understanding how software is structured into tasks or threads is crucial. This section provides an overview of all concurrent execution contexts, their responsibilities, timing expectations, and interactions with other tasks or interrupts. + +Well-documented task models help avoid timing conflicts, resource contention, and make it easier for new developers to reason about system flow. + +What to include: + +* **Task List**: Describe each task's responsibility, its period or trigger mechanism, and its priority. +* **Stack Size**: If stack usage is measured or estimated, note how much each task needs — especially for memory-constrained targets. +* **Event Flow**: Describe how ISRs signal or wake tasks (e.g. semaphores, message queues). +* **Diagrams**: Timeline or interaction charts can help visualise timing and sequence of events. + +| Task Name | Priority | Trigger | Stack | Description | +| ---------- | -------- | ----------- | ----- | ------------------------------- | +| SensorTask | High | Timer | 1kB | Polls sensors every 100ms | +| CommTask | Medium | Event queue | 2kB | Handles all communication flows | +| UITask | Low | Button IRQ | 1kB | Manages display and UI updates | + +> 💡 Documenting stack size and priority together makes it easier to identify task starvation or overflow risk during later reviews or audits. + +--- + +## Concurrency and Synchronisation + +Multithreaded or multitasking systems require careful coordination to avoid race conditions, deadlocks, or corruption of shared data. This section captures the strategy for how tasks and ISRs interact safely and predictably. + +Describe the mechanisms used for synchronisation and inter-task communication. For example, which tasks publish or consume events? What locks are used, and where? How is starvation or blocking handled? + +Include the following if applicable: + +* **Mutexes, Semaphores, Queues**: Which ones are used, and in what situations? +* **Priority Inversion Mitigation**: Does the RTOS support priority inheritance? Are there software strategies in place? +* **Deadlock Avoidance**: Document timeout strategies or watchdog recovery paths. +* **Non-blocking Strategies**: In some systems, blocking on shared resources is avoided altogether — capture this as a design choice. + +> ⚠️ This section is often overlooked but becomes critical when diagnosing hard-to-reproduce bugs or unexpected lockups in production. + +--- + +## ⏱ Timing (`timing.md`) + +Time constraints are common in embedded and real-time systems. Whether you’re reading sensors, driving a user interface, or meeting external communication deadlines, understanding your timing model is essential. + +This section documents which tasks or interrupts are time-sensitive and what their constraints are. + +Consider describing: + +* **ISR Deadlines**: How fast must interrupts complete? What work is deferred? +* **Task Budgeting**: Expected worst-case execution time for each major task. +* **System Tick and Timebase**: Describe the base timer, tick rate, and how time is kept. +* **Time Sync Mechanisms**: Use of RTC, GPS, NTP, or synchronisation with external clocks. + +```yaml +timing-budget: + control-loop: 10ms + display-refresh: 20ms + uart-isr: 2µs ``` +> ⏲ This information is especially useful when validating designs against real-time constraints or reviewing logs from performance instrumentation. + + --- +## Power (`power.md`) + +For embedded or battery-powered systems, power consumption is often one of the most critical design constraints. The goal of this section is to document how your system manages energy usage across different operating conditions and modes. + +This is particularly important for systems where uptime, efficiency, or thermal limits are affected by energy draw. It also helps verify that your design aligns with battery sizing, regulator capabilities, and customer expectations for device longevity. -## 8. updates.md – Startup and Upgrade Process +Suggested contents: -- Reset-to-application sequence -- Bootloader logic and verification -- Firmware image structure: header, CRC, footer -- Upgrade triggers (button, command, version mismatch) -- Rollback or dual-image management -- Upgrade state persistence +* **Supported Low-Power Modes**: Enumerate the power states supported by your MCU or platform (e.g. RUN, SLEEP, STOP, STANDBY). Include any specific characteristics (e.g. which peripherals remain active). +* **Wakeup Sources and Rules**: Describe how the system transitions out of low-power modes — for example, GPIO interrupt, timer, RTC alarm, watchdog. +* **Debounce and Filtering**: If wake sources involve mechanical switches or user input, include debounce timing strategies to avoid false wakeups. +* **Power Budget Estimates**: If available, provide power draw figures in each state (measured or estimated), with context for how often the system enters them. +* **Peripheral Power Control**: Describe how and when peripherals are powered down or gated. E.g. turning off UART transceivers, putting sensors into standby. + +> ⚡ Include a state diagram or table if your system uses multiple modes with complex transitions. --- -## 9. components/ +## Interfaces (`interfaces/`) + +This section documents how software interacts with hardware peripherals or internal drivers. For each interface, provide a dedicated markdown file that describes the protocol or API in full. This helps maintain a clear contract between the software layers and supports portability, testability, and modular design. + +You should break this section into one file per interface. For example: -Each module has: +* `uart-api.md`: Describes the UART driver API. Include init/config calls, transmit/receive flows, DMA usage (if any), and any line-discipline logic (e.g. software flow control, newline translation). +* `i2c-protocol.md`: Documents I2C commands to and from a specific device. Include register layout, timing requirements, expected acknowledgements, and examples. -- What it does and why it exists -- APIs provided -- Dependencies and initialisation order -- Error handling and retry logic -- Unit or integration test notes (if applicable) +Each interface file should typically include: -Example: `rtc.md` might include system timebase source, drift behaviour, backup register config. +* **Overview**: Purpose of the interface, what it controls or monitors +* **API Summary or Command Set**: Function prototypes or I2C/SPI commands +* **Parameters**: Buffer sizes, error codes, retry logic, blocking/non-blocking behaviour +* **Data Structures or Encodings**: Register maps, bitfields, enums +* **Diagrams**: State transitions or bus timing diagrams if relevant +* **Examples**: Sample usage code or communication trace + +> 📂 You may also include an `interfaces.md` index file if you have many interfaces. It can list and link to each subfile with a short summary, but it’s optional if your structure is already clear from folder organisation. --- -## 10. build-system.md – Build and Compilation +## Memory (`memory.md`) + +This section defines how your software uses memory, particularly when there are fixed constraints or persistent data to manage. In embedded systems, memory layout is often tightly coupled to the firmware image and must be coordinated with bootloaders, config storage, or debug tools. + +Document both volatile and non-volatile memory use: -- Toolchain, IDE, Docker image, OS details -- CI build stages -- Profiles: debug, release, test -- Conditional flags and feature toggles -- Linting, formatting, and artefact generation +* **Flash Layout**: Include memory map showing regions for bootloader, application firmware, configuration pages, user data, and diagnostic logs. If dual-image or failover schemes are used, detail the image switching mechanism. +* **RAM Usage Strategy**: Outline how RAM is allocated — statically, dynamically, or per-task. Mention stack sizes and heap if applicable. +* **Emulated EEPROM or NVM**: Describe how configuration data is stored and updated, including wear-leveling, sector rotation, and CRC use. +* **Persistent Variables**: Note any variables that retain values across resets, using backup registers, battery-backed RAM, or flash. -```c -#ifdef ENABLE_DIAG_UART - send_log(msg); -#endif +```plaintext +0x08000000 - 0x08003FFF : Bootloader (16 KB) +0x08004000 - 0x0801FFFF : App Firmware (96 KB) +0x08020000 - 0x08027FFF : Configuration Data (32 KB) +0x20000000 - 0x20002000 : RAM (8 KB) ``` --- -## 11. instrumentation.md – Debugging and Observability +### `nvm-layout.md` + +In addition to the general memory section, a separate file can be created to describe the layout and management of non-volatile memory (NVM). This is useful when configuration or runtime data is stored in flash and managed dynamically. + +Suggested contents: + +* **Storage Regions**: Define sectors or pages allocated to config, counters, logs, etc. +* **Versioning Scheme**: Explain how you track versions of stored data, particularly during firmware upgrades. +* **Integrity Checks**: CRCs, checksums, or signature strategies used to validate contents. +* **Update Strategies**: Whether you use append-only, copy-on-write, sector rotation, or journaling. +* **Error Handling**: What happens if a CRC fails or a config is corrupt? Describe the fallback or reset logic. +* **Tool Compatibility**: Link to any scripts or debug tools that can parse or inspect this memory structure. + +> 📎 Keep this file aligned with memory maps, linker scripts, and config handling logic. -- Log macros and levels -- Log sinks (UART, syslog, file) -- Runtime diagnostics and debug interfaces -- Special builds or test modes -- Tracing and assertions -- Telemetry (for production systems) --- -## 12. limitations.md – Technical Debt and Constraints +## Updates (`updates.md`) + +This section describes how the firmware or software running on the device can be updated. Depending on your system architecture, this could include Over-the-Air (OTA) updates, updates via USB, UART flashing, or during production line programming. It should clearly describe what gets updated, how integrity is validated, and what the fallback strategies are in case of partial or failed updates. + +Include the following details: + +* **Update Mechanism**: Describe supported mechanisms such as OTA (e.g. via LwM2M or MQTT), serial bootloader tools, or SD card updates. +* **Bootloader and Validation**: -- Deferred decisions -- Known issues or incomplete implementations -- External dependencies or blockers -- Banned functions (e.g. `strcpy`, `malloc`) -- Future work already planned + * Does the bootloader validate firmware via CRC, hash, or digital signature? + * What happens on validation failure? + * Include a brief flow: reset → bootloader → validation → app launch. +* **Dual-Image Strategy**: + + * If supported, describe how image switching works. + * Document active vs backup image location. + * Explain the conditions under which rollback happens. +* **Update Triggering**: + + * Who or what initiates an update? (e.g. remote server command, physical button, watchdog failover) + * How is update progress reported? +* **Metadata & Versioning**: + + * Is there a manifest file, checksum table, or version block? + * How is the running firmware version exposed to external systems? + +> 🧪 Include test cases or validation flows to ensure reliability. A diagram or sequence chart can also help visualise the update lifecycle. + +--- + +## Components (`components/`) + +This directory contains one markdown file per key software component or module. This helps keep the design documentation modular and scalable. Each file provides the rationale, API summary, interactions, and special considerations for that module. + +Each component file should include: + +* **Name and Purpose**: What is this component for? +* **Key Functions or Interfaces**: Public APIs exposed by this component. +* **Dependencies**: What peripherals or other modules does it rely on? +* **Startup/Shutdown Behaviour**: What must be initialised? What resets state? +* **Resource Usage**: Stack size, RAM footprint, exclusive hardware access. +* **Constraints and Limitations**: Known edge cases or design trade-offs. + +Example: `rtc.md` ```markdown -## Limitations +## RTC Module -- UART debug shares channel with modem; may delay logs. -- Uses legacy flash layout until migration to dual-bank. +The RTC module handles timekeeping for the system and provides alarms for timed events. + +### Public API +- `rtc_init()` — Initialise the clock domain +- `rtc_get_time()` — Read current RTC value +- `rtc_set_time()` — Set RTC value +- `rtc_set_alarm()` — Configure future wakeup event + +### Uses +- LSE oscillator (32.768kHz) +- Backup registers for retention across resets + +### Constraints +- Drift may occur during STOP mode without calibration +- Cannot use without LSE availability ``` +> 📦 Use consistent headers and formatting across all component docs for easier review. + +--- + +## Build System (`build-system.md`) + +This section documents how the software is built — what toolchains and configurations are supported, how outputs are structured, and what metadata should be captured to support traceability and CI integration. + +A well-documented build system ensures reproducibility, supports automation, and allows future engineers to rebuild or audit the firmware with confidence. + +*** + +### Supported Build Systems + +Describe how the software can be built across environments and by different roles — e.g. developer, CI agent, or release engineer. + +The SDS should explain which build systems are supported and how they interrelate. Typically, this includes: + +* `make` or similar build orchestration for local and CI use +* Generator-based build systems (e.g. `cmake`) if supporting multiple IDEs or cross-platform builds +* IDE-native builds (e.g. IAR project files) for deeply embedded targets +* Scripted or container-based builds for portable, isolated environments + +If multiple systems are supported, note which is canonical. + +*** + +### Toolchains + +Capture the expected compiler and toolchain setup used by developers and CI systems. + +Mention whether the toolchain is installed manually, managed by package manager, or included via container. The SDS should list supported toolchains (e.g. GCC, LLVM, IAR), any minimum version requirements, and a pointer to a `.toolchain` file or Docker image if used. + +*** + +### Build Configurations + +Define the available build variants, their intended use, and any functional differences. + +Typical configurations include: + +* **Debug** – enables diagnostics, logs, and debug symbols for development +* **Release** – optimised firmware, stripped of debugging features, suitable for deployment +* **Test** – includes mocking, test scaffolding, and additional hooks + +Explain how these configurations affect preprocessor defines, output structure, and included modules. This helps others understand which build type is suitable for specific tasks like flashing, validation, or lab testing. + +*** + +### Feature Flags and Defines + +List the key compile-time flags or feature switches used across the system. + +A well-maintained SDS should explain where these are declared (e.g. `build_flags.h`, `CMakeLists.txt`) and what they control — such as optional hardware interfaces, debug modes, or platform-specific adaptations. + +Flags should be consistently named and grouped logically to aid discoverability and maintainability. + +*** + +### Output Artefacts + +Detail what artefacts are produced by the build process and where they are placed. + +These commonly include: +* Executables (`.elf`, `.out`) +* Map files (`.map`), listings, and symbol tables +* Optional: memory reports, coverage files, metrics + +The SDS should state how these files are named and structured, especially if they're used in flashing, testing, or deployment workflows. + +*** + +### Build Metadata + +Describe the metadata embedded or exported by the build to support traceability and debugging. + +This may include: + +* Git commit, branch, and tag +* Build timestamp +* Build configuration and feature flags +* Toolchain ID or version string + +This metadata should be stored in a header file (`version.h`), embedded in flash, and optionally exported in a structured format (`build.json`) for integration with dashboards or CI pipelines. + +*** + +### Firmware Packaging + +Outline how formal release builds are bundled and versioned. + +The SDS should document how release artefacts are packaged for handoff — including: + +* Firmware binaries and supporting files +* Associated map and metadata files +* Optional checksums, signatures, or version manifests +* Directory naming and retention practices (e.g. `/dist/firmware-v1.2.3/`) + +This supports reliable delivery, testing, and rollback of firmware. + +### CI Integration (`ci-build.md`) + +This section documents the expectations for continuous integration (CI) pipelines that validate the design, enforce quality, and protect against regressions. + +A well-integrated CI process ensures every change is verifiable, traceable, and safe to merge. The SDS should make clear what is checked, how failures are detected, and which rules are enforced before code reaches production. + +*** + +#### CI Validates + +Summarise what CI pipelines are expected to do. + +A typical embedded software CI system should: + +* Build all configurations (Debug, Release, Test) +* Run static analysis on source code +* Execute unit or integration tests where applicable +* Enforce style and formatting rules +* Validate SDS structure and metadata +* Track binary sizes and alert on unexpected growth + +Mention if different CI stages are split across jobs or merged into a single runner. + +*** + +#### Code Quality Metrics + +Define which quality metrics are gathered to evaluate changes over time. + +A good SDS should specify that CI calculates and monitors: + +* Cyclomatic complexity +* Halstead metrics +* Function and file sizes +* Nesting/indentation depth +* Code churn and comment density + +The metrics system should highlight regressions and identify hotspots that may need refactoring. Detail thresholds if they exist. + +*** + +#### Binary Size Regression + +The SDS should define how binary size growth is detected and reported. + +CI must compare the size of key artefacts (e.g. `.elf`, `.bin`) to a known baseline and flag regressions. This protects against creeping feature growth, unexpected linker behaviour, or accidental inclusion of unused features. + +Summarise how size trends are tracked (e.g. via diff or dashboard), and when the build should fail. + +*** + +#### Incremental Checks + +Document how CI limits scope to affected areas to optimise feedback loops. + +The SDS should note that: + +* Static analysis and linting focus on changed files +* Code metrics and tests are limited to touched modules when possible +* Full system analysis is run nightly or before release + +This keeps feedback fast while still catching critical issues. + +*** + +#### Feedback & Merge Protection + +Explain how developers are informed of issues and how quality is enforced. + +CI pipelines should: + +* Post warnings or diffs directly into pull requests +* Annotate files or lines that fail checks +* Block merge if thresholds are exceeded, or key checks fail + +Summarise your merge rules — e.g. static analysis clean, no binary size regression, all tests pass, metadata validated. + +*** + +#### Optional Enhancements + +Capture any additional checks required for regulatory, audit, or public release workflows. + +The SDS may include optional CI jobs for: + +* Secret scanning (e.g. env or config leaks) +* Software Bill of Materials (SBOM) generation +* Open source licence validation +* Dependency vulnerability alerts +* Traceability metadata checks (e.g. ADRs linked to features or tests) + +Mention these if your process supports them — or leave as future improvements. + +> 📎 If detailed CI configuration (e.g. job scripts, badge configuration, security policies) is maintained separately, link to a `ci-extras.md` or `.github/workflows/` folder for deeper implementation detail. + + --- +## Instrumentation (`instrumentation.md`) + +Instrumentation allows developers and testers to observe, measure, and validate the behaviour of a running system. Whether it’s to diagnose issues, evaluate performance, or confirm that a state transition occurred as expected, a well-instrumented system offers transparency without requiring invasive access. + +This section should explain how observability is achieved, which tools or techniques are in use, and what runtime flags or modes exist to toggle diagnostic output. + +Recommended contents: + +* **Logging Strategy**: + + * What logging framework is used? (e.g. macros, circular buffers, conditional printfs) + * Describe log levels: `DEBUG`, `INFO`, `WARN`, `ERROR` + * Format: plain text, JSON, binary packets? + * Where does logging go? (UART, memory buffer, external tool?) + +* **Conditional Compilation**: + + * Are there flags like `ENABLE_LOGGING`, `DEBUG_MODE`, or `VERBOSE_UART`? + * What impact do they have on binary size or performance? + +* **Runtime Controls**: + + * Can instrumentation be enabled/disabled on the fly (e.g. via CLI or config register)? + * Describe diagnostic commands or debug shells (if any) + +* **Fault Capture**: + + * What happens on assert or fault? (e.g. stack dump, PC/SP snapshot, reset reason) + * Persistent logs in retained memory or NVM? + * Use of semihosting, debug probes, or test firmware overlays + +> 🧠 Consider documenting how to replay or interpret logs — especially if they use custom formats. + +--- + +## Limitations (`limitations.md`) + +This section is a place for honest reflection. It acknowledges areas where the system isn’t perfect — yet. Capturing technical debt, design shortcuts, or areas needing future refactor helps maintain team awareness and supports informed planning. + +You should aim to answer: “What’s broken, missing, or non-ideal — and what are we going to do about it?” + +Include the following where relevant: + +* **Unimplemented or Deferred Features**: + + * APIs that exist in stub form + * Tasks or behaviours described elsewhere but not yet complete + +* **Known Weaknesses or Risks**: + + * Use of dynamic memory in ISR context + * Shared global state with minimal locking + * Reliance on outdated or unstable third-party libraries -## 13. `metadata.yaml` – Traceability and Review +* **Legacy Components**: -Include: + * Code that is known to be hard to test, slow, or buggy + * Modules earmarked for future replacement + +* **Refactoring Plans or Proposals**: + + * Links to backlog items, tickets, or internal RFCs for cleanup + * Prioritisation of technical debt relative to business features + +> ⚠️ Be realistic — don’t list *everything* that could be better, just what is known and likely to impact the system. + +--- + +## Traceability (`traceability.md`) + +Traceability ensures your design isn’t floating in isolation — it ties every part of the system to the rationale behind it and the tests that verify it. This is especially valuable in safety-critical, audited, or regulated environments — but it also helps with long-term maintainability in any serious project. + +The goal is to allow someone to trace from: + +* A feature or bug → to the design decision → to the implementation → to the validation + +Document: + +* **Linked ADRs**: + + * Reference architecture decisions that led to this design (e.g. “Use MQTT over CoAP”) + +* **Linked RFCs**: + + * Proposals or exploration work that evaluated options (e.g. “rfc-006-iot-messaging.md”) + +* **Associated Tests**: + + * Unit, integration, or system tests that verify correctness + * Use consistent test names or IDs to simplify automation + +* **Metadata Tags**: + + * Use per-section frontmatter or YAML blocks to define traceability metadata + +Example: ```yaml -owner: "system-architecture" -component: "event-bus" -linked-adr: ["adr-015-use-pubsub"] -linked-rfc: ["rfc-008-async-handling"] +owner: "comm-stack" +linked-adr: ["adr-014-mqtt-integration"] +linked-rfc: ["rfc-006-mqtt-vs-coap"] +linked-tests: + - test_mqtt_reconnect + - test_power_on_retry last-reviewed: "2025-05-01" -linked-tests: ["test_pubsub_loss", "test_retry_logic"] ``` +> 🔍 Good traceability gives confidence during change reviews. It answers “why is this here?”, “who owns it?”, and “how do we know it works?” without needing tribal knowledge. + --- +## Exporting to PDF or Word for SharePoint + +Although the SDS should be maintained as markdown in version control, there are often cases where you need a formal, static version — for example: + +* Regulatory audits or compliance documentation +* Certification processes +* Customer deliverables +* SharePoint document repositories +* Project release cutoffs -## Docs-as-Code Tooling +This section explains how to generate exportable formats (PDF, Word, or HTML) from your markdown-based SDS using tools like **Pandoc** or **MkDocs**. These exports should be versioned and traceable to a specific tagged release or commit to ensure clarity and accountability. -- Markdown for content -- Mermaid or PlantUML for diagrams -- YAML for metadata -- CI for linting, PDF export, review expiry -- Static site generation for dev/stakeholder visibility +### Export via MkDocs + +For high-quality, structured exports that include styles, branding, and proper navigation, **MkDocs** (with Material theme and `with-pdf` plugin) is highly recommended. + +MkDocs also supports rendering **inline Mermaid diagrams**, something Pandoc cannot easily do. If your SDS includes architecture diagrams in Mermaid syntax, MkDocs ensures these are fully rendered in the PDF export. + +#### Example MkDocs Configuration + +Here’s a real-world example of an SDS-friendly `mkdocs.yml` setup: ```yaml -- name: Build SDS +site_name: SDS Documentation +site_author: Engineering Team +site_description: Version-controlled Software Design Spec +copyright: © 2025 Engineering Group + +docs_dir: docs/sds + +theme: + name: material + palette: + - scheme: default + toggle: + icon: material/brightness-7 + name: Switch to dark mode + primary: indigo + accent: pink + - scheme: slate + toggle: + icon: material/brightness-4 + name: Switch to light mode + primary: indigo + accent: pink + features: + - navigation.sections + - navigation.expand + - navigation.top + - toc.integrate + - search.highlight + +markdown_extensions: + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format + - toc: + permalink: true + - codehilite + - admonition + - plantuml_markdown: + server: https://www.plantuml.com/plantuml/svg/ + +extra_javascript: + - https://unpkg.com/mermaid@10.4.0/dist/mermaid.min.js + +plugins: + - gen_nav: + enabled: true + - search + - with-pdf: + cover: true + cover_title: "Software Design Specification" + cover_subtitle: "Auto-generated PDF version" + cover_logo: "assets/cover.png" +``` + +#### Export Commands + +To export the entire SDS as a styled PDF: + +```bash +pip install mkdocs mkdocs-material mkdocs-with-pdf +mkdocs build +mkdocs with-pdf export +``` + +### Alternative: Pandoc + +If you prefer Pandoc (e.g. for lightweight pipelines): + +```bash +pandoc sds/index.md -o SoftwareDesignSpec_v1.2.pdf --from markdown --pdf-engine=xelatex +pandoc sds/index.md -o SoftwareDesignSpec_v1.2.docx +``` + +> 📌 Note: Pandoc **cannot render inline Mermaid diagrams**. If you use these, MkDocs is the better choice. + +### CI Integration Example + +```yaml +- name: Install MkDocs dependencies run: | - markdownlint docs/sds/**/*.md - python scripts/check_metadata.py - pandoc docs/sds/index.md -o sds.pdf + pip install mkdocs mkdocs-material mkdocs-with-pdf + +- name: Export SDS as PDF using MkDocs + run: | + mkdocs build + mkdocs with-pdf export ``` +Alternatively, for Pandoc: + +```yaml +- name: Export SDS for release using Pandoc + if: github.ref == 'refs/tags/v1.2.0' + run: | + pandoc docs/sds/index.md -o release-artifacts/SDS_v1.2.0.pdf +``` + +> ✨ Whatever tooling you choose, take time to learn it well. Automate as much as you can, and keep it elegant — a polished export process makes your SDS as credible and useful as your code. +> 🚧 Recommendation: Choose **MkDocs** if you rely on diagrams and advanced layout, or **Pandoc** if simplicity and minimal dependencies are key. + +--- + +## CI Integration + +To make sure the SDS stays accurate and usable, treat it like code. That means validating its structure, checking metadata, linting markdown formatting, and optionally exporting versioned artefacts. + +Below is a sample CI pipeline that can be included in GitHub Actions, GitLab CI, or any other CI platform: + +```yaml +- name: Validate metadata and frontmatter + run: python scripts/validate_metadata.py + +- name: Check markdown syntax + run: markdownlint docs/sds/**/*.md + +- name: Export SDS to PDF + run: pandoc docs/sds/index.md -o docs/SoftwareDesignSpec.pdf +``` + +You might also add: + +* Spellcheck +* Broken link checker +* Diagram generation from `.mmd` or `.puml` files +* Automatic reviewers for docs-only PRs + +> ⚙️ The goal is to embed SDS health checks into the development lifecycle, ensuring the doc is always ready to ship. + +--- + +## Summary of Benefits + +| Docs-as-Code Practice | What it Helps With | +| -------------------------- | ------------------------------------ | +| Modular files | Focused ownership, easier updates | +| Traceable metadata | Clear links to decisions and tests | +| CI integration | Avoids rot, ensures freshness | +| Export to PDF/Word | Works with formal approval workflows | +| Patterns and anti-patterns | Onboarding, review, and consistency | + +A well-structured SDS, maintained using Docs-as-Code, bridges the gap between design and delivery. It gives teams confidence, improves collaboration, and becomes a trusted reference throughout the lifecycle of a system. + + + --- -## Recap and Benefits +## Glossary -| Practice | Benefit | -|---------------------|----------------------------------------------| -| Modular files | Clear ownership and easier reviews | -| Diagrams as code | Always up to date, easy to regenerate | -| Traceability metadata| Links decisions, tests, and components | -| Docs-as-Code pipeline| Consistency, freshness, automation | -| Shared mental model | Cross-team visibility and maintainability | +| Term | Description | +| ----- | --------------------------------- | +| SDS | Software Design Specification | +| ADR | Architecture Decision Record | +| RFC | Request for Comment | +| DaC | Docs-as-Code | +| CI/CD | Continuous Integration/Deployment | +| ISR | Interrupt Service Routine | +| HAL | Hardware Abstraction Layer | +| OTA | Over-the-Air updates | --- -An SDS built this way doesn’t just document what was implemented — it becomes an architectural memory. And whether you're writing for a cloud platform, embedded product, or something in between, this format ensures your design doc scales with your system — and your team. +## Final Thoughts -📘 This is the last part of the series, which starts with [Docs as Code: What It Is, Who It’s For, and When It Works Best](2025-05-12-docs-as-code.md). +A well-written SDS helps your team understand the system, justify decisions, and scale the design. When treated as code, it becomes part of your engineering DNA — continuously maintained, traceable, and aligned with the system itself. Build it once, evolve it forever. diff --git a/site/docs/blog/posts/2025-12-28-an-approach-to-engineering-leadership-and-delivery.md b/site/docs/blog/posts/2025-12-28-an-approach-to-engineering-leadership-and-delivery.md new file mode 100644 index 0000000..6fd70e7 --- /dev/null +++ b/site/docs/blog/posts/2025-12-28-an-approach-to-engineering-leadership-and-delivery.md @@ -0,0 +1,808 @@ +--- +date: 2025-12-28T09:00:00 +authors: + - markbac +categories: + - Team + - Engineering practice +slug: an-approach-to-engineering-leadership-and-delivery +--- + +# An Approach to Engineering Leadership and Delivery + +## An Integrative Philosophy for Modern Engineering Leadership + +Modern engineering leadership demands more than managing tasks or enforcing methodologies. It is about **deliberately designing, shaping, and evolving systems** in which engineers can think clearly, build effectively, and deliver outcomes that matter. Most persistent delivery, quality, and sustainability failures are not caused by a lack of effort or talent, but by the systems within which people operate. Leadership, therefore, is fundamentally a systems design discipline. + + + +This approach presents an **integrative philosophy**, synthesising insights from leadership thinking, systems engineering, and software architecture. It offers a **pragmatic, people-centred philosophy** intended to apply across diverse organisational contexts. It recognises that engineering outcomes emerge from the interaction between people, organisational structures, delivery mechanisms, and technology, rather than from any single practice or methodology. + +At its core, engineering leadership focuses on establishing context, enabling autonomy, fostering purposeful delivery, ensuring sustainable and resilient operations, and guiding teams towards long-term strategic outcomes. +**This document serves as a foundational set of principles, clearly defining what matters and why these elements are critical for effective engineering leadership. It deliberately leaves the tactical “how” to be addressed in subsequent frameworks, operating models, and delivery practices, recognising that implementation must vary based on organisational culture, maturity, regulatory constraints, and domain complexity.** + +Any framework, methodology, or operating model adopted beneath this philosophy must be constrained by these principles. Where a practice contradicts them, it should be considered misaligned regardless of convention, popularity, or historical precedent. + +--- + +## What We Mean by Systems + +Throughout this document, the term *systems* is used deliberately and precisely. Engineering leadership operates across multiple, interacting system types, none of which can be optimised in isolation without degrading overall outcomes. + +### Human Systems +Teams, motivation, trust, incentives, psychological safety, communication patterns, and decision-making dynamics. People behave rationally within the systems they are placed in. Persistent performance issues are almost always systemic rather than individual. + +### Organisational Systems +Operating models, governance structures, funding mechanisms, portfolio management, reporting lines, and performance management. Organisational design determines flow, accountability, and behaviour far more than individual intent. + +### Socio-Technical Systems +The interaction between people and technology, including toolchains, documentation practices, CI/CD pipelines, observability, and operational workflows. Technical systems always encode assumptions about communication, responsibility, and authority. + +### Product and Delivery Systems +How ideas move from concept to customer, including feedback loops, batch sizes, handoffs, validation mechanisms, and risk management. Delivery systems shape learning speed, quality, and predictability more than any single process choice. + +### Technical and Architectural Systems +Software, hardware, platforms, infrastructure, interfaces, and integration boundaries, including their coupling, failure modes, and operational characteristics. Architecture encodes assumptions about change, ownership, and risk. + +### Economic and Business Systems +Markets, cost structures, regulatory forces, incentives, pricing models, and commercial constraints. Engineering decisions are always economic decisions, whether explicitly acknowledged or not. + +Engineering leadership is the act of **aligning, governing, and evolving these systems as a coherent whole**, optimising for long-term systemic health rather than short-term local efficiency. + +--- + +## Core Principles of the Philosophy + +This philosophy is grounded in a small number of **non-negotiable principles**. These principles are not practices, processes, or roles. They exist to guide judgement, resolve trade-offs, and constrain downstream frameworks and operating models. + +They apply regardless of delivery methodology, organisational structure, or technology stack. + +### 1. Systems Shape Outcomes +Engineering outcomes are primarily determined by the systems within which people operate, not by individual capability or effort. Leadership is therefore responsible for designing and maintaining systems that enable clarity, flow, and sustainable performance. + +### 2. Context Enables Autonomy +Autonomy without context creates chaos; control without context creates disengagement. Leaders exist to establish purpose, boundaries, and constraints so that teams can make high-quality local decisions without constant escalation. + +### 3. People Are the Primary Constraint +Human cognitive capacity, motivation, and wellbeing are finite. Sustainable performance depends on psychological safety, manageable cognitive load, and a pace of work that can be maintained indefinitely. + +### 4. Architecture Is a Delivery Enabler +Architecture exists to enable change, learning, and flow. It should reduce cognitive load, make failure survivable, and preserve future options. Architectural decisions are economic and organisational decisions with long-term consequences. + +### 5. Flow Matters More Than Utilisation +The speed and quality of delivery are governed by flow through the system, not by keeping individuals or resources fully utilised. Slack is not waste; it is resilience, learning capacity, and optionality. + +### 6. Outcomes Trump Outputs +Success is measured by customer and business outcomes, not by activity, artefacts, or milestones. Shipping work that does not change outcomes is not progress. + +### 7. Transparency Enables Trust and Learning +Decisions, trade-offs, risks, and failures must be visible. Transparency reduces friction, improves alignment, and enables learning over time. Blame obscures truth; learning depends on honesty. + +### 8. Leadership Is Accountable for System Health +Leaders are accountable for the long-term health of the systems they oversee. Burnout, chronic urgency, recurring failures, and unmanaged technical debt are systemic signals and therefore leadership concerns. + +--- + +## The Role of Leadership + +The role of leadership is **not** to direct day-to-day execution, enforce compliance, or act as the primary decision-maker for work close to delivery. Leadership exists to **design, maintain, and evolve the system** within which engineering operates. + +In principle, leadership is responsible for: + +* **Establishing Context and Direction** + Defining purpose, priorities, constraints, and success criteria so teams can act with clarity and intent. + +* **Designing Decision Systems** + Determining where decisions are made, who is accountable, what information is required, and how risk and reversibility are managed. Reducing unnecessary decision latency is a core leadership responsibility. + +* **Creating Enabling Boundaries** + Setting non-negotiables, architectural guardrails, and governance boundaries that preserve coherence while enabling autonomy. + +* **Protecting System Health** + Ensuring sustainability, resilience, manageable cognitive load, and long-term technical health. Heroics and burnout are leadership failures, not cultural virtues. + +* **Resolving System-Level Trade-offs** + Intervening where tensions arise between speed and sustainability, autonomy and coherence, or local optimisation and global outcomes. These trade-offs cannot be delegated. + +* **Ensuring Learning and Adaptation** + Creating conditions for honest feedback, blameless learning, and continuous improvement. Repeated issues without systemic change indicate leadership failure. + +Leadership success is measured not by visibility or control, but by the **quality, durability, and adaptability of the system left behind**. + +--- + +## The Purpose of the Engineering Function + +The purpose of the engineering function is **to deliver value to customers and, in doing so, create sustained value for the business**. + +This is not achieved through activity, utilisation, or output volume, but through the creation and stewardship of systems that reliably turn intent into outcomes. + +In practice, the engineering function exists to: + +* **Deliver Customer Value Reliably** + Build products and services that solve real problems, perform predictably, and continue to do so as conditions change. + +* **Create Business Value Sustainably** + Enable growth, efficiency, differentiation, and compliance without accumulating unsustainable technical or operational risk. + +* **Translate Intent into Operable Reality** + Convert business and product intent into systems that work under real-world constraints, including scale, regulation, security, and failure. + +* **Enable Learning and Adaptation** + Shorten feedback loops so the organisation can learn from actual customer behaviour and operational reality rather than assumptions or plans. + +* **Preserve Optionality Over Time** + Design systems that can evolve without disproportionate cost, protecting the organisation from lock-in and fragility. + +* **Maintain Long-Term System Health** + Own operability, resilience, and technical debt as ongoing responsibilities, recognising that value cannot be delivered sustainably on unhealthy systems. + +Engineering does not maximise value by shipping more, faster, or louder. +It maximises value by **making value delivery safe, repeatable, and adaptable**. + +--- + +## Leadership and Engineering: A Deliberate Boundary + +Leadership and engineering are interdependent but not interchangeable. + +* Leadership owns the system. +* Engineering owns the work within it. + +When leadership interferes in execution, autonomy collapses. +When leadership abdicates system responsibility, chaos emerges. + +This boundary is foundational to the philosophy and underpins every principle, structure, and trade-off described in the sections that follow. +## Leading with Context, Not Control + +Effective engineering leadership does not attempt to control outcomes directly. Instead, it creates the conditions in which good outcomes emerge naturally from clear context, sound decision-making, and aligned incentives. Control-driven leadership increases dependency, decision latency, and fragility. Context-driven leadership increases ownership, adaptability, and system resilience. + +Leading with context means ensuring that engineers understand not just *what* they are doing, but *why* it matters, *what constraints apply*, and *how success is defined*. When context is clear, teams can make high-quality decisions locally without constant escalation. + +Key elements of context-driven leadership include: + +* **Clear Purpose and Intent** + Leaders articulate goals, priorities, and desired outcomes in a way that connects engineering work to customer value and business strategy. Purpose provides coherence across teams and disciplines, even as specific plans evolve. + +* **Explicit Constraints and Guardrails** + Boundaries should be stated deliberately rather than implied. Architectural principles, regulatory requirements, risk tolerances, and non-negotiables must be visible so teams can act with confidence and autonomy. + +* **Decision Authority Close to the Work** + Decisions should be made as close as possible to where the relevant knowledge exists. Escalation is appropriate for ambiguity, conflict between principles, or irreversible decisions, not for routine execution. + +* **Reduced Decision Latency** + Slow decisions are a hidden tax on delivery and morale. Leadership systems should minimise unnecessary approval layers and clarify who decides what, under which conditions. + +* **Consistency Over Certainty** + Teams benefit more from consistent decision logic than from perfect answers. Predictable reasoning builds trust and enables faster, safer action even in uncertain environments. + +Leadership that provides context rather than control creates organisations that are resilient, adaptable, and capable of sustained high performance without constant oversight. + +--- + +## Architectural Thinking: Delivery Through Design + +Good engineering leadership treats architecture as a **means to enable delivery**, not as an academic exercise or static artefact. Architecture shapes how quickly teams can respond to change, how safely systems can evolve, and how much cognitive load engineers must carry. + +This philosophy favours **architectural thinking over prescriptive design**. Rather than attempting to predict and design for all future needs upfront, leaders focus on creating structures that allow systems to evolve safely over time. + +Drawing from Richards, Ford, Bass, and Ousterhout, architectural leadership centres on the following principles: + +### Simplicity and Modularity + +Prefer simple, modular designs that evolve incrementally. Systems should be understandable by the teams that build and operate them. Simplicity reduces cognitive load, lowers defect rates, and shortens onboarding time. + +Complexity is sometimes unavoidable, but it should be introduced deliberately and paid for explicitly. + +### Lightweight Governance Through Guardrails + +Architecture requires governance, but governance should exist to enable progress rather than constrain it. Leaders establish **architectural guardrails** that define acceptable boundaries while allowing teams freedom within them. + +Governance answers questions such as: +- What must be consistent across the system? +- What decisions are reversible versus irreversible? +- Where is divergence acceptable, and where is it not? + +Governance is mandatory; ceremony is optional. The goal is coherence without bureaucracy. + +### Decision Transparency and Intent Preservation + +Architectural decisions are long-lived and context-dependent. Documenting decisions and their rationale is critical to preserving intent as teams, constraints, and technologies change. + +**Architecture Decision Records (ADRs)** and similar mechanisms capture: +- The problem being addressed +- Options considered +- Trade-offs and constraints +- The reasoning behind the chosen approach + +This transparency supports learning, accountability, and future evolution without forcing teams to rediscover past reasoning. + +### Technical Debt as a Strategic Concern + +Technical debt is not a moral failing; it is a strategic trade-off. However, unmanaged debt compounds over time, reducing delivery speed, reliability, and morale. + +Leadership must ensure that technical debt is: +- Visible +- Understood +- Actively prioritised alongside feature work + +Ignoring technical debt is a business decision with predictable long-term costs. + +### Operational Resilience by Design + +Architecture must support operability, observability, and recovery. Failure is an inherent property of complex systems, not an exceptional condition. + +Systems should be designed to: +- Fail in controlled, observable ways +- Degrade gracefully rather than catastrophically +- Recover quickly with minimal manual intervention + +Operational resilience is a design responsibility, not an afterthought. + +### Designing for Change and Optionality + +Requirements, technologies, and markets evolve. Architecture should preserve **optionality**, favouring designs that allow change without disproportionate cost. + +Leaders should encourage: +- Incremental evolution over large rewrites +- Reversible decisions where possible +- Interfaces that localise change + +The objective is not to eliminate uncertainty, but to make it survivable. + +--- + +## Architecture as a Leadership Responsibility + +Architecture is not owned solely by architects or senior technologists. It is a **leadership concern** because architectural choices directly affect delivery speed, organisational structure, and long-term business outcomes. + +Leadership is responsible for: +- Ensuring architectural coherence across teams +- Balancing local optimisation against system-wide health +- Making trade-offs explicit and visible +- Intervening when architectural drift threatens sustainability + +Strong architectural leadership enables teams to move quickly without breaking the system they depend on. + +--- + +## Architecture, Teams, and Conway’s Law + +System structure and organisational structure are tightly coupled. Team boundaries, communication patterns, and incentives shape the architecture that emerges over time. + +Leaders must recognise that: +- Poor team design produces brittle architectures +- Excessive coupling between teams creates delivery friction +- Clear ownership enables accountability and faster change + +Architectural thinking therefore cannot be separated from team design and organisational structure. These are different views of the same underlying system. + +--- + +## Process: Structure Without Bureaucracy + +Processes exist to manage risk, enable scale, and support coordination. They do not exist to create certainty, enforce compliance for its own sake, or substitute for judgement. Poorly designed processes increase friction, hide problems, and slow learning. Well-designed processes make work visible, reduce cognitive load, and improve flow. + +From a leadership perspective, the objective is not to minimise process, but to **apply just enough structure** to support consistent outcomes while preserving adaptability. Drawing from DeMarco, Humphrey, and McConnell, effective process design exhibits the following characteristics: + +### Lightweight, Purposeful Documentation + +Documentation should support understanding, continuity, and governance without becoming a burden. **Docs-as-Code** practices integrate documentation into normal engineering workflows, ensuring it is version-controlled, reviewable, and kept close to the systems it describes. + +Documentation should exist to: +- Preserve intent and decisions +- Support onboarding and continuity +- Enable safe change and governance +- Satisfy regulatory and assurance needs where applicable + +Documentation that does not serve one of these purposes should be questioned. + +### Data-Informed, Not Data-Driven, Decision-Making + +Metrics exist to support learning and improvement, not to replace judgement. Leaders should prioritise a small number of meaningful measures that illuminate system behaviour, avoiding vanity metrics that reward activity over outcomes. + +Effective use of data: +- Highlights bottlenecks and constraints +- Reveals trends rather than snapshots +- Informs prioritisation and trade-offs +- Supports constructive dialogue rather than performance theatre + +Measurement should prompt questions, not dictate answers. + +### Iterative Delivery Underpinned by Systems Thinking + +Iteration is not an end in itself. Iterative delivery is valuable because it shortens feedback loops, reduces batch size, and enables learning under real conditions. + +For complex, multidisciplinary products, leaders must balance iterative execution with system-level thinking, ensuring that: +- Short-term iterations contribute to long-term coherence +- Local optimisations do not undermine global outcomes +- Risk is surfaced early rather than deferred + +Iteration without systems thinking leads to fragmentation; systems thinking without iteration leads to stagnation. + +### Process and Framework Consistency Over Tool Uniformity + +Consistency of thinking and approach matters more than uniformity of tools. Shared principles, common delivery language, and aligned expectations enable collaboration across teams far more effectively than mandated toolsets. + +Tool standardisation is justified when it: +- Reduces systemic risk (e.g. security, compliance) +- Improves onboarding and mobility +- Lowers operational overhead + +Tool diversity is acceptable when it demonstrably improves team effectiveness without increasing system-level friction. Tool choices should be deliberate, explicit, and revisited as context changes. + +### Regulatory Compliance Without Delivery Paralysis + +In regulated or safety-critical environments, process and documentation requirements are unavoidable. However, compliance should be integrated into the flow of work rather than treated as a separate phase or gate. + +Leaders should ensure that: +- Compliance artefacts are produced incrementally +- Assurance activities support learning rather than delay +- Evidence is generated as a by-product of delivery + +Compliance that cannot coexist with flow is a design failure, not an inevitability. + +Processes should **enable engineers**, not encumber them. + +--- + +## Teams as Systems: Organising for Flow + +Teams are not static containers for work; they are dynamic systems that shape communication, decision-making, and delivery outcomes. Organisational structure is therefore a primary lever for improving flow and reducing complexity. + +Effective team design focuses on reducing unnecessary coordination while maximising end-to-end ownership. + +### Stream-Aligned Teams + +Stream-aligned teams are organised around the continuous delivery of value to a specific customer or user need. They own work from concept through to production and operation, minimising handoffs and external dependencies. + +Characteristics of effective stream-aligned teams include: +- Clear ownership boundaries +- End-to-end accountability +- Direct access to feedback +- Authority to make local decisions + +These teams optimise for learning speed and customer impact. + +### Enabling and Platform Teams + +Enabling and platform teams exist to reduce friction and cognitive load for stream-aligned teams. They provide shared services, tooling, infrastructure, and expertise that would be inefficient or distracting to duplicate. + +Their success is measured by: +- Reduced complexity for consuming teams +- Improved reliability and consistency +- Accelerated delivery across the system + +They act as force multipliers, not gatekeepers. + +### Cognitive Load as a First-Order Constraint + +Cognitive load limits a team’s ability to reason, learn, and deliver safely. Excessive scope, unclear ownership, or tightly coupled dependencies degrade performance even in highly capable teams. + +Leadership must actively manage: +- The number of responsibilities a team carries +- The complexity of systems they own +- The volume of coordination required + +Reducing cognitive load is a leadership responsibility, not an individual coping strategy. + +### Deliberate Communication and Team Interfaces + +Communication paths and team interfaces should be treated as explicit design choices. Clear interaction patterns reduce misunderstandings, delays, and duplicated effort. + +Effective organisations: +- Make ownership and interfaces explicit +- Minimise cross-team dependencies +- Encourage direct communication where appropriate +- Avoid reliance on informal hero networks + +Well-designed teams reduce complexity and accelerate delivery. + +--- + +## Outcomes, Not Outputs + +Delivery success is measured by the value created for customers and the business, not by the volume of work completed. Outputs are means; outcomes are ends. + +Focusing on outcomes requires a shift in how work is planned, prioritised, and assessed. + +### Product-Centric, Outcome-Oriented Delivery + +Engineering effort should be aligned to long-lived products or services rather than short-term projects. Products represent ongoing value streams with measurable outcomes. + +Outcome orientation requires: +- Clear hypotheses about value +- Explicit success measures +- Willingness to stop or change direction based on evidence + +Shipping features that do not change outcomes is not progress. + +### Fast, Meaningful Feedback Loops + +Short feedback loops enable learning under real conditions. Continuous integration, continuous delivery, telemetry, and user feedback provide insight into how systems behave in practice. + +Leaders should prioritise: +- Reducing time from idea to feedback +- Making outcomes visible +- Acting on what is learned + +Feedback without action is noise. + +### Integrating Technical Debt and Operational Risk into Delivery + +Technical debt and operational risk are not secondary concerns; they directly affect the organisation’s ability to deliver value in the future. + +Effective delivery systems: +- Make debt and risk visible +- Treat them as planning inputs +- Balance short-term gains against long-term costs + +Ignoring these factors creates an illusion of speed while eroding future capability. + +Success is not delivering more work. +Success is **reliably delivering outcomes that matter**. + +--- + +## Partnering with the Business + +Engineering leadership cannot be effective in isolation from the business it serves. Technology is not a support function that merely executes requests; it is a primary driver of organisational capability, speed, and differentiation. As such, engineering leadership must operate as a **peer partner** to product, commercial, and operational leadership. + +This partnership is not transactional. It is a shared responsibility for outcomes. + +### Shared Ownership of Outcomes + +Engineering, product, and business leaders jointly own delivery outcomes. Successes and failures are collective, not functionally isolated. When ownership is fragmented, incentives diverge and local optimisation replaces systemic thinking. + +Shared ownership: +- Aligns priorities across disciplines +- Encourages honest trade-off discussions +- Prevents engineering from becoming a downstream execution function + +Outcomes belong to the system, not to a single role or department. + +### Transparent Roadmapping and Trade-offs + +Roadmaps are communication tools, not guarantees. They exist to make intent, priorities, constraints, and uncertainty visible. + +Effective roadmapping: +- Surfaces technical dependencies and risks +- Makes capacity and trade-offs explicit +- Enables negotiation rather than false certainty + +Engineering leadership must ensure that technical reality is visible in roadmap discussions, including the cost of delay, the impact of technical debt, and the consequences of deferring risk. + +### Customer and Commercial Awareness + +Engineers should understand the customer and commercial context in which they operate. This includes user needs, market pressures, regulatory drivers, and economic constraints. + +Commercial awareness: +- Improves decision quality at the point of execution +- Connects technical work to real-world impact +- Reinforces purpose beyond internal delivery metrics + +Business context is not a distraction from engineering work; it is essential input to it. + +### Technology as a Strategic Lever + +Engineering leadership must participate in shaping business strategy, not merely executing it. Technical capabilities and constraints inform what is feasible, scalable, and sustainable. + +When technology is excluded from strategy: +- Plans become fragile +- Risk is hidden until late +- Opportunities are missed + +Technology is not neutral; it actively shapes strategic options. + +### Shared Metrics and Language + +Engineering and business functions must share a common language for success. Metrics should reflect outcomes meaningful to both perspectives, rather than reinforcing functional silos. + +Shared metrics: +- Reinforce collective accountability +- Reduce adversarial planning +- Focus attention on value creation rather than activity + +Strong partnerships replace handovers with collaboration and contracts with trust. + +--- + +## Sustainability, Resilience, and Incident Management + +Sustained high performance is impossible without systems and people that can withstand stress and recover from failure. Sustainability and resilience are not secondary concerns; they are prerequisites for long-term value delivery. + +### Sustainable Pace as a Leadership Responsibility + +Sustainable delivery requires managing workload, prioritisation, and expectations over time. Chronic urgency and burnout are signals of systemic failure, not dedication. + +Leadership must: +- Protect teams from unsustainable demand +- Make explicit trade-offs when capacity is exceeded +- Treat recovery and slack as essential, not optional + +A pace that cannot be sustained indefinitely is not success. + +### Operational Resilience by Design + +Complex systems will fail. The question is not whether failures occur, but how systems behave when they do. + +Resilient systems: +- Fail in controlled, observable ways +- Degrade gracefully rather than catastrophically +- Recover quickly with minimal manual intervention + +Resilience emerges from architecture, process, and team design working together. + +### Incident Management as Learning + +Incidents are opportunities to learn about system behaviour under stress. Blameless postmortems are essential for uncovering systemic causes rather than assigning fault. + +Effective incident management: +- Focuses on understanding contributing factors +- Produces actionable improvements +- Feeds learning back into architecture, process, and training + +Repeated incidents indicate unresolved systemic issues. Leadership is accountable for ensuring learning leads to change. + +### Continuous Improvement and Adaptation + +Resilient organisations treat change as constant. Learning does not stop once systems are in production. + +A culture of continuous improvement: +- Encourages experimentation within safe boundaries +- Regularly revisits assumptions and decisions +- Evolves processes and structures based on evidence + +Resilience is as much about people and processes as it is about technology. + +--- + +## Strategy and Portfolio-Level Thinking + +Engineering leadership extends beyond individual teams or products to the broader system of work across the organisation. Strategic thinking is required to align investment, capability, and risk over time. + +### Shaping and Communicating Technical Strategy + +A clear technical strategy provides direction without prescribing implementation. It articulates how technology supports business goals and how systems should evolve over time. + +Effective technical strategy: +- Aligns teams around shared intent +- Guides investment and prioritisation +- Provides context for local decisions + +Strategy sets direction; teams determine execution. + +### Managing Architectural Coherence Across Programmes + +As organisations scale, architectural fragmentation becomes a significant risk. Leadership must ensure coherence across teams and initiatives without imposing rigid uniformity. + +This requires: +- Shared principles and standards +- Visibility into cross-cutting decisions +- Active stewardship of system boundaries + +Coherence enables scale without excessive coordination overhead. + +### Balancing Local Optimisation and System Health + +Teams naturally optimise for their local goals. Without oversight, this can degrade overall system performance. + +Leadership must: +- Encourage collaboration across boundaries +- Resolve conflicts between local and global objectives +- Optimise for end-to-end outcomes rather than isolated efficiency + +System health ultimately determines organisational performance. + +### Portfolio Management as a Leadership Discipline + +Engineering initiatives should be managed as a portfolio, balancing: +- Innovation and exploration +- Maintenance and operational stability +- Risk reduction and technical debt +- Scaling and optimisation + +Capacity is finite. Trade-offs are unavoidable and must be made deliberately. + +### Developing Future Technical Leaders + +Long-term success depends on a pipeline of capable leaders who understand both technology and systems thinking. + +Leadership development should: +- Emphasise judgement over process compliance +- Encourage systems-level perspective +- Prepare individuals to navigate complexity and trade-offs + +Strategic leadership ensures engineering effort remains aligned with organisational goals as scale and complexity increase. + +## Interplay and Resolution of Principles: Navigating Inherent Tensions + +This philosophy explicitly recognises that its principles, while individually sound, are not always perfectly aligned in practice. Complex systems contain unavoidable tensions. Effective engineering leadership is revealed not by eliminating these tensions, but by **resolving them deliberately and consistently** in service of long-term systemic health. + +Trade-offs are inevitable. What matters is the logic used to make them. + +### Autonomy vs. Architectural Coherence + +Autonomy enables speed, ownership, and engagement. Unbounded autonomy, however, leads to architectural fragmentation, increased cognitive load, duplicated effort, and accumulating technical debt. + +Resolution lies in **Lightweight Governance** and **Decision Transparency**: +- Teams operate autonomously within clearly defined architectural guardrails +- Decisions with system-wide impact are made visible +- Intent and rationale are preserved over time + +The guiding principle for resolution is **Systemic Health and Long-Term Value Creation**, ensuring local autonomy strengthens rather than weakens the whole. + +### Fast Feedback vs. Sustainable Pace + +Rapid feedback accelerates learning and adaptation, but unbounded urgency erodes sustainability and judgement. + +Resolution requires: +- **Data-informed understanding of capacity** +- Explicit prioritisation when demand exceeds supply +- Treating sustainable pace as a leadership constraint, not a team preference + +The philosophical arbiter is the recognition that **long-term productivity depends on human wellbeing**, not continuous acceleration. + +### Local Optimisation vs. System-Wide Outcomes + +Teams optimising solely for their own efficiency can inadvertently create bottlenecks, dependencies, and misalignment elsewhere in the system. + +Resolution demands **Strategic and Portfolio-Level Thinking**: +- Outcomes are defined and measured at the system level +- Shared success metrics reinforce collective accountability +- Leadership intervenes when local gains harm global performance + +The health of the whole system determines the effectiveness of its parts. + +### Tool Uniformity vs. Optimal Alignment + +Mandated tool uniformity can simplify governance and support, but can also reduce effectiveness when imposed without regard for context. + +Resolution is grounded in **deliberate, value-driven choice**: +- Uniformity is justified when it demonstrably reduces risk or friction +- Diversity is acceptable when it improves effectiveness without harming coherence +- Decisions are revisited as constraints and needs evolve + +Dogma is replaced with informed judgement. + +These tensions are not resolved once, but continuously, through transparent decision-making and shared understanding of purpose. + +--- + +## Framing Model and Progressive Adoption + +This philosophy is intentionally structured to support **progressive, context-driven adoption** rather than wholesale transformation. Organisations differ in maturity, constraints, and readiness for change. + +The approach can be understood through four interconnected pillars: + +* **Leadership Foundations** + Purpose, decision systems, boundaries, and accountability for system health. + +* **Architectural and Operational Enablers** + Architecture, governance, resilience, and technical health. + +* **Delivery Systems** + Process design, team structures, flow, and feedback mechanisms. + +* **Strategic and Business Integration** + Portfolio thinking, business partnership, and long-term alignment. + +Adoption should evolve incrementally: + +* **Level 1: Establish Foundations** + Clarify purpose, decision rights, and non-negotiables. + +* **Level 2: Enable Flow and Delivery Discipline** + Improve team structures, reduce bottlenecks, and shorten feedback loops. + +* **Level 3: Strategic Optimisation** + Align technical strategy, portfolio management, and business outcomes. + +* **Level 4: Continuous Evolution** + Embed learning, adaptation, and leadership development as ongoing practices. + +Each level builds on the previous, allowing progress without destabilising the system. + +--- + +## Principle Application Cautions and Organisational Realities + +While powerful in principle, this philosophy must be applied with realism and judgement. Its effectiveness depends on organisational maturity and a willingness to confront uncomfortable truths. + +* **Lightweight governance must still provide discipline.** + It is not an excuse for inconsistency or lack of rigour. In regulated or safety-critical environments, governance may still be extensive, but it must remain purposeful and flow-oriented. + +* **Autonomy must be bounded.** + Without clear scope, interfaces, and accountability, autonomy degrades into fragmentation. Leadership must actively manage this boundary. + +* **Business partnership requires maturity.** + Where trust or shared accountability is lacking, leadership must invest deliberately in building these relationships rather than assuming they exist. + +* **Sustainability must be actively protected.** + Without explicit leadership intervention, delivery pressure will erode wellbeing and long-term capability. + +* **Outcome orientation is hard.** + Measuring real outcomes requires data capability, patience, and cultural change. It cannot be achieved through reporting alone. + +* **Existing dysfunctions must be acknowledged.** + Deep technical debt, siloed structures, or low trust cannot be bypassed. Progress may be incremental and uneven. + +* **Tool decisions require discipline.** + Avoid substituting tool alignment for process and principle alignment. Tools amplify intent; they do not define it. + +Adoption is a journey of continuous improvement, not a one-time implementation. + +--- + +## Emphasising Business Outcomes + +Engineering leadership is fundamentally a business discipline. Beyond technical excellence, it exists to: + +* Drive customer and commercial outcomes +* Link architectural and delivery decisions to business priorities +* Co-create strategy with product and commercial leadership +* Articulate the business value of engineering clearly and credibly + +Technology decisions that are not grounded in business reality eventually become liabilities. + +--- + +## In Summary + +This approach to engineering leadership and delivery combines people-centred leadership, architectural thinking, and outcome-focused delivery into a coherent, system-oriented philosophy. + +It provides: +- A clear statement of purpose and responsibility +- A set of non-negotiable principles to guide judgement +- A lens for resolving inevitable trade-offs +- A foundation upon which context-specific frameworks can be built + +It is not a rigid framework, methodology, or toolset. +It is a philosophy that defines *what matters* and *why*, while leaving *how* to be shaped by context. + +Its success depends on continuous learning, cultural evolution, and deliberate leadership intent. + +--- + +## Supporting Bibliography + +* The Deadline: A Novel About Project Management – Tom DeMarco +* How to Measure Anything – Douglas W. Hubbard +* Thinking in Systems – Donella Meadows +* Why Motivating People Doesn’t Work... and What Does – Susan Fowler +* Turn the Ship Around! – L. David Marquet +* Extreme Ownership – Jocko Willink & Leif Babin +* Reinventing Organizations – Frederic Laloux +* Team of Teams – General Stanley McChrystal +* Drive – Dan Pink +* Start With Why – Simon Sinek +* Leaders Eat Last – Simon Sinek +* The Infinite Game – Simon Sinek +* The Mythical Man-Month – Frederick Brooks +* Peopleware – Tom DeMarco & Timothy Lister +* Waltzing with Bears: Managing Risk on Software Projects – Tom DeMarco & Timothy Lister +* Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency – Tom DeMarco +* Software Architecture: The Hard Parts – Mark Richards & Neal Ford +* Building Evolutionary Architectures – Ford, Parsons, Kua +* Fundamentals of Software Architecture – Mark Richards & Neal Ford +* A Philosophy of Software Design – John Ousterhout +* Software Systems Architecture – Rozanski & Woods +* Team Topologies – Matthew Skelton & Manuel Pais +* Accelerate: The Science of Lean Software and DevOps – Nicole Forsgren, Jez Humble, Gene Kim +* The DevOps Handbook – Gene Kim, Patrick Debois, John Willis, Jez Humble +* The Art of Scalability – Martin Abbott & Michael Fisher +* Managing the Design Factory – Donald G. Reinertsen +* Principles of Product Development Flow – Donald G. Reinertsen +* Docs as Code – Anne Gentle +* Communication Patterns – Brandenburg & Strohschneider +* The Agile Manifesto +* The Five Dysfunctions of a Team – Patrick Lencioni +* Radical Candor – Kim Scott +* The Manager’s Path – Camille Fournier +* An Elegant Puzzle: Systems of Engineering Management – Will Larson +* Team Geek – Ben Collins-Sussman, Brian Fitzpatrick, and Dan Pilone +* Managing Humans – Michael Lopp + +This approach exists to be adapted. What matters most is not following this document to the letter, but using it to ask better questions, design better systems, and deliver meaningful value to customers and the business over time. + +These ideas grew into [Firmitas](/firmitas/), a framework for sustainable engineering delivery. diff --git a/site/docs/radar-chart/index.md b/site/docs/radar-chart/index.md new file mode 100644 index 0000000..ad75947 --- /dev/null +++ b/site/docs/radar-chart/index.md @@ -0,0 +1,1653 @@ +--- +title: Tech Radar chart +hide: [toc] +--- + +# Tech Radar chart + +Welcome to our **Interactive Technology Radar**. This responsive living map visualises the 77 technologies, frameworks, operational platforms, and engineering patterns evaluated across our software and systems practice. + +- **Touch & Click Friendly**: Tap or click any numbered marker on the radar or in the mobile-friendly directory below to view detailed rationale, tags, and guideline links. +- **Filter & Search**: Use the quadrant and ring filter buttons or type in the search bar to highlight specific technologies. +- **Methodology & Rules**: Read **[Overview & Methodology](../radar/)** for ring criteria and ARB governance. +- **Tabular Directory**: View the complete reference catalog in the **[Complete Blips Directory](../radar/directory/)**. + +
+ + +
+ + +
+ + +
+ +
+ +
+ + +
+ + +
+ + +
+ Ring: +
+ + + + + +
+
+ + +
+ Domain: +
+ + + + + + + +
+
+
+ + +
+
+ + + + + + + + + + + + + + + + ADOPT + TRIAL + ASSESS + HOLD + 1. LANGUAGES & FRAMEWORKS + 2. PLATFORMS & OPERATIONS + 3. TOOLS & UTILITIES + 4. METHODS & PATTERNS + + + + 1 + #1 ADRs (ADOPT) + + + + + 2 + #2 Agile (ADOPT) + + + + + 3 + #3 arc42 (ADOPT) + + + + + 4 + #4 AsyncAPI (ADOPT) + + + + + 5 + #5 Barr Group Coding Guidelines (ADOPT) + + + + + 6 + #6 C# (ADOPT) + + + + + 7 + #7 C++ (ADOPT) + + + + + 8 + #8 C (ADOPT) + + + + + 9 + #9 C4 Model (ADOPT) + + + + + 10 + #10 C4 (ADOPT) + + + + + 11 + #11 CertC (ASSESS) + + + + + 12 + #12 CICD (ADOPT) + + + + + 13 + #13 CloudEvents (ASSESS) + + + + + 14 + #14 Coverity (ASSESS) + + + + + 15 + #15 CQRS (ASSESS) + + + + + 16 + #16 Diátaxis (ADOPT) + + + + + 17 + #17 Docker (ADOPT) + + + + + 18 + #18 Docs as code (ADOPT) + + + + + 19 + #19 DoD (ADOPT) + + + + + 20 + #20 Domain-driven design (ADOPT) + + + + + 21 + #21 DoR (ASSESS) + + + + + 22 + #22 DRY (Don't Repeat Yourself) (ADOPT) + + + + + 23 + #23 Edge Architectures for IoT (ADOPT) + + + + + 24 + #24 ETSI EN 303 645 (ADOPT) + + + + + 25 + #25 Event-Driven Architecture (EDA) (ADOPT) + + + + + 26 + #26 Event Driven (ADOPT) + + + + + 27 + #27 Event Sourcing (ASSESS) + + + + + 28 + #28 Event storming (ADOPT) + + + + + 29 + #29 Event Streaming (ADOPT) + + + + + 30 + #30 FreeRTOS (ADOPT) + + + + + 31 + #31 gcc (ADOPT) + + + + + 32 + #32 Go (golang) (ADOPT) + + + + + 33 + #33 Google Cloud Platform (ADOPT) + + + + + 34 + #34 GraphQL (ADOPT) + + + + + 35 + #35 gRPC (ADOPT) + + + + + 36 + #36 ISO 30141 (ASSESS) + + + + + 37 + #37 JSON (ADOPT) + + + + + 38 + #38 Kanban (ADOPT) + + + + + 39 + #39 KISS (Keep It Simple, Stupid) (ADOPT) + + + + + 40 + #40 Layered Architecture (ADOPT) + + + + + 41 + #41 Lightweight Architecture Evaluations (ADOPT) + + + + + 42 + #42 Markdown (ADOPT) + + + + + 43 + #43 Mermaid (ADOPT) + + + + + 44 + #44 Mermaid Diagrams (TRIAL) + + + + + 45 + #45 Microservices (ADOPT) + + + + + 46 + #46 MISRA C++ (ASSESS) + + + + + 47 + #47 MISRA C (ADOPT) + + + + + 48 + #48 Modular Monolith (ADOPT) + + + + + 49 + #49 MQTT (ADOPT) + + + + + 50 + #50 oauth2 (ADOPT) + + + + + 51 + #51 OpenAPI (ADOPT) + + + + + 52 + #52 OWASP (ADOPT) + + + + + 53 + #53 PEP 8 (ADOPT) + + + + + 54 + #54 PlantUML (ADOPT) + + + + + 55 + #55 protobuf (ADOPT) + + + + + 56 + #56 Python (ADOPT) + + + + + 57 + #57 REST (ADOPT) + + + + + 58 + #58 RFC9457 (ASSESS) + + + + + 59 + #59 rRPC (ASSESS) + + + + + 60 + #60 Rust (ADOPT) + + + + + 61 + #61 Scrum (ADOPT) + + + + + 62 + #62 SEI ATAM (ASSESS) + + + + + 63 + #63 SEI CBAM (ASSESS) + + + + + 64 + #64 Serverless (ADOPT) + + + + + 65 + #65 SOA (ASSESS) + + + + + 66 + #66 SQLite (ADOPT) + + + + + 67 + #67 Structurizr (ASSESS) + + + + + 68 + #68 TOGAF (ADOPT) + + + + + 69 + #69 UML Modeling (ADOPT) + + + + + 70 + #70 V-Model (ASSESS) + + + + + 71 + #71 Visual Studio Code (ADOPT) + + + + + 72 + #72 VS code (ADOPT) + + + + + 73 + #73 Waterfall (HOLD) + + + + + 74 + #74 YAGNI (You Aren’t Gonna Need It) (ADOPT) + + + + + 75 + #75 YAML (ADOPT) + + + + + 76 + #76 Yocto Project (ADOPT) + + + + + 77 + #77 Zachman Framework (ASSESS) + + +
+ + +
+ 💡 Tap or click any numbered marker on the radar to inspect complete technical details. +
+
+ + + + + +
+

Technologies by Quadrant

+
+
+ +
+ + + + +