Steve Kaschimer - Tech Notes

Why GitHub is the DevSecOps Platform of Choice

2025-10-27T00:00:00Z

In the evolving landscape of software development, DevSecOps has emerged as a critical discipline - one that integrates security into every phase of the software delivery lifecycle. As organizations strive to ship faster without compromising safety, the tools we choose become more than just enablers - they shape our workflows, our culture, and ultimately, our outcomes.

Among the many platforms available, GitHub stands out. Once known primarily as a code hosting service, GitHub has matured into a robust ecosystem that supports the full spectrum of DevSecOps practices. For architects and engineers tasked with embedding security into development pipelines, GitHub offers a compelling blend of automation, visibility, and developer-first design.

This post explores why GitHub is increasingly becoming the platform of choice for DevSecOps professionals, and how it can help teams move from theory to practice.

The DevSecOps Imperative

DevSecOps isn’t just a buzzword. It’s a response to real-world challenges. Traditional security models often treated security as a gatekeeper, bolted onto the end of the development process. This led to delays, friction between teams, and vulnerabilities slipping through the cracks.

DevSecOps flips that model. It embeds security into every stage of development, from code commit to deployment. It encourages collaboration between developers, security engineers, and operations teams. And it relies heavily on automation to ensure that security checks are consistent, scalable, and fast.

But implementing DevSecOps is easier said than done. Tool sprawl, lack of integration, and resistance to change are common hurdles. That’s where platform choice becomes critical and why GitHub deserves a closer look.

GitHub’s Strengths for DevSecOps

GitHub’s appeal lies in its ability to meet developers where they already are. It’s the default platform for millions of developers, which means DevSecOps initiatives don’t have to fight for adoption. Instead, they can build on existing habits and workflows.

Here are some of the key reasons GitHub excels as a DevSecOps platform:

Developer Familiarity

GitHub is already deeply embedded in the daily routines of most development teams. Pull requests, issues, and discussions are part of the rhythm. This familiarity reduces the learning curve and makes it easier to introduce security practices without disrupting productivity.

Built-in Automation with GitHub Actions

GitHub Actions allows teams to automate everything from builds and tests to security scans and compliance checks. Workflows can be triggered on pull requests, commits, or scheduled intervals, making it easy to enforce security policies continuously.

Whether you’re running SAST tools, checking for secrets, or validating infrastructure-as-code, GitHub Actions provides a flexible and native way to integrate these steps into your pipeline.

Native Security Tooling

GitHub has invested heavily in security features that align with DevSecOps principles:

CodeQL: A powerful static analysis engine that lets you write custom queries to detect vulnerabilities in code.
Secret Scanning: Automatically detects credentials and tokens committed to repositories.
Dependency Review: Highlights changes to dependencies in pull requests and flags known vulnerabilities.
Security Overview: Provides a centralized dashboard for tracking vulnerabilities across repositories.

These tools are tightly integrated into the GitHub experience, reducing the need for external platforms and making security more accessible to developers.

Auditability and Traceability

Every action on GitHub, from commits to workflow runs, is logged and traceable. This makes it easier to meet compliance requirements, conduct forensic analysis, and demonstrate accountability.

Open Source Ecosystem

GitHub’s open nature allows teams to leverage community tools while maintaining enterprise-grade controls. Whether you’re integrating with Snyk, Trivy, or custom linters, GitHub’s extensibility supports a wide range of security use cases.

Real-World Use Cases

Let’s look at how GitHub supports DevSecOps in practice.

Automating Security Checks

A DevSecOps team might use GitHub Actions to run CodeQL scans on every pull request. If a vulnerability is detected, the workflow can block the merge and notify the developer with actionable feedback. This ensures that security is enforced without manual intervention.

Managing Secrets

GitHub’s secret scanning can detect exposed credentials in real time. Combined with environment secrets and access controls, teams can reduce the risk of accidental leaks and enforce secure handling of sensitive data.

Dependency Hygiene

With dependency review and Dependabot alerts, teams can stay ahead of known vulnerabilities in third-party packages. These features integrate directly into pull requests, making it easy to assess risk before merging.

These examples aren’t hypothetical. They’re part of the daily workflow for many DevSecOps teams using GitHub.

Common Pitfalls and How GitHub Helps

No platform is perfect, and GitHub is no exception. But many of the common challenges in DevSecOps are mitigated by GitHub’s design.

Security vs. Speed

One of the biggest concerns is that security slows down delivery. GitHub’s automation features help strike a balance. Security checks run in parallel with development, and issues are surfaced early when they’re easier to fix.

Tool Fragmentation

Managing multiple tools across different platforms can be a nightmare. GitHub consolidates many security functions into a single interface, reducing complexity and improving visibility.

Lack of Visibility

Security teams often struggle to see what’s happening in development. GitHub’s dashboards, logs, and integrations provide a clear view of code changes, workflow runs, and security alerts.

Strategic Considerations

For organizations considering GitHub as a DevSecOps platform, there are a few strategic questions to address:

Do you need GitHub Advanced Security?

While many features are available for free, GAS unlocks deeper capabilities like custom CodeQL queries and enterprise-wide security insights.
How does GitHub align with compliance needs?

GitHub’s audit logs, access controls, and workflow automation can support compliance frameworks like SOC 2, ISO 27001, and NIST.
Can GitHub scale across teams?

With organization-level policies, reusable workflows, and role-based access, GitHub supports DevSecOps at scale.

Closing Thoughts

DevSecOps is no longer optional. It’s a necessity. As threats evolve and delivery cycles accelerate, security must be built into the fabric of development. GitHub offers a platform that supports this vision, combining developer-first design with powerful security tooling.

For DevSecOps architects and engineers, GitHub isn’t just a place to host code. It’s a strategic enabler of secure, scalable, and efficient software delivery.

If you haven’t explored GitHub’s security features recently, now is a good time to dive in. Start small, automate what you can, and build a culture where security is everyone’s responsibility.

Need help? Ask me!

steve.kaschimer@slalom.com

5 Tailwind CSS Tips for Better Productivity

2025-10-29T00:00:00Z

Tailwind CSS has revolutionized the way I write CSS. Here are five tips that have significantly improved my workflow.

1. Use @apply for Repeated Patterns

While Tailwind promotes utility-first CSS, sometimes you have patterns that repeat. Use @apply to create reusable components:

.btn {
  @apply px-4 py-2 rounded-lg font-medium transition-colors duration-200;
}

.btn-primary {
  @apply bg-blue-600 hover:bg-blue-700 text-white;
}

2. Leverage the JIT Compiler

The Just-In-Time compiler generates styles on-demand, giving you:

Faster build times
Smaller file sizes
Arbitrary values: w-[347px]

3. Create Custom Utilities

Extend Tailwind with your own utilities in tailwind.config.js:

module.exports = {
  theme: {
    extend: {
      colors: {
        brand: {
          500: '#3B82F6',
          600: '#2563EB',
        }
      }
    }
  }
}

4. Use Dark Mode Variants

Tailwind makes dark mode incredibly easy:

<div class="bg-white dark:bg-gray-900 text-gray-900 dark:text-white">
  Content that adapts to theme
</div>

5. Install the Tailwind CSS IntelliSense Extension

If you're using VS Code, this extension is a must-have. It provides:

Autocomplete for class names
Linting and validation
Hover previews of CSS values

Conclusion

These tips have made working with Tailwind even more enjoyable. The framework's flexibility allows you to build beautiful, responsive designs quickly.

What are your favorite Tailwind tips? Let me know!

steve.kaschimer@slalom.com

Getting Started with Eleventy

2025-10-30T00:00:00Z

Eleventy (or 11ty) is a fantastic static site generator that's simple, flexible, and incredibly fast. If you're looking to build a blog, documentation site, or any static website, Eleventy is an excellent choice.

Why Eleventy?

Here are some reasons why I love working with Eleventy:

Simple & Flexible: Works with multiple template languages
Fast Build Times: Incredibly quick, even for large sites
No Client-Side JavaScript Required: Pure static HTML by default
Great Documentation: Easy to learn and well-documented
Active Community: Lots of plugins and starter templates available

Basic Setup

Getting started with Eleventy is straightforward. Here's a quick overview:

# Install Eleventy
npm install @11ty/eleventy

# Create a simple template
echo '# Hello World' > index.md

# Run Eleventy
npx @11ty/eleventy --serve

That's it! You now have a working Eleventy site.

Key Concepts

Layouts

Layouts are templates that wrap your content. They're perfect for creating consistent page structures.

Collections

Collections let you group related content together. For a blog, you'd typically have a "posts" collection.

Filters

Filters transform data in your templates. For example, formatting dates or truncating text.

Next Steps

Now that you know the basics, here are some things to explore:

Add styling with your favorite CSS framework
Create custom filters for your specific needs
Explore plugins to extend functionality
Deploy to GitHub Pages, Netlify, or Vercel

Conclusion

Eleventy strikes a perfect balance between simplicity and power. It gets out of your way and lets you focus on creating content.

Happy building!

Need help? Ask me!

steve.kaschimer@slalom.com

Secrets Management on GitHub: Best Practices and Pitfalls

2025-11-05T00:00:00Z

Secrets are the lifeblood of modern applications. API keys, database credentials, encryption tokens - these tiny strings unlock access to critical systems and sensitive data. But when secrets are mishandled, they become one of the fastest paths to a breach. In fact, exposed credentials are among the most common causes of security incidents today.

If you’ve ever seen a developer hardcode an API key into a config file or commit a password to a public repository, you know how easy it is for secrets to leak. And once they’re out, attackers don’t need to break encryption or exploit zero-days. They simply use the keys you left behind.

This article dives deep into how GitHub helps you manage secrets securely, what best practices you should adopt, and the pitfalls that can derail even well-intentioned teams. We’ll cover secret scanning, environment variables, and strategies for secure storage, all through the lens of real-world DevSecOps challenges.

Why Secrets Management Matters

Secrets are everywhere in modern software. They connect microservices, authenticate APIs, and enable cloud deployments. But the convenience of secrets comes with risk. When credentials are embedded in source code, they often end up in version control systems, which are designed to preserve history forever. That means even if you remove a secret later, it can still be retrieved from old commits.

Attackers know this. Automated bots constantly scan public repositories for exposed keys. If they find one, they can exploit it within minutes, sometimes before you even realize it’s there. The consequences range from unauthorized access to full-blown data breaches, and the cost of remediation skyrockets when secrets are compromised in production environments.

Managing secrets properly isn’t just a technical best practice; it’s a compliance requirement. Frameworks like SOC 2, PCI DSS, and ISO 27001 mandate secure handling of sensitive information. Hardcoding credentials violates these standards and can lead to regulatory penalties.

The GitHub Landscape for Secrets Management

GitHub has evolved beyond being a code hosting platform. It now offers a suite of features designed to help teams detect, prevent, and manage secrets securely. These include:

Secret Scanning: GitHub automatically scans repositories for patterns that match known credential formats. If it finds something suspicious, it alerts you immediately.
Environment Secrets: GitHub Actions allows you to store secrets at the repository, organization, or environment level. These secrets are encrypted and injected into workflows at runtime.
Dependabot Alerts: While primarily focused on dependency vulnerabilities, Dependabot complements secret scanning by reducing the risk of compromised libraries that might expose secrets indirectly.

Let’s break these down and see how they fit into a secure development workflow.

Secret Scanning: Your First Line of Defense

Secret scanning is GitHub’s proactive approach to preventing leaks. It works by analyzing commits for patterns that resemble credentials, such as API keys, tokens, and passwords, and flags them before they become a problem.

When secret scanning is enabled, GitHub checks every push to your repository. If it detects a secret, it sends an alert to repository administrators and, in some cases, automatically notifies the service provider so they can revoke the compromised key.

This feature is particularly powerful for public repositories, where exposure can lead to immediate exploitation. But it’s equally valuable for private repos, because insider mistakes are just as dangerous as external threats.

The key to making secret scanning effective is enabling it across all repositories-not just the ones you think are sensitive. Secrets have a way of showing up in unexpected places, like test scripts or temporary configuration files.

Environment Secrets: Secure Injection for Workflows

GitHub Actions introduced a game-changing feature for secrets management: environment secrets. Instead of hardcoding credentials into workflow files, you store them securely in GitHub’s encrypted vault. At runtime, these secrets are injected into the workflow as environment variables.

This approach solves two major problems. First, it keeps secrets out of version control, so they’re never exposed in commits. Second, it allows you to rotate credentials without modifying workflow files, reducing operational friction.

Secrets can be scoped at different levels:

Repository-level: Accessible to workflows in a single repository.
Organization-level: Shared across multiple repositories, ideal for enterprise environments.
Environment-level: Tied to specific deployment environments like staging or production, adding an extra layer of control.

When using environment secrets, it’s critical to follow the principle of least privilege. Only grant workflows access to the secrets they need, and avoid overloading a single environment with unrelated credentials.

Dependabot: Keeping Dependencies Secure

While Dependabot isn’t a secrets management tool in the strict sense, it plays a critical role in reducing the risk of compromised credentials through vulnerable dependencies. Secrets often interact with third-party libraries such as SDKs, API clients, or infrastructure modules, and if those libraries contain security flaws, your secrets can be exposed indirectly.

Dependabot continuously monitors your project’s dependencies for known vulnerabilities. When it detects an issue, it automatically opens a pull request with the recommended version upgrade. This proactive approach ensures that the libraries handling your secrets remain secure and up to date.

Including Dependabot in your security strategy is about defense in depth. Even if you manage secrets perfectly, a vulnerable dependency can undermine your efforts. By automating dependency updates, you reduce the attack surface and strengthen the overall integrity of your workflows.

Common Pitfalls in Secrets Management

Even with GitHub’s tooling, secrets management can go wrong. One of the most common mistakes is assuming that private repositories are inherently safe. They’re not. Insider threats, misconfigured permissions, and accidental sharing can all lead to exposure.

Another pitfall is neglecting to rotate secrets. Credentials that never change become ticking time bombs. If a secret is compromised and you don’t rotate it promptly, attackers can maintain access indefinitely.

Teams also struggle with visibility. Secrets often sprawl across multiple repositories, environments, and cloud services. Without centralized tracking, it’s easy to lose control. GitHub provides some visibility through its security dashboard, but for large organizations, integrating with a dedicated secrets manager like HashiCorp Vault or AWS Secrets Manager is essential.

Best Practices for Secure Secrets Management

The foundation of secure secrets management is simple: never store credentials in source code. But that's just the beginning. A mature approach includes:

Enabling secret scanning on all repositories.
Using environment secrets for workflows instead of hardcoding values.
Rotating credentials regularly and automating the process where possible.
Limiting access based on least privilege principles.
Auditing secret usage and reviewing logs for anomalies.
Integrating GitHub with external secret managers (such as Hashicorp Vault or Azure KeyVault) for enterprise-scale control.

These practices don’t just reduce risk, they make compliance easier and improve operational resilience.

The Future of Secrets Management on GitHub

As software supply chain attacks become more sophisticated, secrets management will continue to evolve. GitHub is already experimenting with advanced features like push protection, which blocks commits containing secrets before they even reach the repository.

Looking ahead, expect tighter integration between GitHub and cloud providers, automated secret rotation, and AI-driven anomaly detection. The goal is to make secrets management seamless, so developers can focus on building features without compromising security.

Closing Thoughts

Secrets are powerful... and dangerous. Managing them securely is one of the most important responsibilities in modern software development. GitHub provides strong tools to help, but technology alone isn’t enough. It takes discipline, clear policies, and a culture that treats security as a shared responsibility.

Start by enabling secret scanning, move your credentials into environment secrets, and adopt a rotation strategy. From there, integrate with external managers and automate wherever possible. The sooner you take these steps, the less likely you are to wake up to a breach caused by a forgotten API key in a commit from six months ago.

Security isn’t about perfection. It’s about reducing risk. And with GitHub’s capabilities, you have everything you need to make secrets management a strength, not a vulnerability.

Need help securing your secrets? Ask me!

steve.kaschimer@slalom.com

Security as Code with GitHub Actions: Automating DevSecOps

2025-11-10T00:00:00Z

Security as Code is more than a buzzword. It’s a practical approach to embedding security into the development lifecycle. Instead of treating security as a separate process, we codify policies, checks, and controls so they run automatically alongside builds and deployments. For DevSecOps professionals, this is the foundation of scalable, repeatable security.

GitHub Actions makes this vision achievable. By leveraging workflows, you can integrate security checks into CI/CD pipelines without slowing down delivery. In this post, we’ll explore what Security as Code means, why it matters, and how to implement it using GitHub Actions.

Why Security as Code Matters

Traditional security practices often rely on manual reviews and ad-hoc scans. These approaches don’t scale in modern development environments where teams push code multiple times a day. Security as Code solves this by:

Automating enforcement: Policies and checks run consistently.
Reducing human error: Less reliance on manual steps.
Improving speed: Security becomes part of the pipeline, not a bottleneck.
Enhancing visibility: Logs and reports are centralized and auditable.

For DevSecOps engineers, this approach aligns perfectly with the “shift-left” philosophy. that is, catching issues early when they’re cheaper and easier to fix.

“Shift-left” is a software development principle that moves critical activities, like testing and security, earlier in the lifecycle. Instead of waiting until code is complete or deployed to check for vulnerabilities, teams integrate these checks during development. The goal is simple: catch issues sooner, fix them faster, and reduce risk. By shifting security left, DevSecOps teams prevent costly late-stage fixes and make security a natural part of coding, not an afterthought.

GitHub Actions: The Engine Behind Security Automation

GitHub Actions is a workflow automation tool built into GitHub. It allows you to define jobs triggered by events like pushes, pull requests, or scheduled intervals. For security, this means:

Running static analysis on every commit.
- Static analysis examines source code without executing it, looking for patterns that indicate potential bugs, vulnerabilities, or compliance issues.
Scanning for secrets and credentials before merging.
Enforcing dependency checks to prevent vulnerable packages.
Validating infrastructure-as-code for compliance.
- such as: no large VMs, resources created in the correct region, affixing tags to each resource, etc.

Key Features for Security

Reusable Workflows: Share security workflows across repositories. One of the most powerful features of GitHub Actions is the ability to create reusable workflows. Instead of duplicating security checks in every repository, you can define a single workflow in a central location and reference it across multiple projects. This approach ensures consistency, reduces maintenance overhead, and accelerates adoption of security best practices.

Best Practice: Combine reusable workflows with organization-level policies to enforce usage across teams. This ensures security automation is embedded in the development process.
Marketplace Actions: Integrate tools like Snyk, Trivy, and Checkov. One of GitHub Actions’ biggest strengths is its Marketplace, which hosts thousands of pre-built actions created by GitHub and the community. For DevSecOps engineers, this means you don’t have to reinvent the wheel because security tools are ready to plug into your workflows.

Best Practice: Combine multiple Marketplace actions in a single workflow to cover different layers (dependency, containers, IaC, etc.) to ensure comprehensive coverage without adding complexity
Matrix Builds: Test security across multiple environments. Matrix builds in GitHub Actions allow you to run the same job across multiple configurations (i.e. operating systems, language versions, dependency sets, etc.) in parallel. For DevSecOps, this is a game-changer because vulnerabilities often surface only under certain conditions.

Best Practice: Combine matrix builds with Reusable workflows for consistency, Marketplace actions for specialized scans, and fail-fast strategies so a critical vulnerability halts the pipeline immediately.

Implementing Security as Code with GitHub Actions

Here’s a practical example of a workflow that runs CodeQL and secret scanning on every pull request:

name: Security Checks
on:
  pull_request:
    branches: [ main ]
jobs:
  codeql-analysis:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: github/codeql-action/init@v2
        with:
          languages: javascript
      - uses: github/codeql-action/analyze@v2

  secret-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: github/secret-scanning-action@v1

This workflow ensures that every pull request undergoes static analysis and secret scanning before merging.

Best Practices

Start Small: Begin with one or two critical checks, then expand.
Fail Fast: Configure workflows to block merges on high-severity findings.
Use Reusable Components: Standardize workflows across teams.
Monitor and Iterate: Review logs and metrics regularly.

Common Challenges

False Positives: Tune your tools to reduce noise.
Developer Resistance: Communicate the benefits and provide quick fixes.
Performance Impact: Optimize workflows to run in parallel.

Closing Thoughts

Security as Code isn’t optional. It’s essential for modern software delivery. GitHub Actions provides the flexibility and power to make it real. By automating security checks, you can reduce risk, improve compliance, and keep development moving at full speed.

Start small, iterate, and share your workflows. The sooner you embed security into your pipelines, the stronger your software supply chain becomes.

Need help? Ask me!

steve.kaschimer@slalom.com

Shift Left Without Slowing Down: DevSecOps Pipeline Design

2025-11-17T00:00:00Z

Modern software delivery is a race against time. Teams push code faster than ever, deploying multiple times a day to meet customer demands. But speed without security is a recipe for disaster. Vulnerabilities introduced early in development can cascade into production, where they’re exponentially harder and more expensive to fix. That’s why the principle of “shift left” has become a cornerstone of DevSecOps.

Shifting left means moving security checks earlier in the development lifecycle, embedding them into the same workflows that developers use every day. It’s a powerful idea, but it comes with a challenge: how do you integrate security without slowing down the pipeline? Developers want velocity. Security teams want control. The goal is to design a pipeline that satisfies both.

This article explores how to achieve that balance using GitHub as the foundation. We’ll look at the philosophy behind shift left, the practical steps to embed security into CI/CD, and the strategies that keep your pipeline fast while making it secure.

Why Shift Left Matters

Traditional security models treated security as a gatekeeper. Code would flow through development and testing, and only at the end, right before deployment, would security teams step in. This approach worked when release cycles were measured in months. It doesn’t work in a world of continuous delivery.

Late-stage security checks create bottlenecks. They force developers to rework code they wrote weeks ago, slowing releases and creating friction between teams. Worse, they allow vulnerabilities to linger until the last possible moment, increasing the risk of exposure.

Shift left flips the model. Instead of waiting until the end, security becomes part of the development process. Vulnerability scans run on every pull request. Secrets are checked before they hit the repository. Infrastructure-as-code is validated before provisioning resources. The result is fewer surprises, faster remediation, and a culture where security is everyone’s responsibility.

The Fear of Slowing Down

If shift left is so effective, why do some teams resist it? The answer is simple: performance anxiety. Developers worry that adding security checks will make pipelines sluggish. Security teams worry that developers will bypass controls to keep things moving.

The truth is, poorly implemented security can slow things down. If scans take 30 minutes to run or generate endless false positives, developers will see security as an obstacle, not an enabler. That’s why pipeline design matters. The goal isn’t just to add security, it’s to integrate it intelligently so it complements speed rather than killing it.

Designing a DevSecOps Pipeline on GitHub

GitHub provides a rich ecosystem for building secure pipelines without sacrificing agility. At the heart of this is GitHub Actions, which allows you to automate workflows triggered by events like pushes, pull requests, or scheduled intervals.

A well-designed pipeline starts with a clear separation of concerns. Security checks should run where they make sense, and they should run in parallel whenever possible. For example, static analysis can run alongside unit tests, while dependency checks can execute independently of build steps.

The key is modularity. Instead of one monolithic workflow that does everything, break your pipeline into smaller jobs. Each job handles a specific responsibility (build, test, scan) and runs concurrently. This approach minimizes bottlenecks and makes troubleshooting easier.

Embedding Security Without Friction

The first step is to identify which security controls belong in the pipeline. At a minimum, you want static analysis, secret scanning, and dependency checks. These are lightweight and can run quickly on every pull request.

Static analysis tools like CodeQL examine source code for vulnerabilities without executing it. They’re ideal for catching issues early, and when configured properly, they add only a few minutes to the pipeline. Secret scanning prevents accidental exposure of credentials, and GitHub provides this natively. Dependency checks, powered by tools like Dependabot, ensure that third-party libraries remain secure.

For heavier scans, like container image analysis or infrastructure compliance, you can schedule them to run nightly or on merge to main. This keeps pull request workflows lean while still providing comprehensive coverage.

Parallelization and Caching: The Unsung Heroes

One of the easiest ways to keep pipelines fast is to run jobs in parallel. GitHub Actions supports matrix builds, which allow you to test across multiple environments simultaneously. This is particularly useful for security because vulnerabilities can be environment-specific.

Caching is another performance booster. Many security tools rely on large databases of vulnerability signatures. By caching these between runs, you avoid downloading them every time, shaving minutes off your workflow.

Handling False Positives

Nothing kills developer trust faster than noisy security alerts. If every pull request triggers a dozen false positives, developers will tune out. The solution is tuning. Configure your tools to focus on high-severity issues and suppress rules that don’t apply to your codebase.

It’s also important to provide actionable feedback. A vague “security issue detected” message isn’t helpful. Developers need context about what’s wrong, why it matters, and how to fix it. GitHub’s integration with CodeQL and other tools makes this possible by surfacing detailed findings directly in pull requests.

Culture Is the Glue

Technology alone won’t make shift left successful. You need a culture that values security as much as speed. That means involving developers in the process, explaining why controls exist, and celebrating wins when vulnerabilities are caught early.

Security champions (developers who advocate for best practices) can help bridge the gap between teams. Training sessions, documentation, and clear communication go a long way toward making security feel like a shared goal rather than an imposed burden.

A Sample Pipeline Design

Imagine a pipeline that runs on every pull request. It starts by checking out the code and running unit tests. In parallel, it launches three security jobs: static analysis with CodeQL, secret scanning, and dependency checks. Each job runs independently, and the workflow is configured to fail fast if a critical vulnerability is found.

On merge to main, the pipeline triggers additional jobs: container image scanning with Trivy and infrastructure compliance checks with Checkov. These heavier scans run asynchronously, so they don’t block developers waiting for feedback on their pull requests.

The result is a pipeline that enforces security without slowing development. Developers get quick feedback on critical issues, and security teams get the assurance that controls are in place.

You can find some examples below

Pull Request workflow - fast feedback, parallel security

File: '.github/workflows/pr-pipeline.yml'

{% raw %}

name: PR Pipeline (Fast Feedback)

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
    branches: [main]
  workflow_dispatch:

# Prevent redundant runs on the same PR head sha
concurrency:
  group: pr-${{ github.ref }}-${{ github.head_ref }}
  cancel-in-progress: true

permissions:
  contents: read
  actions: read
  security-events: write   # for CodeQL to upload SARIF
  pull-requests: write     # to annotate PRs with findings
  id-token: write          # optional: for OIDC to cloud scanners (if needed)

env:
  NODE_VERSION: '20'
  # Example registry mirror settings (adjust to your org)
  # NPM_REGISTRY: 'https://registry.npmjs.org'

jobs:
  build_and_test:
    name: Build & Unit Tests (matrix)
    runs-on: ubuntu-latest
    timeout-minutes: 20
    strategy:
      fail-fast: true
      matrix:
        node: [18, 20]
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js ${{ matrix.node }}
        uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: 'npm'

      - name: Install deps
        run: npm ci

      - name: Unit tests
        run: npm test -- --ci --reporter=junit
      # Optionally upload coverage/test reports to your system

  codeql:
    name: Static Analysis (CodeQL)
    runs-on: ubuntu-latest
    timeout-minutes: 25
    permissions:
      contents: read
      security-events: write
      actions: read
    steps:
      - uses: actions/checkout@v4

      - name: Initialize CodeQL
        uses: github/codeql-action/init@v3
        with:
          languages: javascript # add more e.g., javascript,python,java,go,cpp,csharp
          queries: +security-and-quality

      - name: Autobuild
        uses: github/codeql-action/autobuild@v3

      - name: Perform CodeQL Analysis
        uses: github/codeql-action/analyze@v3
        with:
          category: '/language:javascript'

  dependency_review:
    name: Dependency Checks (PR Diff)
    runs-on: ubuntu-latest
    timeout-minutes: 10
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - name: Dependency Review
        uses: actions/dependency-review-action@v4
        with:
          fail-on-severity: critical
          comment-summary-in-pr: true

  secrets_scan:
    name: Secret Scanning (Push Protection Guide)
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      # Native GitHub Secret Scanning runs automatically on Advanced Security-enabled repos.
      # This step enforces a quick pre-commit/PR check with gitleaks as a complement (optional).
      - name: Run Gitleaks
        uses: zricethezav/gitleaks-action@v2
        with:
          args: detect --no-git -v --redact
      # Note: enable "Push Protection" in repo/org settings to block secrets before they land.

  # Optional: run lightweight container scan on PRs, keep it fast
  trivy_pr:
    name: Container Scan (Trivy)
    runs-on: ubuntu-latest
    timeout-minutes: 10
    needs: [build_and_test]
    steps:
      - uses: actions/checkout@v4

      - name: Build app image (local)
        run: |
          docker build -t app:${{ github.sha }} .

      - name: Cache Trivy DB
        uses: actions/cache@v4
        with:
          path: ~/.cache/trivy
          key: trivy-db-${{ runner.os }}-${{ hashFiles('**/Dockerfile') }}
          restore-keys: |
            trivy-db-${{ runner.os }}-

      - name: Scan image with Trivy (critical only)
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: app:${{ github.sha }}
          severity: CRITICAL,HIGH
          exit-code: '1'
          ignore-unfixed: true
          vuln-type: 'os,library'

  # Keep IaC checks in PR but quick
  checkov_pr:
    name: IaC Compliance (Checkov)
    runs-on: ubuntu-latest
    timeout-minutes: 8
    steps:
      - uses: actions/checkout@v4
      - name: Run Checkov
        uses: bridgecrewio/checkov-action@v12
        with:
          directory: .
          quiet: true
          soft_fail: false
          framework: terraform,kubernetes,cloudformation,arm

  # Gate: if any critical job fails, whole PR is blocked (default behavior)

{% endraw %}

Why this works for speed + security

Jobs run in parallel (build/tests, CodeQL, dependency review, secrets, light Trivy, Checkov).
Matrix ensures cross-version coverage without serial runs.
Caching speeds Trivy DB and Node modules.
Fail on severity and exit codes keep signal strong and avoid noisy false positives.

Main branch workflow - heavier scans on merge

File: '.github/workflows/main-security.yml'

{% raw %}

name: Main Branch Security (Heavier Coverage)

on:
  push:
    branches: [main]
  schedule:
    - cron: "17 2 * * *"   # nightly deeper scan (UTC)
  workflow_dispatch:

concurrency:
  group: main-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read
  security-events: write
  actions: read
  id-token: write

jobs:
  build_release_artifacts:
    name: Build Release Artifacts
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run build
      - name: Archive build
        uses: actions/upload-artifact@v4
        with:
          name: app-build
          path: dist/

  trivy_image_scan:
    name: Container Image Scan (Trivy - full)
    runs-on: ubuntu-latest
    needs: build_release_artifacts
    steps:
      - uses: actions/checkout@v4

      - name: Build production image
        run: |
          docker build -t app:release .

      - name: Cache Trivy DB
        uses: actions/cache@v4
        with:
          path: ~/.cache/trivy
          key: trivy-db-${{ runner.os }}-${{ github.sha }}
          restore-keys: |
            trivy-db-${{ runner.os }}-

      - name: Trivy scan (fail on High/Critical)
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: app:release
          severity: CRITICAL,HIGH
          exit-code: '1'
          ignore-unfixed: false
          format: 'sarif'
          output: 'trivy-results.sarif'

      - name: Upload SARIF to code scanning
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: trivy-results.sarif

  checkov_full:
    name: IaC Compliance (Checkov - full)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Checkov (report + fail on high)
        uses: bridgecrewio/checkov-action@v12
        with:
          directory: .
          quiet: true
          soft_fail: false
          skip_check: CKV_SECRET_1  # example of tuning; adjust to your baseline
      - name: Upload Checkov results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: checkov-report
          path: results_json/*.json

{% endraw %}

Optional: Reusable workflow for org-wide consistency

If you manage many repos, create a reusable workflow and call it from each repo.

File: '.github/workflows/reusable-security.yml'

{% raw %}

name: Reusable Security
on:
  workflow_call:
    inputs:
      languages:
        required: false
        type: string
        default: 'javascript'
    secrets:
      token:
        required: false

jobs:
  codeql:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: github/codeql-action/init@v3
        with:
          languages: ${{ inputs.languages }}
          queries: +security-and-quality
      - uses: github/codeql-action/autobuild@v3
      - uses: github/codeql-action/analyze@v3

{% endraw %}

Then invoke it:

{% raw %}

jobs:
  security:
    uses: your-org/your-repo/.github/workflows/reusable-security.yml@main
    with:
      languages: 'javascript,python'

{% endraw %}

Additional settings that will provide more options for protection and performance:

Push Protection & Secret Scanning: Enable at the org/repo level to block secrets before they land; use a lightweight PR scanner as a safety net.
Tuning & Noise Reduction: Set 'fail-on-severity', 'ignore-unfixed', and 'skip_check' to align with your baseline; revisit quarterly.
Parallelization: Keep PR feedback fast by running security jobs concurrently and shifting heavier scans to 'push'/'schedule'.
Least Privilege: Use minimal 'permissions' and OIDC ('id-token') for cloud scanners instead of long‑lived secrets.

Looking Ahead

Shift left isn’t a one-time project. It’s an ongoing evolution. As threats change and tools improve, your pipeline will need to adapt. GitHub is investing heavily in security features like push protection, which blocks commits containing secrets before they even hit the repository. Expect more automation, better integrations, and smarter alerts in the future.

The goal is simple: make security invisible. When developers don’t have to think about it (because it’s baked into their workflows) you’ve achieved true DevSecOps.

Final Thoughts

Balancing speed and security isn’t easy, but it’s possible. By designing pipelines that integrate security intelligently, you can shift left without slowing down. Start small, iterate often, and keep the conversation open between development and security teams.

In the end, the fastest pipeline isn’t the one that skips security. It’s the one that makes security seamless.

Need help shifting left? Contact me!

steve.kaschimer@slalom.com

CodeQL Deep Dive: Static Analysis for DevSecOps Engineers

2025-11-24T00:00:00Z

Modern software development moves at breakneck speed. Continuous integration and continuous delivery (CI/CD) pipelines have transformed how teams build and ship applications, enabling rapid iteration and frequent releases. But with this velocity comes risk. Vulnerabilities can slip through unnoticed, and if they make it into production, the cost of remediation skyrockets, not just in dollars, but in reputation and trust.

This is where static analysis becomes indispensable. Among the tools available today, CodeQL stands out as a game-changer for DevSecOps engineers. It’s not just another scanner; it’s a query engine for your code. CodeQL allows you to treat your codebase like a database, asking sophisticated questions about patterns, flows, and behaviors that might indicate security flaws. In this deep dive, we’ll explore what makes CodeQL unique, how it works under the hood, how you can customize it to fit your organization’s needs, and how to integrate it seamlessly into your workflows.

By the end of this article, you’ll understand why CodeQL is more than a tool. It’s a mindset shift for secure development.

What Is CodeQL and Why Does It Matter?

CodeQL is GitHub’s semantic code analysis engine. Unlike traditional static analysis tools that rely on predefined rules and pattern matching, CodeQL converts your source code into a relational database. Every function, variable, class, and dependency becomes part of a structured schema. This means you can write queries to search for vulnerabilities, design flaws, or even coding style violations, similar to how you write queries for SQL.

Why is this approach powerful? Because vulnerabilities often share structural similarities. For example, SQL injection vulnerabilities typically involve unsanitized user input flowing into a database query. With CodeQL, you can express this concept as a query and apply it across your entire codebase. Instead of scanning for hardcoded patterns, you’re analyzing relationships and data flows, which makes detection far more accurate and adaptable.

For DevSecOps engineers, this flexibility is gold. It allows you to go beyond generic checks and tailor security analysis to your application’s architecture, coding standards, and threat model.

How CodeQL Works Behind the Scenes

To appreciate CodeQL’s capabilities, it helps to understand its workflow. When you run CodeQL, three major steps occur:

Step 1: Code Extraction CodeQL parses your source code and builds a database that represents the code’s abstract syntax tree (AST), control flow, and data flow. This database is language-specific, and CodeQL supports a wide range of languages including JavaScript, Python, Java, Go, C#, and C/C++.

Step 2: Query Execution Queries are written in CodeQL’s own language, which borrows concepts from logic programming and relational algebra. These queries operate on the database created in Step 1. For example, you might write a query to find all functions that concatenate user input into SQL statements without sanitization.

Step 3: Results and Reporting The results of these queries are returned in SARIF (Static Analysis Results Interchange Format), which integrates seamlessly with GitHub’s code scanning alerts. This means developers see actionable findings directly in their pull requests, complete with explanations and remediation guidance.

This architecture makes CodeQL incredibly versatile. You’re not limited to the queries GitHub provides. You can write your own, combine them, and even share them across teams.

The Query Language: Your Superpower

At the heart of CodeQL is its query language. If you’ve ever written SQL, you’ll feel at home, but CodeQL is designed for code analysis, not relational data. A typical query consists of:

Imports: Specify the language libraries you need (e.g., import javascript).
Predicates: Define conditions that match certain code elements.
Select statements: Determine what results to return and how to annotate them.

Here’s a simple example that detects hardcoded AWS access keys in JavaScript:

import javascript

from Literal l
where l.getValue().matches("AKIA[0-9A-Z]{16}")
select l, "Possible AWS Access Key detected."

This query imports the JavaScript library, iterates over all literals, and flags any that match the regex for AWS keys. It’s concise, expressive, and easy to adapt.

But CodeQL can do much more. You can write queries that track data flow across functions, identify tainted inputs, and detect complex vulnerability patterns. For instance, finding SQL injection risks involves tracing user input from its source to a sink (e.g., a database call) without proper sanitization. CodeQL’s libraries provide built-in predicates for common sources and sinks, making these queries easier to write.

Customizing Queries for Your Organization

Out-of-the-box, CodeQL includes thousands of queries covering common vulnerabilities and best practices. But every organization has unique requirements. Maybe you have internal APIs that require special handling, or coding standards that go beyond what generic queries enforce. Customization is where CodeQL shines.

You can:

Extend existing queries by adding conditions or exceptions.
Write new queries for project-specific risks.
Suppress false positives by refining predicates.

For example, suppose your team uses a custom sanitization function called sanitizeInput. You can modify the standard SQL injection query to treat calls to this function as safe. This reduces noise and builds developer trust.

Testing custom queries is straightforward with the CodeQL CLI. You can run queries locally against your codebase, iterate quickly, and then integrate them into your CI/CD pipeline once validated.

Integrating CodeQL into Your Workflows

Static analysis is most effective when it’s automated and continuous. GitHub Actions makes CodeQL integration seamless. Here’s a sample workflow you can use:

name: CodeQL Analysis

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * 0'

jobs:
  analyze:
    runs-on: ubuntu-latest
    permissions:
      actions: read
      contents: read
      security-events: write
    steps:
      - uses: actions/checkout@v4
      - uses: github/codeql-action/init@v3
        with:
          languages: javascript,python
      - uses: github/codeql-action/autobuild@v3
      - uses: github/codeql-action/analyze@v3

This workflow runs CodeQL on every push and pull request to main, plus a scheduled weekly scan. It initializes CodeQL, builds the project, and analyzes the code. Results appear in GitHub’s Security tab and as annotations in pull requests.

For larger projects, consider splitting workflows into modular jobs and using caching to speed up builds. You can also configure fail-on-severity thresholds to block merges when critical vulnerabilities are detected.

Best Practices for CodeQL Adoption

Integrating CodeQL is just the beginning. To maximize its value:

Run scans early and often. Pull request analysis provides fast feedback and prevents vulnerabilities from entering the main branch.
Tune queries to reduce false positives. Developer trust is essential because noisy alerts lead to alert fatigue.
Combine CodeQL with other security checks like secret scanning and dependency review for layered defense.
Educate developers on interpreting CodeQL findings. The more they understand the “why” behind alerts, the more likely they are to fix issues promptly.

Advanced Use Cases

CodeQL isn’t limited to security. You can use it for:

Code quality enforcement: Detect anti-patterns or deprecated APIs.
Compliance checks: Ensure code adheres to regulatory requirements.
Architecture analysis: Identify cyclic dependencies or excessive coupling.

These use cases make CodeQL a versatile tool for both security and engineering excellence.

The Future of CodeQL

GitHub continues to invest heavily in CodeQL. Expect improvements in query packs, language support, and performance. Features like push protection and deeper integration with GitHub Advanced Security will make secure development even more frictionless.

For DevSecOps engineers, mastering CodeQL is a career-defining skill. It empowers you to move beyond reactive scanning and embrace proactive, intelligent security.

Final Thoughts

Static analysis is no longer optional. It’s a necessity in modern software delivery. CodeQL offers a unique approach that combines precision, flexibility, and automation. By understanding how it works, customizing queries, and integrating it into your workflows, you can elevate your security posture without sacrificing speed.

Start small. Enable CodeQL on a critical repository, experiment with queries, and iterate. Over time, you’ll build a library of custom checks that reflect your organization’s priorities. And as you do, you’ll transform security from a bottleneck into a seamless part of development.

Need help getting your CodeQL just right? Contact me!

steve.kaschimer@slalom.com

DevOps Culture: What It Is, Why It Exists, and Why It Matters

2025-12-01T00:00:00Z

DevOps has become one of the most talked-about concepts in modern software delivery. It’s often associated with automation tools, CI/CD pipelines, and cloud-native architectures. But the truth is, DevOps isn’t primarily about technology. It’s about culture. Without cultural transformation, even the most advanced tools will fail to deliver the promised benefits.

So, what exactly is DevOps culture? Why did it emerge? Why should organizations care? And perhaps most importantly, how do we build it? This article dives deep into these questions, drawing on real-world examples and lessons learned from enterprise transformations, including insights from projects.

What Is DevOps Culture?

DevOps culture is more than a set of practices. It’s a mindset that transforms how organizations build and deliver software. At its core, DevOps culture breaks down silos between development, operations, and security teams, fostering collaboration and shared responsibility across the entire software delivery lifecycle. Instead of developers writing code and tossing it over the wall to operations, DevOps encourages everyone involved, including developers, testers, security engineers, and operations, to work toward a common goal: delivering reliable, secure software quickly and efficiently.

To understand DevOps culture, it helps to look at the Three Ways described in The Phoenix Project, which serve as guiding principles for high-performing technology organizations:

The First Way: Flow

Flow is about creating a fast, smooth movement of work from development to operations and ultimately to the customer. It emphasizes systems thinking, or viewing the entire value stream as one continuous system rather than isolated silos. Practices like reducing batch sizes, limiting work in progress, and eliminating bottlenecks help accelerate delivery while improving quality. In a DevOps culture, flow ensures that ideas move quickly from concept to production without unnecessary friction.

The Second Way: Feedback

Feedback is the lifeblood of continuous improvement. The Second Way focuses on shortening and amplifying feedback loops so problems are detected and corrected early. Automated testing, continuous integration, proactive monitoring, and regular retrospectives create a two-way exchange of insights between development and operations. This principle reinforces shared responsibility and helps teams learn from each other, preventing defects from cascading downstream.

The Third Way: Continuous Learning and Experimentation

The Third Way promotes a culture of continual learning and innovation. It encourages teams to take calculated risks, experiment, and learn from failures without fear of blame. Practices like blameless post-mortems, dedicated time for experimentation, and open knowledge sharing make improvement part of everyday work. This principle ensures that organizations adapt quickly to change and continuously evolve their capabilities.

Together, these Three Ways form the backbone of DevOps culture. They shift the focus from isolated tasks to holistic outcomes, from rigid processes to adaptive learning, and from siloed accountability to shared ownership. When these principles are embraced, DevOps becomes more than a methodology. It becomes a cultural movement that drives speed, quality, and resilience.

Why Does DevOps Culture Exist?

The roots of DevOps culture lie in the shortcomings of traditional software development models. Waterfall methodologies, with their rigid phases and long release cycles, were ill-suited for a world where customer expectations change overnight. Agile development addressed part of the problem by speeding up coding and testing, but it often left operations behind. The result? Faster development paired with slow, painful deployments.

The 2009 "10+ Deploys Per Day" talk by John Allspaw and Paul Hammond at the Velocity conference is widely considered the spark that ignited the DevOps movement. At the time, Flickr was deploying code to production more than 10 times per day, which was revolutionary when most companies were doing quarterly or monthly releases. The talk challenged the conventional wisdom that development and operations had inherently conflicting goals. Instead of accepting the "wall of confusion" between Dev (who wanted to move fast and ship features) and Ops (who wanted stability and minimal change), Allspaw and Hammond demonstrated how their teams collaborated through shared tools, shared metrics, and shared responsibility. They showed that with the right culture and automation, velocity and stability weren't trade-offs, but rather they reinforced each other.

The key insight was that deploying frequently actually reduces risk because each change is smaller, easier to test, and faster to roll back if needed. Their approach included automated testing, one-step builds and deploys, feature flags for safer releases, shared metrics visible to everyone, and most importantly, a culture of mutual respect and trust between developers and operations. The talk resonated so deeply because it offered a practical alternative to the status quo, proving that cross-functional collaboration, automation, and continuous delivery weren't just theoretical ideals. They were achievable realities. This presentation became the blueprint for what would soon be formalized as the DevOps movement, influencing countless organizations to rethink how they deliver software.

DevOps emerged as the bridge between Agile and operational excellence. It extended the principles of iteration and feedback beyond coding to include deployment, monitoring, and incident response. Organizations realized that speed without stability was a recipe for disaster. DevOps culture exists to align innovation with reliability, enabling teams to deliver value continuously without sacrificing quality.

Why Should We Care About DevOps Culture?

Culture drives behavior, and behavior drives outcomes. You can implement every automation tool on the market, but if your teams don’t collaborate, share responsibility, and embrace continuous improvement, you’ll never achieve true DevOps maturity.

DevOps culture matters because it impacts every metric that matters to the business:

Time-to-market: Faster releases mean quicker response to customer needs.
Quality: Shared responsibility reduces defects and improves reliability.
Employee engagement: Teams that collaborate and learn together are more motivated.
Business value: Efficient delivery translates to competitive advantage and profitability.

How Do We Get There?

Building a DevOps culture isn't about buying a tool or adopting a framework. It's about changing mindsets and behaviors through deliberate practices, organizational design, and measured progress. Here's a comprehensive roadmap for cultural transformation:

Specific Practices That Enable DevOps Culture

Infrastructure as Code (IaC)

Infrastructure as Code treats infrastructure provisioning like software development, with version control, code reviews, and automated testing. Instead of manually configuring servers through GUI consoles or ad-hoc scripts, teams define infrastructure declaratively in files that can be reviewed, tested, and deployed consistently.

For example, using Terraform, you might define an Azure Kubernetes Service cluster like this:

resource "azurerm_kubernetes_cluster" "main" {
  name                = "prod-aks-cluster"
  location            = azurerm_resource_group.main.location
  resource_group_name = azurerm_resource_group.main.name
  dns_prefix          = "prodaks"
  
  default_node_pool {
    name       = "default"
    node_count = 3
    vm_size    = "Standard_D2_v2"
  }
  
  identity {
    type = "SystemAssigned"
  }
}

This approach makes infrastructure changes transparent, auditable, and repeatable. When operations engineers and developers collaborate on IaC, they build shared understanding of both application and infrastructure requirements. Code reviews become opportunities for knowledge transfer. Automated testing catches configuration drift before it reaches production.

Shift-Left Security Practices

Shift-left security means integrating security checks early in the development pipeline rather than treating security as a gate before production. This includes static application security testing (SAST) in CI pipelines, dependency scanning for vulnerable packages, container image scanning, and infrastructure security validation.

For instance, integrating GitHub Advanced Security into your CI/CD pipeline automatically scans for secrets, detects vulnerable dependencies, and runs CodeQL queries on every pull request. Developers get immediate feedback about security issues when the fix is cheapest and easiest. Security teams define policies as code, like "no critical vulnerabilities in production" or "all secrets must be stored in Azure Key Vault," and automation enforces them consistently.

The cultural shift here is critical: security isn't something done to developers; it's something done with them. Security engineers become enablers rather than gatekeepers, providing tools, training, and guardrails that help developers ship secure code confidently.

Observability and Monitoring Strategies

Observability goes beyond traditional monitoring. While monitoring tells you what is wrong (CPU usage is high, error rate increased), observability helps you understand why by providing insights into system behavior through logs, metrics, traces, and events.

A mature observability strategy includes:

Structured logging with correlation IDs to trace requests across distributed systems
Distributed tracing to visualize request flows and identify bottlenecks (using tools like Jaeger or Azure Application Insights)
Metrics dashboards that show business KPIs alongside technical metrics
Proactive alerting based on SLOs (Service Level Objectives) rather than arbitrary thresholds
Blameless postmortems that treat incidents as learning opportunities

When developers have access to production metrics and logs, they understand how their code performs in the real world. When operations teams understand application architecture and business context, they can prioritize incidents effectively. Shared observability creates shared responsibility.

ChatOps and Communication Patterns

ChatOps brings operational work into chat platforms like Slack or Microsoft Teams, making actions transparent and collaborative. Instead of operations engineers deploying code through opaque terminal sessions, deployments happen via chat commands visible to the entire team.

For example, a deployment might look like: /deploy api-service v2.3.1 to production executed in a Slack channel. The bot responds with deployment status, runs automated tests, and notifies the team when complete. If issues arise, the entire team sees the context and can collaborate on resolution in the same thread.

This transparency breaks down information silos. Junior engineers learn by observing how seniors troubleshoot issues. Product managers understand operational challenges. Security teams can audit actions without requesting logs. ChatOps doesn't just automate tasks; it democratizes knowledge.

Team Topologies: Organizing for Flow

DevOps culture requires deliberate organizational design. The book Team Topologies by Matthew Skelton and Manuel Pais provides a framework for structuring teams to optimize flow and minimize cognitive load.

Stream-Aligned Teams

These are cross-functional product teams aligned to a single value stream (a product, service, or user journey). A stream-aligned team includes developers, testers, operations expertise, and sometimes designers or data analysts. They own their service end-to-end, from code to production. For example, a "Checkout Service Team" owns everything related to the checkout experience: backend APIs, frontend components, database schemas, infrastructure, and monitoring.

This structure eliminates handoffs and waiting. The team can move quickly because they don't depend on separate operations or QA teams to progress. They feel ownership because they're accountable for outcomes, not just outputs.

Platform Teams

Platform teams build internal products that reduce cognitive load for stream-aligned teams. They provide self-service capabilities like CI/CD pipelines, infrastructure templates, observability tooling, and developer portals. A good platform team treats other engineering teams as customers, focusing on developer experience and ease of use.

For instance, a platform team might create a "golden path" deployment pipeline where stream-aligned teams can deploy containerized applications to Kubernetes with a single YAML file, while the platform handles secrets management, network policies, monitoring setup, and compliance checks automatically.

Enabling Teams

Enabling teams help stream-aligned teams adopt new technologies and practices. They're specialists (security engineers, SREs, data engineers) who embed temporarily with product teams to transfer knowledge. Unlike traditional centralized teams that do work for others, enabling teams work with others to build capability.

For example, an enabling team might help a product team adopt observability practices by pairing on instrumentation code, explaining tracing concepts, and setting up dashboards. After a few weeks, the product team has the skills to continue independently.

Complicated Subsystem Teams

These teams handle complex technical domains that require specialized expertise, like machine learning models, payment processing, or compliance engines. They provide services to stream-aligned teams through well-defined APIs.

The key principle is team interaction modes: collaboration (working together), X-as-a-Service (consuming through APIs), and facilitation (helping others learn). Clear interaction modes prevent teams from stepping on each other's toes and reduce cognitive overload.

Transformation Roadmap: From Assessment to Optimization

DevOps transformation isn't a big-bang change. It's a phased journey that respects organizational constraints while driving continuous improvement.

Phase 1: Assessment (2-4 weeks)

Start by understanding your current state. Conduct interviews with developers, operations, security, and business stakeholders. Map your value streams: how does code move from idea to production? Identify bottlenecks, waste, and cultural friction points.

Measure baseline metrics: How often do you deploy? What's your lead time from commit to production? What percentage of deployments cause incidents? How long does it take to recover from failures? These become your benchmarks for improvement.

Assess organizational readiness. Who are your potential champions? What's leadership's appetite for change? What constraints (regulatory, technical, political) will you face? Create a stakeholder map and change management strategy.

Phase 2: Pilot (3-6 months)

Select one stream-aligned team (ideally working on a non-critical but meaningful product) to pilot DevOps practices. This team becomes your laboratory for experimentation and your showcase for success.

Provide this team with support: automation tools, training, time to refactor, and executive air cover to take calculated risks. Help them implement continuous integration, automated testing, and deployment automation. Introduce infrastructure as code. Set up observability. Establish metrics dashboards.

Document everything: what worked, what didn't, and what you learned. Run retrospectives. Share progress through demos and internal blog posts. The goal is to build a proven model and create advocates who can help spread practices to other teams.

Phase 3: Scale (6-18 months)

With a successful pilot, begin scaling practices across the organization. This isn't about mandating tools; it's about sharing patterns, providing platforms, and building momentum.

Form a platform team to codify lessons learned from the pilot into reusable services. Create documentation, runbooks, and training materials. Establish communities of practice where practitioners share knowledge. Identify and empower champions in each department.

Roll out changes incrementally. Start with teams that are ready and willing. Let success stories drive adoption. Provide enabling team support to teams that need extra help. Measure progress against DORA metrics and celebrate improvements publicly.

Phase 4: Optimize (Ongoing)

DevOps transformation never "finishes." Optimization is continuous. Regularly revisit metrics and identify new bottlenecks. Experiment with advanced practices like chaos engineering, feature flags, and progressive delivery.

Invest in organizational learning. Run internal conferences. Encourage teams to attend external conferences and bring back ideas. Create time and space for innovation. Most importantly, maintain the cultural practices that got you here: blameless postmortems, cross-functional collaboration, and psychological safety.

Change Management Tactics: Building Momentum

Cultural change is hard because it threatens the status quo. People fear losing status, competence, or control. Here's how to overcome resistance:

Start with Why

Connect DevOps transformation to business outcomes people care about. For executives, emphasize competitive advantage and faster time-to-market. For engineers, highlight reduced toil and more interesting work. For operations, emphasize stability through automation and reduced burnout. Make the case compelling and personal.

Build Champions

Identify influential people at every level who believe in the vision. These aren't necessarily managers. They're people others trust and respect. Empower them with resources, training, and visibility. Let them tell the story authentically.

Create Quick Wins

People need to see progress quickly. Choose visible pain points with achievable solutions. Automate a painful manual process. Reduce deployment time from hours to minutes. Fix a longstanding monitoring gap. Document the improvement and share it widely. Small wins build confidence that larger changes are possible.

Provide Psychological Safety

Fear kills transformation. If people are punished for failures or blamed for outages, they'll stick to safe, slow processes. Leaders must model vulnerability, admit their own mistakes, and celebrate learning from failures. Make it safe to experiment, to ask questions, and to challenge assumptions.

Make the Transition Easy

Reduce friction wherever possible. Provide training before expecting new skills. Offer pairing and mentoring. Create clear documentation. Build self-service tools. Don't expect people to figure it out alone.

Metrics That Matter: DORA Metrics Explained

The DevOps Research and Assessment (DORA) team identified four key metrics that distinguish elite performers from low performers. These metrics should guide your transformation:

Deployment Frequency

How often does your organization deploy code to production? Elite teams deploy multiple times per day. Low performers deploy monthly or less. Higher deployment frequency indicates that your teams can deliver value quickly and respond rapidly to feedback.

To improve deployment frequency, reduce batch sizes (smaller pull requests, feature flags), automate testing and deployment, and eliminate manual approval gates that don't add value.

Lead Time for Changes

How long does it take for a commit to reach production? Elite teams measure lead time in hours. Low performers measure it in months. Short lead times mean faster feedback cycles and reduced risk per deployment.

To improve lead time, identify and eliminate bottlenecks in your delivery pipeline. Common culprits include slow test suites, manual handoffs, and infrequent merge cycles. Visualize your value stream and optimize the slowest steps.

Mean Time to Recovery (MTTR)

When incidents occur, how quickly can you restore service? Elite teams recover in under an hour. Low performers take more than a week. Fast recovery requires excellent observability, practiced incident response, and the ability to roll back or roll forward quickly.

To improve MTTR, invest in monitoring and alerting, practice incident response through game days, automate rollback procedures, and conduct blameless postmortems that focus on system improvements rather than individual blame.

Change Failure Rate

What percentage of deployments cause production incidents? Elite teams have change failure rates under 15%. Low performers are above 45%. Lower change failure rates indicate better quality practices and effective feedback loops.

To improve change failure rate, strengthen automated testing (unit, integration, contract, and end-to-end tests), implement progressive delivery techniques (canary deployments, blue-green deployments), and use feature flags to decouple deployment from release.

These four metrics provide a balanced view of software delivery performance. Track them visibly, review them regularly, and use them to guide improvement experiments. But remember: metrics are means to an end, not the end itself. The goal is better outcomes for customers and teams, not just better numbers.

This is a lot of information to digest. Just remember,

Start with collaboration. Encourage developers and operations to work together from the beginning of a project. Create cross-functional teams that share ownership of outcomes. This was a key takeaway from the a recent project, where teams learned to align backlog management with deployment strategies, reducing friction between roles.

Invest in automation, but pair it with process improvement. Automate repetitive tasks like builds, tests, and deployments to free up time for innovation. Use metrics and monitoring to create feedback loops that inform decisions and drive continuous improvement.

Most importantly, lead by example. Culture change starts at the top. Leaders must champion collaboration, transparency, and learning. Celebrate successes, learn from failures, and make DevOps a shared responsibility across the organization.

What Are the Benefits?

The benefits of DevOps culture are well-documented and measurable. Organizations that embrace it see:

Faster delivery cycles.
Improved software quality.
Greater agility in responding to market changes.
Higher employee satisfaction.
Increased ROI through efficiency and innovation.
Increased Customer satisfaction
Accelereated innovation

Companies that adopt DevOps practices report significant reductions in lead time, deployment frequency, and mean time to recovery. They also experience fewer failures and faster resolution when issues occur. These aren’t just numbers, they represent real competitive advantage.

What Are the Downsides?

DevOps culture isn’t a silver bullet. It requires investment in tools, training, and time. It can be challenging to overcome resistance to change, especially in organizations with entrenched silos. There’s also a risk of burnout if teams interpret “continuous delivery” as “never stop working.”

Another downside is the complexity of scaling DevOps across large enterprises. Aligning multiple teams, standardizing processes, and maintaining governance without stifling agility can be difficult. But these challenges are surmountable with the right strategy and leadership commitment.

We also often see several common anti-patterns emerge when introducing a DevOps culture to an organization:

The "DevOps Team" Anti-Pattern Organizations create a separate "DevOps team" that sits between development and operations, essentially adding another silo instead of breaking them down. This team becomes a new bottleneck, handling deployments and infrastructure requests while developers and ops remain isolated. Real DevOps means cross-functional collaboration, not a new middle layer.
Rebrand Without Reform The operations team gets renamed to "DevOps Engineers" or "Site Reliability Engineers," but nothing actually changes. They still work in isolation, receive work via tickets, and maintain the same adversarial relationship with developers. It's a cosmetic change that preserves the old culture while claiming transformation.
Automation Without Collaboration Teams invest heavily in CI/CD pipelines, infrastructure as code, and monitoring tools, but developers and operations still don't talk to each other. Automated deployments fail because ops wasn't consulted on infrastructure requirements. Alerts fire constantly because developers don't understand operational concerns. Tools don't fix broken relationships.
"You Build It, You Run It" Without Support Organizations push operational responsibility to developers without providing training, access, or support. Developers get paged at 3 AM for production issues they don't know how to debug. This isn't empowerment, it's abdication. Real DevOps means shared responsibility with proper enablement.
Speed Without Safety Teams focus obsessively on deployment frequency while ignoring quality, security, and stability. They ship broken code faster, rack up technical debt, and burn out from constant firefighting. DevOps is about sustainable velocity, not just moving fast.
Metrics Theater Organizations track deployment frequency and lead time but don't use them to drive improvement. Metrics become performative checkboxes rather than feedback mechanisms. Teams game the numbers (deploying trivial changes to boost frequency) while real problems persist.
Tool Sprawl The organization adopts every trendy DevOps tool - Jenkins, GitLab CI, CircleCI, Kubernetes, Terraform, Ansible, Prometheus, Grafana, Datadog - without standardization or strategy. Teams spend more time integrating tools than delivering value. DevOps requires thoughtful tooling, not a collection of shiny objects.
Security as an Afterthought "DevOps" pipelines deploy code rapidly but security reviews still happen at the end, creating a bottleneck. DevSecOps means security is integrated from the start, meaning threat modeling in design, automated security testing in CI/CD, and security champions embedded in teams.
Agile Dev, Waterfall Ops Development teams work in two-week sprints, but operations still requires three-month lead times for infrastructure provisioning. The "agile transformation" stops at the deployment boundary. Real DevOps extends agility through the entire value stream.
Blame Culture in Disguise Despite talk of blameless postmortems, incidents still result in finger-pointing and CYA behavior. Engineers fear making changes because failures are punished. Psychological safety is lip service, not reality. DevOps requires genuine trust and learning from failures.

These anti-patterns share a common theme: focusing on superficial changes (tools, titles, processes) while avoiding the hard work of cultural transformation: building trust, breaking down silos, fostering collaboration, and creating shared responsibility.

Common Objections and How to Address Them

When proposing DevOps cultural transformation, you'll inevitably encounter resistance. Here are the most common objections and practical ways to address them:

"We're too regulated for DevOps"

Regulation doesn't prevent DevOps. In fact, heavily regulated industries like finance and healthcare have successfully adopted DevOps practices. The key is automated compliance. Infrastructure as code, automated testing, and audit trails actually make compliance easier by creating repeatable, documented processes. Organizations like Capital One and Nationwide Insurance are proof that DevOps and regulation coexist successfully. Shift your conversation from "can we?" to "how do we automate compliance checks into our pipelines?"

"Our legacy systems can't support this"

Legacy systems are a reason to adopt DevOps, not a reason to avoid it. You don't need to rewrite everything. Start by applying DevOps principles to deployment processes, monitoring, and incident response for existing systems. Use strangler fig patterns to gradually modernize while maintaining stability. Many organizations run containerized microservices alongside mainframes. The goal isn't technology replacement; it's improving how you deliver value regardless of the underlying tech stack.

"Developers don't want operational responsibilities"

This objection often stems from misunderstanding what shared responsibility means. DevOps doesn't expect developers to become sysadmins overnight. It means providing developers with self-service platforms, observability tools, and operational expertise. Embed operations engineers into development teams to transfer knowledge. Start with on-call rotations for high-severity issues only, with proper training and escalation paths. Most developers appreciate understanding how their code runs in production. It makes them better engineers.

"We don't have time for cultural change"

This is the most dangerous objection because it confuses urgency with importance. The reality is you're already paying the cost of poor culture through slow delivery, frequent outages, and low morale. Cultural transformation doesn't require stopping work. It happens incrementally. Start with small experiments: one cross-functional team, one automated deployment pipeline, one blameless postmortem. Demonstrate value quickly and build momentum. The question isn't whether you have time for change. It's whether you can afford to keep doing things the old way.

Actionable Insights from Enterprise Projects

From internal initiatives and recent project work, several lessons stand out:

Start small. Pilot DevOps practices in one team before scaling.
Focus on outcomes, not tools. Tools enable culture. They don’t create it.
Measure what matters. Track deployment frequency, lead time, and recovery time.
Invest in people. Training and communication are as important as automation.

Final Thoughts

DevOps culture is the foundation of modern software delivery. It’s what turns automation into acceleration and collaboration into innovation. Without it, tools are just tools, and processes are just paperwork.

Building this culture takes time, effort, and leadership. But the payoff (faster delivery, better quality, happier teams, and stronger business outcomes) is worth every step.

Need help building or changing culture? I can help!

steve.kaschimer@slalom.com

DevSecOps Metrics That Matter: What to Measure, How to Track It in GitHub, and Why It Matters

2025-12-08T00:00:00Z

Modern software delivery is a balancing act. Teams strive to move fast, but every shortcut can introduce risk. DevSecOps exists to resolve that tension by embedding security into development workflows without slowing innovation. Yet there’s a catch: you can’t improve what you don’t measure. Metrics are the compass that keeps your DevSecOps journey on course.

The challenge isn’t data scarcity. GitHub and other platforms generate plenty of signals. The challenge is knowing which metrics matter, how to track them effectively, and why they’re worth your attention. In this post, we’ll explore the essential DevSecOps metrics, show how to capture them using GitHub’s capabilities, and explain why these numbers should influence decisions across your organization.

Why Metrics Matter in DevSecOps

Metrics aren’t about policing teams or assigning blame. They’re about creating feedback loops that drive improvement. When developers and security teams see clear, actionable data, they can make better decisions, automate guardrails, and reduce friction. Without metrics, DevSecOps becomes a slogan rather than a practice.

The most impactful metrics align three outcomes: velocity to value, risk reduction, and operational reliability. If you measure only speed, you risk cutting corners. If you measure only security, you risk slowing delivery to a crawl. The goal is balance, that is, fast, safe, and resilient software delivery.

The Core Delivery Signals

High-performing teams track a handful of delivery metrics that reveal how efficiently and safely code moves from idea to production. These are often called DORA metrics, and they’ve become the gold standard for assessing software delivery performance.

Deployment Frequency

Frequent deployments in small batches reduce risk and accelerate feedback. In GitHub, you can track this by querying deployment events tied to protected environments.

# Deployment frequency for production environment (last 30 days)
gh api /repos/<org>/<repo>/deployments \
  -F environment=production \
  --jq '[.[] | select(.created_at > (now - 2592000 | todate))] | length'

Benchmarks: Elite performers deploy multiple times per day (on-demand deployment). High performers deploy between once per day and once per week. Medium performers deploy between once per week and once per month. Low performers deploy less than once per month.

Common Pitfalls: Counting every commit to any branch inflates your numbers without measuring actual production deployment. Measuring deployments to test or staging environments instead of production gives false signals. Including automated dependency updates or infrastructure-only changes that don't deliver user value skews the metric.

How to Improve: Reduce batch size by breaking large features into smaller, independently deployable increments. Automate the entire deployment pipeline to eliminate manual handoffs and approval gates that don't add value. Use feature flags to decouple deployment from release, allowing you to deploy code to production safely without immediately exposing it to users. Establish trunk-based development practices with short-lived branches to reduce integration complexity.

Lead Time for Changes

Shorter lead times indicate healthy pipelines and fewer bottlenecks. GitHub’s GraphQL API lets you correlate commit timestamps with pull request merges and deployment events.

{
  repository(owner: "<org>", name: "<repo>") {
    pullRequests(last: 10, states: MERGED) {
      nodes {
        title
        createdAt
        mergedAt
        commits(first: 1) {
          nodes {
            commit {
              oid
              authoredDate
            }
          }
        }
      }
    }
  }
}

Change Failure Rate

Tag deployment statuses and link them to incident issues or rollback workflows in GitHub Actions.

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Deploy
        run: ./scripts/deploy.sh
      - name: Emit Deployment Status
        if: always()
        run: |
          jq -n --arg status "${{ job.status }}" \
                --arg dt "$(date -Iseconds)" \
                '{status: $status, timestamp: $dt}' > deploy.json
      - uses: actions/upload-artifact@v4
        with:
          name: deploy-meta
          path: deploy.json

Benchmarks: Elite performers maintain a change failure rate of 0-15% (meaning 85%+ of deployments succeed without causing incidents or requiring rollback). High performers experience 16-30% failure rates. Medium performers see 31-45% failures. Low performers exceed 45% failure rates.

Common Pitfalls: Defining "failure" inconsistently across teams makes comparison meaningless. Some teams count any rollback as failure; others only count customer-impacting incidents. Excluding specific types of changes (configuration updates, database migrations, infrastructure changes) provides an artificially optimistic picture. Not tracking near-misses (issues caught in production monitoring before customer impact) misses opportunities for improvement.

How to Improve: Strengthen your automated testing strategy across the pyramid: unit tests for fast feedback on logic, integration tests for component interactions, contract tests for API compatibility, and end-to-end tests for critical user journeys. Implement progressive delivery techniques like canary deployments (route a small percentage of traffic to new versions), blue-green deployments (maintain parallel environments for instant rollback), and feature flags (enable features gradually for specific user cohorts). Establish comprehensive monitoring with Service Level Indicators (SLIs) that detect degradation before customers notice. Conduct blameless postmortems after failures to identify systemic improvements rather than individual blame.

Mean Time to Restore

GitHub issues and deployment logs provide the timestamps you need to calculate MTTR.

Benchmarks: Elite performers restore service in less than one hour. High performers recover in less than one day. Medium performers require between one day and one week. Low performers take more than one week to restore service after an incident.

Common Pitfalls: Starting the clock when someone begins working on the problem rather than when the incident actually occurred understates your true MTTR. Stopping the clock when a fix is deployed rather than when service is fully restored to customers gives false confidence. Excluding incidents that resolve themselves (transient failures, auto-scaling responses) or only counting "major" incidents creates blind spots.

How to Improve: Invest in observability to detect issues faster. Structured logging with correlation IDs, distributed tracing across services, and real-time dashboards showing business and technical metrics reduce time to detection. Practice incident response through game days and chaos engineering experiments so teams know their playbooks when real incidents occur. Automate rollback procedures so reverting to known-good states takes seconds, not hours. Reduce deployment size and complexity so understanding the blast radius of changes is straightforward. Establish clear escalation paths and on-call rotations with runbooks that guide responders through common scenarios. Most importantly, conduct blameless postmortems that focus on improving systems rather than punishing individuals. Psychological safety is essential for honest learning.

Security Metrics That Drive Action

Velocity is only half the story. DevSecOps is about embedding security into the development process, and that means measuring how effectively you identify and remediate risks. GitHub Advanced Security (GHAS) offers powerful signals here.

Open Vulnerabilities and Aging

# Code scanning alerts by severity and age
gh api /repos/<org>/<repo>/code-scanning/alerts \
  --jq '.[] | {rule_id, severity, created_at, dismissed_at, fixed_at}'

Industry Benchmarks: High-performing teams maintain fewer than 10 critical vulnerabilities open at any time and resolve critical findings within 24-48 hours. Medium performers may carry 10-50 open critical issues with resolution times of 1-2 weeks. Low performers accumulate hundreds of open vulnerabilities with remediation measured in months.

What "Good" Looks Like: Your critical and high-severity vulnerability count trends downward over time. No critical vulnerability remains open longer than your SLA (typically 7 days). You have zero known vulnerabilities older than 90 days. Your backlog of medium and low-severity findings decreases quarter over quarter, indicating you're not just fixing new issues but addressing technical debt.

Common Measurement Challenges: False positives inflate your numbers and erode trust in scanning tools; invest time tuning rules and suppressing noise. Not all vulnerabilities are exploitable in your context; consider exploitability and reachability analysis rather than counting every theoretical issue. Alert fatigue sets in when teams see hundreds of findings; prioritize ruthlessly by severity, exploitability, and business impact.

Time to Remediate

Track created and resolved timestamps on alerts to measure how quickly vulnerabilities are fixed.

Industry Benchmarks: Elite security programs remediate critical vulnerabilities within 24 hours and high-severity issues within 7 days. Medium and low-severity findings should be addressed within 30 and 90 days respectively. Organizations with mature DevSecOps practices often achieve median remediation times under 5 days for all severities.

What "Good" Looks Like: Your remediation time consistently meets or beats your internal SLAs. The time-to-fix decreases as your team builds muscle memory and automation around common vulnerability patterns. You measure time from discovery to deployed fix, not just time to code commit. You differentiate between remediation (actually fixing the vulnerability) and mitigation (implementing compensating controls), tracking both separately.

Common Measurement Challenges: Disagreement about when the clock starts: is it when the scanner first detects the issue, when a ticket is created, or when a human triages it? Ambiguity about when it stops: when code is merged, when it's deployed to production, or when the scanner confirms the fix? Dismissed or "won't fix" vulnerabilities skew averages if not handled separately. Dependency vulnerabilities where you're waiting for upstream maintainers require different measurement approaches than code you control.

Dependency Health

# Dependabot alerts aging
gh api /repos/<org>/<repo>/dependabot/alerts \
  --jq '.[] | {package: .dependency.package.name, severity, created_at, dismissed_at}'

Industry Benchmarks: Organizations with strong supply chain security maintain zero critical dependency vulnerabilities in production code and keep 95%+ of dependencies up to date within one major version of current releases. They track dependency age and proactively update libraries before vulnerabilities are announced. A healthy dependency refresh rate is monthly for patch updates and quarterly for minor version updates.

What "Good" Looks Like: Your dependency alert count trends toward zero over time. You have automated processes (like Dependabot) that propose updates regularly, and your team merges them quickly. You maintain an inventory of all dependencies including transitive ones. Critical dependencies have identified maintainers and fallback plans if projects are abandoned. You've eliminated dependencies with known vulnerabilities older than 30 days.

Common Measurement Challenges: Transitive dependencies (dependencies of your dependencies) are invisible to many teams but represent significant risk. Not all updates are straightforward: breaking changes require testing and refactoring effort that's hard to predict. Alert fatigue when automated tools propose dozens of updates weekly; teams need filtering and prioritization logic. License compliance issues get conflated with security issues, creating confusion about what needs immediate action.

Secret Exposure Prevention

# Secret scanning alerts
gh api /repos/<org>/<repo>/secret-scanning/alerts \
  --jq '.[] | {secret_type, state, created_at, resolved_at}'

Industry Benchmarks: Best-in-class organizations maintain zero exposed secrets in their repositories at any given time. When secrets are accidentally committed, they're revoked within 1 hour and rotated immediately. The occurrence rate should trend toward zero as teams adopt secret management solutions and pre-commit hooks. Organizations with mature secret hygiene see fewer than 1 secret exposure per 1000 commits.

What "Good" Looks Like: You have automated secret scanning on every push, with immediate notifications to committers and security teams. Exposed secrets are automatically revoked through integration with secret management platforms (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault). Your team uses environment variables, secrets management tools, and encrypted configuration files instead of hardcoding credentials. Developers are trained to recognize secrets and use tooling (like git-secrets or detect-secrets) locally before pushing code.

Common Measurement Challenges: False positives from test credentials, dummy API keys, and string patterns that look like secrets but aren't. Historical secrets in old commits that can't be removed without rewriting git history, creating tension between security and traceability. Secrets in configuration files that change format or location, requiring constant tuning of detection rules. Third-party integrations that generate tokens automatically, creating alert noise if not properly categorized. Determining when a secret was truly exposed (commit time, push time, or PR merge time) affects measurement and response urgency.

Why These Numbers Matter to the Business

Deployment frequency and lead time show whether your investment in automation and CI/CD is paying off. Change failure rate and MTTR reveal the true cost of speed and the resilience of your systems. Vulnerability aging and remediation time demonstrate security posture and compliance readiness. Dependency health and secret scanning metrics protect against supply chain attacks and catastrophic breaches.

For executives, these numbers translate into risk and cost. Faster recovery means less downtime and happier customers. Shorter lead times mean quicker delivery of features and revenue opportunities. For security leaders, remediation metrics provide evidence of policy adherence and help prioritize resources. For developers, clear feedback loops reduce friction and make security part of the daily workflow rather than an afterthought.

Building a Governance Framework Around Metrics

Collecting data is not enough. Enterprises need a governance model that defines who owns these metrics, how often they’re reviewed, and what actions follow. Successful organizations establish oversight domains (platform teams, security councils, centers of excellence) and create a cadence for reviewing risk and reliability trends.

Here’s an example of a nightly export workflow:

# .github/workflows/security-export.yml
on:
  schedule:
    - cron: "0 2 * * *"
jobs:
  export:
    runs-on: ubuntu-latest
    permissions:
      security-events: read
      contents: read
    steps:
      - name: Export code scanning alerts
        run: gh api /repos/$ORG/$REPO/code-scanning/alerts > code-alerts.json
      - name: Export dependabot alerts
        run: gh api /repos/$ORG/$REPO/dependabot/alerts > dep-alerts.json
      - name: Export secret scanning alerts
        run: gh api /repos/$ORG/$REPO/secret-scanning/alerts > secret-alerts.json
      - uses: actions/upload-artifact@v4
        with:
          name: security-alerts
          path: "*.json"

Putting It All Together

DevSecOps is not a destination; it’s a continuous improvement journey. Metrics are the map that keeps you on course. By focusing on a handful of meaningful signals, such as deployment frequency, lead time, change failure rate, MTTR, vulnerability remediation, dependency health, and secret exposure, you can balance speed and security without sacrificing either.

GitHub makes it possible to track these metrics without adding friction. With built-in dashboards, APIs, and automation workflows, you can turn raw data into actionable insights. The challenge is cultural: using metrics to drive learning and improvement, not blame. When teams see metrics as a tool for empowerment, DevSecOps becomes more than a buzzword, it becomes a competitive advantage.

Next Steps for Readers

Start small. Pick two or three metrics that matter most to your organization and implement the queries and workflows shared here. Build a central repository for data exports and dashboards. Establish a monthly review cadence with platform and security teams. Over time, expand your coverage and automate more of the process. The payoff is worth it: faster delivery, stronger security, and greater confidence in every release.

Need help understanding your metrics or putting together meaningful reports to help you take you DevSecOps game from good to great? Email me!

steve.kaschimer@slalom.com

GitHub Advanced Security: What You Get and How to Use It

2025-12-15T00:00:00Z

Security is no longer an afterthought in modern software development. With the rise of DevSecOps, security practices are woven into every stage of the development lifecycle. GitHub, as one of the most widely used platforms for code collaboration, has stepped up its game with GitHub Advanced Security (GHAS), a suite of premium features designed to help teams identify, prevent, and remediate vulnerabilities before they reach production.

If you’re a DevOps practitioner new to GitHub Advanced Security, this guide will walk you through what GHAS offers, why it matters, and how to use its features effectively. By the end, you’ll understand how to integrate these tools into your workflow and elevate your security posture without slowing down development.

Why GitHub Advanced Security matters

Traditional security models often rely on periodic audits or post-release vulnerability scans. These approaches are reactive and costly. DevSecOps flips the script by embedding security checks into the development pipeline, catching issues early when they’re cheaper and easier to fix.

GitHub Advanced Security is built on this principle. It provides automated, developer-friendly tools that surface security risks directly in your repositories. Instead of waiting for a penetration test or a compliance review, your team can address problems as part of everyday coding.

What’s Included in GitHub Advanced Security?

GHAS is not just one feature. Instead, it's a collection of capabilities designed to tackle different aspects of application security. The four pillars you'll work with are:

Code Scanning (powered by CodeQL)
Secret Scanning
Dependency Review
Security Overview

Each of these plays a unique role in safeguarding your codebase. Let's break them down.

Code Scanning: Find Vulnerabilities in Your Code

Code Scanning is GitHub's flagship static application security testing (SAST) tool, powered by CodeQL, a semantic code analysis engine. Unlike simple pattern-matching tools that look for suspicious strings, CodeQL understands the structure and flow of your code. It can trace how data moves through your application, identify where user input enters the system, and detect when that untrusted data reaches a dangerous sink without proper sanitization.

How CodeQL Works

CodeQL treats your code as a database. It builds a semantic model of your entire codebase, including control flow, data flow, and the relationships between functions and variables. You then query this database using a declarative language to find patterns that represent vulnerabilities.

For example, CodeQL can detect SQL injection by identifying code paths where:

User input enters the system (source)
That data flows through the application (data flow analysis)
The data is used to construct a SQL query without sanitization (sink)

This approach catches vulnerabilities that simpler tools miss, including complex multi-step exploits where tainted data passes through several functions before reaching a vulnerable point.

What CodeQL Catches

CodeQL comes with hundreds of built-in queries covering the most critical security issues across multiple languages (JavaScript/TypeScript, Python, Java, C#, C/C++, Go, Ruby):

Injection Flaws: SQL injection, command injection, LDAP injection, XPath injection
Cross-Site Scripting (XSS): Reflected, stored, and DOM-based XSS
Path Traversal: Directory traversal and arbitrary file access
Authentication Issues: Hardcoded credentials, weak crypto, insecure random number generation
Authorization Bypasses: Missing access controls, IDOR vulnerabilities
Resource Management: Memory leaks, resource exhaustion, uncontrolled recursion
Cryptographic Issues: Weak algorithms, improper key management, insufficient entropy

Enabling Code Scanning

Using the GitHub UI

Navigate to your repository.
Click Security → Code scanning.
Click Set up code scanning.
Choose CodeQL Analysis and select Default or Advanced setup.

The default setup automatically configures CodeQL for your detected languages and runs on every push and pull request.

Using GitHub Actions (Advanced)

For more control, create .github/workflows/codeql.yml:

{% raw %}

name: "CodeQL"
on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]
  schedule:
    - cron: '0 6 * * 1'  # Weekly scan on Mondays

jobs:
  analyze:
    name: Analyze
    runs-on: ubuntu-latest
    permissions:
      actions: read
      contents: read
      security-events: write

    strategy:
      matrix:
        language: [ 'javascript', 'python' ]

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Initialize CodeQL
        uses: github/codeql-action/init@v3
        with:
          languages: ${{ matrix.language }}
          queries: security-extended  # Include additional security queries

      - name: Autobuild
        uses: github/codeql-action/autobuild@v3

      - name: Perform CodeQL Analysis
        uses: github/codeql-action/analyze@v3
        with:
          category: "/language:${{ matrix.language }}"

{% endraw %}

Custom CodeQL Queries

Beyond the built-in queries, you can write custom queries tailored to your organization's specific security requirements. For example, you might want to flag usage of deprecated internal APIs or enforce that certain sensitive functions are always called with specific security parameters.

Here's a simple custom query that finds direct use of eval() in JavaScript:

import javascript

from CallExpr call
where call.getCalleeName() = "eval"
select call, "Direct use of eval() is dangerous and should be avoided."

To use custom queries, add them to your repository in a .github/codeql/queries directory and reference them in your workflow:

- name: Initialize CodeQL
  uses: github/codeql-action/init@v3
  with:
    languages: javascript
    queries: ./.github/codeql/queries

What Good Coverage Looks Like

High-performing teams using Code Scanning typically see:

90%+ of repositories with Code Scanning enabled
Critical and high-severity alerts resolved within 7 days
False positive rate below 10% (achieved through query tuning)
Weekly or bi-weekly scans on active branches, plus scans on every PR
Zero critical vulnerabilities in production code paths

Teams often start with the default query suite and gradually expand to security-extended or security-and-quality as they mature.

Why Organizations Choose GHAS: Real-World Impact

Before diving into setup and configuration, let's look at how real organizations use GitHub Advanced Security and the tangible value it delivers. These examples illustrate why GHAS has become essential for DevSecOps teams.

Case Study: Catching Leaked AWS Credentials Before Exploitation

A fintech startup building a payment processing platform accidentally committed AWS access keys to their public repository. Within minutes of the commit, GitHub's Secret Scanning detected the credentials and sent alerts to both the repository maintainers and the security team.

The Response:

The security team received an immediate notification via Slack integration
They revoked the exposed AWS credentials through their AWS account within 15 minutes
They rotated all related secrets and updated the application configuration
They implemented a pre-commit hook using git-secrets to prevent future incidents
The entire incident was resolved in under an hour, before any external party could exploit the credentials

The Impact: Without Secret Scanning, those credentials could have remained exposed for days or weeks. The company estimated this early detection saved them from potential unauthorized AWS charges (potentially tens of thousands of dollars) and regulatory compliance issues related to PCI-DSS.

Case Study: Supply Chain Attack Prevention Through Dependency Review

A healthcare SaaS company using GHAS received a Dependabot alert about a critical vulnerability in a popular logging library they used. The vulnerability (CVE-2021-44228, Log4Shell) had a CVSS score of 10.0 and was being actively exploited in the wild.

The Response:

Dependency Review flagged the vulnerable version in all pull requests attempting to merge code
The platform team created a dedicated task force to assess impact across 200+ repositories
Using the Security Overview dashboard, they identified 47 repositories using the vulnerable version
They used GitHub's bulk operations API to create automated pull requests with the patched version
Within 72 hours, 45 of 47 repositories were patched and deployed

The Impact: The centralized visibility through Security Overview turned what could have been a months-long remediation effort into a coordinated 3-day sprint. Their competitors without similar tooling took an average of 3-6 weeks to fully remediate.

Enterprise Migration Strategy: From Manual Reviews to Automated Security

A global enterprise with 500+ repositories and 200+ developers was struggling with their manual security review process. Security reviews were creating a bottleneck, with a median 5-day wait time before security approval. Developers saw security as an impediment rather than an enabler.

The Transformation:

Phase 1 (Month 1-2): Enabled Code Scanning on 10 pilot repositories representing different tech stacks (Node.js, Python, Java, .NET). Tuned false positive rates to below 15%.
Phase 2 (Month 3-4): Rolled out Secret Scanning and Dependabot alerts to all 500 repositories. Integrated alerts with their existing ticketing system (Jira) for tracking.
Phase 3 (Month 5-6): Implemented branch protection rules requiring passing Code Scanning and Dependency Review checks before merge. Reduced manual security reviews from 100% to only high-risk changes (infrastructure changes, authentication modifications, API design changes).
Phase 4 (Month 7-8): Established security champions program with two developers per team trained on GHAS. Created internal documentation and runbooks for common alert types.

The Impact:

Median time to security approval dropped from 5 days to 4 hours
Critical vulnerability detection increased by 300% (from catching ~25% to ~75% based on penetration test results)
Developer satisfaction with security processes increased from 2.1/5 to 4.3/5
Security team shifted focus from manual code review to threat modeling and security architecture

The ROI Case: GHAS Cost vs. Breach Cost

GitHub Advanced Security costs approximately $49 per active committer per month. For a team of 50 developers, that's $29,400 per year. This investment must be weighed against security risks:

Cost of a security breach:

Average data breach cost: $4.45 million (IBM 2023 Cost of a Data Breach Report)
Regulatory fines: GDPR fines up to €20 million or 4% of annual revenue; HIPAA fines up to $1.5 million per violation
Reputational damage: Customer churn typically 5-10% after a public breach
Incident response costs: $245 per hour for forensics, $500-$1,000 per hour for specialized consultants
Legal costs: Average $1.2 million for breach-related litigation

Break-even analysis: If GHAS prevents even one moderate security incident (estimated cost $150,000 in remediation, notification, and regulatory response), it pays for itself 5x over for a 50-person team.

Additional value beyond breach prevention:

Velocity preservation: Automated security checks don't slow developers down like manual reviews do
Developer empowerment: Immediate, actionable feedback rather than abstract security guidelines
Compliance evidence: Auditors love documented, automated security controls
Insurance benefits: Some cyber insurance providers offer premium reductions for organizations with SAST/DAST tooling

For most organizations shipping customer-facing applications, the question isn't whether GHAS is worth the cost, but whether they can afford not to have it.

Getting Started: Enabling Core Features

The case studies above demonstrate GHAS's value, but how do you actually implement it? This section walks through enabling each core feature. The key principle: start simple with basic enablement, prove value quickly, then expand with advanced configuration.

Secret Scanning: Stop Leaks Before They Happen

Secrets, such as API keys, tokens and passwords, are the crown jewels of your application. Accidentally committing them to a repository can lead to catastrophic breaches. GitHub’s Secret Scanning feature helps prevent this.

How It Works

Secret Scanning automatically scans your commits for patterns that match known secret formats. This includes credentials for cloud providers, database connection strings, and more. When it detects a secret, it alerts you so you can revoke and rotate it immediately.

Enabling Secret Scanning

Using the GitHub UI

Navigate to your repository.
Click Settings → Code security and analysis.
Under Secret scanning, click Enable.

Using GitHub API

curl \
  -X PATCH \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  https://api.github.com/repos/OWNER/REPO \
  -d '{"security_and_analysis":{"secret_scanning":{"status":"enabled"}}}'

Replace OWNER and REPO with your repository details.

Dependency Review: Know What You’re Shipping

Modern applications rely heavily on third-party libraries. While this accelerates development, it also introduces risk. Vulnerabilities in dependencies can become entry points for attackers. Dependency Review helps you manage this risk by providing visibility into changes to your dependency graph.

How Dependency Review Works

Dependency Review integrates with pull requests to show you exactly what dependencies are being added, removed, or updated. It displays:

New dependencies introduced in the PR
Known vulnerabilities in those dependencies (powered by GitHub Advisory Database)
License information to catch licensing issues before merge
Dependency graph changes showing direct and transitive dependencies

When you open a pull request that modifies a manifest file (package.json, requirements.txt, pom.xml, Gemfile, etc.), Dependency Review automatically generates a comparison showing the security impact.

Understanding Dependabot vs. Dependency Review

These two features work together but serve different purposes:

Dependabot Alerts:

Continuously monitors your existing dependencies
Notifies you when vulnerabilities are discovered in dependencies you're already using
Generates automated pull requests to update vulnerable dependencies
Runs on a schedule (daily checks)

Dependency Review:

Runs on pull requests before code is merged
Prevents new vulnerable dependencies from being introduced
Blocks merges based on configurable severity thresholds
Provides just-in-time security feedback during development

Think of Dependabot as your continuous monitoring system and Dependency Review as your gate keeper.

Enabling Dependency Review

Using the GitHub UI

Go to Settings → Code security and analysis.
Under Dependency review, click Enable.

Example Workflow with License Controls

You can enforce dependency review checks using GitHub Actions with additional license compliance:

name: Dependency Review
on: [pull_request]
jobs:
  dependency-review:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Dependency Review
        uses: actions/dependency-review-action@v3
        with:
          fail-on-severity: high
          deny-licenses: GPL-3.0, AGPL-3.0
          allow-licenses: MIT, Apache-2.0, BSD-3-Clause

Understanding the Dependency Graph

The dependency graph visualizes all packages your project depends on, distinguishing between:

Direct dependencies: Packages explicitly declared in your manifest files
Transitive dependencies: Dependencies of your dependencies

Most vulnerabilities (80-90%) exist in transitive dependencies, making the graph view essential for understanding your complete security exposure. The graph also helps identify which direct dependency is pulling in a problematic transitive dependency, making it easier to address the issue.

Prioritizing Dependency Updates

Not all vulnerabilities require immediate action. Use these criteria to prioritize:

Priority	CVSS Score	Characteristics	Action Timeline
Immediate	9.0-10.0	Active exploits, network-accessible, no auth required	24 hours
High	7.0-8.9	Exploitable with user interaction or limited scope	7 days
Medium	4.0-6.9	Requires specific conditions or configuration	30 days
Low	0.1-3.9	Difficult to exploit or minimal impact	90 days

To check if a vulnerability has known exploits, query the GitHub Advisory Database:

query {
  securityVulnerabilities(first: 1, ecosystem: NPM, package: "lodash") {
    nodes {
      advisory {
        summary
        severity
        cvss {
          score
        }
        references {
          url
        }
      }
    }
  }
}

Cross-reference with the CISA KEV (Known Exploited Vulnerabilities) catalog and EPSS (Exploit Prediction Scoring System) scores for additional context.

Security Overview: Your Command Center

Managing security across multiple repositories can feel overwhelming. Security Overview provides a centralized dashboard for your organization’s security posture. It aggregates alerts from Secret Scanning, Dependabot, and Code Scanning, giving you a bird’s-eye view of risks.

Accessing Security Overview

Navigate to:

Organization Settings → Security → Security Overview

Advanced Configuration & Customization

Now that you have the basics running, it's time to tailor GHAS to your organization's specific needs. The default configurations provide solid coverage, but customization unlocks the full power of GHAS for your unique security requirements and development workflows.

Custom Secret Patterns for Internal Tokens

GitHub's Secret Scanning includes patterns for hundreds of popular services (AWS, Azure, GitHub tokens, Stripe keys, etc.), but your organization likely has internal secrets that don't match public patterns. You can define custom patterns to detect these.

Creating a Custom Pattern:

Navigate to Organization Settings → Code security and analysis → Secret scanning
Click New pattern
Define your pattern using regular expressions

Example: Internal API Token Pattern

company_api_key_[a-zA-Z0-9]{32}

Example: Database Connection String

Server=.+;Database=.+;User Id=.+;Password=.+;

Custom patterns support:

Test strings to validate your regex before publishing
Dry run mode to see what would be detected without generating alerts
False positive suppression through comment annotations in code

Configuring Severity Thresholds and Alert Routing

Not all alerts require the same urgency. You can configure how alerts are prioritized and who receives notifications based on severity.

Branch Protection Rules Tied to Security:

# .github/settings.yml (using probot/settings)
branches:
  - name: main
    protection:
      required_status_checks:
        strict: true
        contexts:
          - "CodeQL Analysis"
          - "Dependency Review"
          - "Secret Scanning Check"
      required_pull_request_reviews:
        required_approving_review_count: 1
        dismiss_stale_reviews: true

Alert Routing with GitHub Actions:

Route different severity levels to different channels:

{% raw %}

name: Security Alert Router
on:
  code_scanning_alert:
    types: [created, reopened]
jobs:
  route-alert:
    runs-on: ubuntu-latest
    steps:
      - name: Route Critical Alerts
        if: github.event.alert.rule.severity == 'critical'
        run: |
          curl -X POST ${{ secrets.PAGERDUTY_WEBHOOK }} \
            -H "Content-Type: application/json" \
            -d '{"severity":"critical","summary":"Critical security alert in ${{ github.repository }}"}'
      
      - name: Route High Alerts
        if: github.event.alert.rule.severity == 'high'
        run: |
          curl -X POST ${{ secrets.SLACK_SECURITY_CHANNEL }} \
            -H "Content-Type: application/json" \
            -d '{"text":"High severity alert: ${{ github.event.alert.rule.description }}"}'
      
      - name: Route Medium/Low Alerts
        if: github.event.alert.rule.severity == 'medium' || github.event.alert.rule.severity == 'low'
        run: |
          gh issue create \
            --title "Security Alert: ${{ github.event.alert.rule.description }}" \
            --label security,automated \
            --body "Alert details: ${{ github.event.alert.html_url }}"

{% endraw %}

Integrating with Jira and ServiceNow

For enterprises with existing ticketing systems, you can automatically create tickets for security alerts.

Jira Integration Example:

{% raw %}

name: Create Jira Ticket for Security Alerts
on:
  code_scanning_alert:
    types: [created]
jobs:
  create-jira-ticket:
    runs-on: ubuntu-latest
    steps:
      - name: Create Jira Issue
        uses: atlassian/gajira-create@v3
        with:
          project: SECURITY
          issuetype: Bug
          summary: "[${{ github.event.alert.rule.severity }}] ${{ github.event.alert.rule.description }}"
          description: |
            Alert detected in repository ${{ github.repository }}
            Severity: ${{ github.event.alert.rule.severity }}
            File: ${{ github.event.alert.instances[0].location.path }}
            Line: ${{ github.event.alert.instances[0].location.start_line }}
            
            GitHub Alert: ${{ github.event.alert.html_url }}
          fields: '{"priority": {"name": "${{ github.event.alert.rule.severity == 'critical' && 'Highest' || 'High' }}"}}'

{% endraw %}

Setting Up Security Policies at Organization Level

Instead of configuring security settings repository-by-repository, you can establish organization-wide policies that apply to all repositories (or specific subsets).

Organization Security Policy (SECURITY.md in .github repo):

# Security Policy

## Reporting a Vulnerability

Report vulnerabilities to security@company.com or through our private disclosure program at https://hackerone.com/company

## Security Scanning Requirements

All repositories must have:
- Code Scanning enabled with at least weekly scans
- Secret Scanning enabled with push protection
- Dependabot alerts enabled with auto-merge for patch updates

## Remediation SLAs

- **Critical vulnerabilities**: 24 hours
- **High vulnerabilities**: 7 days
- **Medium vulnerabilities**: 30 days
- **Low vulnerabilities**: 90 days

## Branch Protection

Production branches (`main`, `production`) must:
- Require passing Code Scanning and Dependency Review
- Require at least one approval from CODEOWNERS
- Prohibit force pushes
- Require signed commits

Enforcing Policies with GitHub API:

# Enable GHAS features for all repos in an organization
for repo in $(gh repo list myorg --json name --jq '.[].name'); do
  gh api -X PATCH /repos/myorg/$repo \
    -f security_and_analysis[secret_scanning][status]=enabled \
    -f security_and_analysis[secret_scanning_push_protection][status]=enabled \
    -f security_and_analysis[dependabot_security_updates][status]=enabled
done

Customizing CodeQL Queries

You can adjust which CodeQL queries run to balance security coverage with false positive rates.

Query Suites:

default: Standard security queries, good balance
security-extended: Additional security queries, more comprehensive but higher false positive rate
security-and-quality: Security plus code quality checks

Custom Query Configuration (.github/codeql/codeql-config.yml):

name: "Custom CodeQL Config"
queries:
  - uses: security-extended
  - uses: ./.github/codeql/custom-queries

query-filters:
  - exclude:
      id: js/incomplete-sanitization
  - exclude:
      tags:
        - experimental

paths-ignore:
  - "**/*.test.js"
  - "**/vendor/**"
  - "**/node_modules/**"

paths:
  - "src/**"
  - "lib/**"

Managing False Positives

False positives are inevitable with any security tool. The key is having a systematic process for handling them.

Dismissing Alerts with Reason Tracking:

# Dismiss a false positive via API with reason
gh api -X PATCH /repos/OWNER/REPO/code-scanning/alerts/ALERT_NUMBER \
  -f state=dismissed \
  -f dismissed_reason="false positive" \
  -f dismissed_comment="This regex pattern only matches internal test data, not user input"

Common Dismissal Reasons:

False positive: The tool incorrectly identified an issue
Won't fix: The issue is real but accepted risk (document why!)
Used in tests: The code only runs in test environments

Best Practices:

Require a comment explaining every dismissal
Review dismissed alerts quarterly to ensure decisions still make sense
Track dismissal rates by team to identify training opportunities
Use suppressions in code for persistent false positives:

# github/codeql: disable sql-injection
# This query is safe because user_input is validated against whitelist
query = f"SELECT * FROM users WHERE role = '{user_input}'"

Integrating GHAS into CI/CD Workflows

Configuration alone isn't enough-security checks must be enforced in your development workflow. This section shows how to weave GHAS into CI/CD pipelines, transforming security from optional to mandatory. By shifting security left, you catch issues in pull requests before they reach production.

Enforcing Secret Scanning in CI/CD

Block merges when secret scanning detects exposed credentials:

{% raw %} {% raw %}

name: Block Merge on Secret Alerts
on: [pull_request]
jobs:
  check-secrets:
    runs-on: ubuntu-latest
    steps:
      - name: Check for Secret Scanning Alerts
        run: |
          alerts=$(gh api repos/$GITHUB_REPOSITORY/secret-scanning/alerts --jq '.[] | select(.state=="open")')
          if [ -n "$alerts" ]; then
            echo "Open secret scanning alerts detected. Failing build."
            exit 1
          fi
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

{% endraw %} {% endraw %}

Enforcing Dependency Review in CI/CD

Prevent vulnerable dependencies from being merged:

name: Dependency Review
on: [pull_request]
jobs:
  dependency-review:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Dependency Review
        uses: actions/dependency-review-action@v3
        with:
          fail-on-severity: high
          deny-licenses: GPL-3.0, AGPL-3.0
          allow-licenses: MIT, Apache-2.0, BSD-3-Clause

Enforcing Code Scanning in CI/CD

Block pull request merges when Code Scanning detects critical or high-severity issues:

{% raw %}

name: Enforce Code Scanning
on: [pull_request]
jobs:
  check-code-scanning:
    runs-on: ubuntu-latest
    steps:
      - name: Check for Critical Alerts
        run: |
          alerts=$(gh api repos/$GITHUB_REPOSITORY/code-scanning/alerts \
            --jq '[.[] | select(.state=="open" and (.rule.severity=="critical" or .rule.severity=="high"))] | length')
          if [ "$alerts" -gt 0 ]; then
            echo "Critical or high-severity code scanning alerts detected. Failing build."
            exit 1
          fi
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

{% endraw %}

Branch Protection Rules

Configure branch protection to require passing security checks before merge:

GitHub UI:

Navigate to Settings → Branches
Add a branch protection rule for main
Enable "Require status checks to pass before merging"
Select your security workflows (Code Scanning, Dependency Review, Secret Scanning)
Enable "Require branches to be up to date before merging"

Using GitHub API:

curl -X PUT \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer $GITHUB_TOKEN" \
  https://api.github.com/repos/OWNER/REPO/branches/main/protection \
  -d '{
    "required_status_checks": {
      "strict": true,
      "contexts": ["CodeQL", "Dependency Review", "Secret Scanning"]
    },
    "enforce_admins": true,
    "required_pull_request_reviews": {
      "required_approving_review_count": 1
    }
  }'

Best Practices for Long-Term Success

With GHAS enabled and integrated into your CI/CD pipeline, focus shifts to operational excellence. These practices help teams maintain security effectiveness over time.

Establish Clear Remediation SLAs

Security alerts are only valuable if teams act on them. Establish service level agreements (SLAs) for remediation based on severity:

Critical: 24 hours - These represent actively exploitable vulnerabilities or exposed secrets
High: 7 days - Serious vulnerabilities that could lead to compromise
Medium: 30 days - Issues that increase attack surface but aren't immediately exploitable
Low: 90 days - Code quality or defense-in-depth improvements

Track compliance with these SLAs and surface teams that consistently miss targets. This isn't about punishment; it's about identifying training needs or resource constraints.

Example SLA Dashboard Query:

# Get all open high/critical alerts older than 7 days
gh api /repos/OWNER/REPO/code-scanning/alerts \
  --jq '.[] | select(.state=="open" and (.rule.severity=="critical" or .rule.severity=="high") and (now - (.created_at | fromdateiso8601) > 604800)) | {number, severity: .rule.severity, age: ((now - (.created_at | fromdateiso8601)) / 86400 | floor)}'

Handle False Positives Systematically

False positives erode trust in security tools. When developers see too many incorrect alerts, they start ignoring all alerts, including real ones.

Strategies to manage false positives:

Tune your queries: Start with default CodeQL queries, then gradually add security-extended queries as your team gains expertise

Use path filters: Exclude test code, vendor libraries, and generated files from scanning

Document dismissals: Require a clear explanation for every dismissed alert

Review dismissals quarterly: Ensure past decisions still make sense

Create custom suppressions: For persistent false positives, use in-code suppressions with explanatory comments

Acceptable false positive rate: Aim for under 10%. If you're above 20%, invest time in tuning queries and training your team on what constitutes a real vulnerability.

Run Security Checks Efficiently

Security scans can slow down your CI/CD pipeline if not configured properly. Here are strategies to keep things fast:

Parallel Execution: {% raw %}

jobs:
  security:
    strategy:
      matrix:
        check: [code-scanning, secret-scanning, dependency-review]
    runs-on: ubuntu-latest
    steps:
      - name: Run ${{ matrix.check }}
        run: ./scripts/${{ matrix.check }}.sh

{% endraw %}

Caching: {% raw %}

- name: Cache CodeQL
  uses: actions/cache@v3
  with:
    path: ~/.codeql
    key: codeql-${{ runner.os }}-${{ hashFiles('**/codeql-config.yml') }}

{% endraw %}

Incremental Analysis: Only scan changed files on pull requests: {% raw %}

- name: Get changed files
  id: changed-files
  run: |
    echo "files=$(git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.sha }} | tr '\n' ' ')" >> $GITHUB_OUTPUT

- name: Run CodeQL on changed files
  if: steps.changed-files.outputs.files != ''
  run: codeql analyze --sarif-category=pr --paths=${{ steps.changed-files.outputs.files }}

{% endraw %}

Benchmark: Well-configured GHAS scans should add no more than 5-10 minutes to your CI/CD pipeline for most repositories.

Integrate Alerts with Communication Channels

Developers are most likely to act on security alerts when they see them in their existing workflows. Don't expect them to regularly check a dashboard.

Slack Integration:

- name: Notify Slack on Critical Alert
  if: github.event.alert.rule.severity == 'critical'
  run: |
    curl -X POST -H 'Content-type: application/json' \
    --data '{"text":"🚨 Critical security alert in ${{ github.repository }}: ${{ github.event.alert.rule.description }}\nView: ${{ github.event.alert.html_url }}"}' \
    ${{ secrets.SLACK_WEBHOOK }}

Microsoft Teams Integration:

- name: Notify Teams
  uses: toko-bifrost/ms-teams-deploy-card@master
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    webhook-uri: ${{ secrets.TEAMS_WEBHOOK }}
    card-layout-start: cozy
    show-on-start: false
    show-on-exit: true
    custom-facts: |
      - name: Severity
        value: ${{ github.event.alert.rule.severity }}
      - name: Rule
        value: ${{ github.event.alert.rule.description }}

PagerDuty for Critical Issues:

- name: Page on-call for critical vulnerability
  if: github.event.alert.rule.severity == 'critical'
  run: |
    curl -X POST https://events.pagerduty.com/v2/enqueue \
      -H 'Content-Type: application/json' \
      -d '{
        "routing_key": "${{ secrets.PAGERDUTY_ROUTING_KEY }}",
        "event_action": "trigger",
        "payload": {
          "summary": "Critical vulnerability in ${{ github.repository }}",
          "severity": "critical",
          "source": "GitHub Advanced Security"
        }
      }'

Build a Security Metrics Dashboard

Track your security posture over time with key metrics:

Alert volume trends: Are new alerts decreasing as your code improves?

Remediation time by severity: Are you meeting your SLAs?

False positive rate: Is your tuning working?

Coverage metrics: What percentage of repositories have GHAS enabled?

Alert aging: How many alerts are older than 90 days?

Example: Query for metrics collection

#!/bin/bash
# Collect security metrics across all repos

ORG="your-org"
OUTPUT="security-metrics-$(date +%Y-%m-%d).json"

echo "{" > $OUTPUT
echo "  \"timestamp\": \"$(date -Iseconds)\"," >> $OUTPUT
echo "  \"repositories\": [" >> $OUTPUT

for repo in $(gh repo list $ORG --json name --jq '.[].name'); do
  echo "    {" >> $OUTPUT
  echo "      \"name\": \"$repo\"," >> $OUTPUT
  
  # Code scanning alerts by severity
  critical=$(gh api /repos/$ORG/$repo/code-scanning/alerts --jq '[.[] | select(.state=="open" and .rule.severity=="critical")] | length')
  high=$(gh api /repos/$ORG/$repo/code-scanning/alerts --jq '[.[] | select(.state=="open" and .rule.severity=="high")] | length')
  
  # Secret scanning alerts
  secrets=$(gh api /repos/$ORG/$repo/secret-scanning/alerts --jq '[.[] | select(.state=="open")] | length')
  
  # Dependabot alerts
  deps=$(gh api /repos/$ORG/$repo/dependabot/alerts --jq '[.[] | select(.state=="open")] | length')
  
  echo "      \"code_scanning\": {\"critical\": $critical, \"high\": $high}," >> $OUTPUT
  echo "      \"secret_scanning\": $secrets," >> $OUTPUT
  echo "      \"dependabot\": $deps" >> $OUTPUT
  echo "    }," >> $OUTPUT
done

echo "  ]" >> $OUTPUT
echo "}" >> $OUTPUT

Provide Developer Training

The most sophisticated security tools are useless if developers don't understand them. Invest in training:

Onboarding for New Developers:

30-minute GHAS overview session
Hands-on lab: trigger an alert, triage it, fix it, verify resolution
Documentation on how to dismiss false positives correctly

Ongoing Education:

Monthly "security office hours" where developers can ask questions
Quarterly reviews of common vulnerability patterns found in your codebase
Annual security training with real examples from your organization

Security Champions Program:

Identify 1-2 developers per team interested in security
Provide deeper training (OWASP Top 10, threat modeling, secure coding)
Give them time (20%) to triage alerts and mentor teammates

Start Small, Scale Gradually

Don't try to enable everything everywhere on day one. Follow a phased approach:

Phase 1: Pilot (1-2 months)

Enable GHAS on 5-10 repositories that represent your tech stack diversity. Focus this phase on tuning the configuration and learning how the tools work in your environment. Gather feedback from developers to understand their experience and identify any friction points.

Phase 2: Expand (3-6 months)

Roll out GHAS to 25% of your repositories, prioritizing those with the highest business impact. During this phase, integrate security checks into your CI/CD pipelines to enforce quality gates. Establish clear remediation SLAs so teams know how quickly they need to address different severity levels of security issues.

Phase 3: Scale (6-12 months)

Enable GHAS on all active repositories across your organization to achieve full security coverage. Implement branch protection rules that prevent merges when security issues are detected, ensuring no vulnerabilities slip through to production. Enforce compliance through automation by creating organizational policies and using GitHub Actions to maintain consistent security standards across all teams.

Phase 4: Optimize (ongoing)

Continuously improve your GHAS implementation by reducing false positive rates through better query tuning and path filters. Work to decrease remediation times by streamlining workflows and providing better training to developers. Add custom queries tailored to your organization's specific risks and coding patterns, ensuring GHAS catches vulnerabilities unique to your technology stack and business domain.

Build Effective Security Champions Teams

Organizations that succeed with GHAS typically don't rely solely on a central security team. They establish a security champions program where developers across teams receive additional security training and act as the first line of defense.

Typical Structure:

Central Security Team (2-5 people): Owns security policy, manages GHAS configuration at the organization level, tunes alert rules, conducts security architecture reviews
Security Champions (1-2 per team): Embedded developers with 20% time allocation to security, triage GHAS alerts within their team, provide peer education, participate in security council meetings
Platform Team: Maintains security automation, manages CI/CD security gates, creates shared GitHub Actions for security checks
Development Teams: Own remediation of alerts in their codebases, integrate security checks into their workflows, participate in game days and security training

This distributed model ensures security knowledge spreads throughout the organization while keeping security experts focused on high-value activities.

Troubleshooting & Common Pitfalls

Even with careful planning, you'll encounter challenges when operating GHAS at scale. Here's how to address the most common issues teams face.

Even with the best planning, you'll encounter challenges when rolling out GHAS. Here are the most common issues and how to address them:

Alert Fatigue

Problem: Teams receive hundreds of alerts on day one and become overwhelmed, leading to alerts being ignored entirely.

Solution:

Start with critical and high severity alerts only
Use the security-severity filter in CodeQL to focus on high-impact issues
Implement a phased rollout where you fix existing issues before enabling additional scanning
Set up alert routing so only relevant teams see their alerts (not organization-wide notifications)

Prevention strategy:

# Enable CodeQL with limited severity
- uses: github/codeql-action/init@v3
  with:
    queries: security-extended
    # Only fail on critical/high issues initially
- uses: github/codeql-action/analyze@v3
  with:
    upload: true
    wait-for-processing: true

False Positives Derailing Adoption

Problem: Developers lose trust in the tool when they see too many false positives, especially in legacy codebases.

Solution:

Create a documented process for dismissing alerts (require justification in comments)
Use CodeQL query exclusions for known false positive patterns specific to your codebase
Invest time upfront to tune queries before requiring remediation
Track false positive rates and continuously improve

Example: Suppress specific CWE in CodeQL config:

name: "CodeQL Config"
disable-default-queries: false
queries:
  - uses: security-extended
packs:
  - codeql/javascript-queries
paths-ignore:
  - test/**
  - vendor/**
query-filters:
  - exclude:
      id: js/sql-injection
      problem.severity: warning

Performance Impact on CI/CD Pipelines

Problem: CodeQL analysis adds 5-15 minutes to build times, slowing down development velocity.

Solution:

Run CodeQL on scheduled workflows (nightly) rather than on every commit
Use incremental analysis (only scan changed code) for pull requests
Run security scans in parallel with other CI jobs, not sequentially
Use self-hosted runners with better CPU resources for large repositories
Enable caching for CodeQL databases to speed up subsequent runs

Performance-optimized workflow:

name: "CodeQL - Optimized"
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  schedule:
    - cron: '0 2 * * 1'  # Weekly deep scan

jobs:
  analyze:
    runs-on: ubuntu-latest-8-cores  # Use larger runners
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v3
      - uses: github/codeql-action/init@v3
      - uses: github/codeql-action/autobuild@v3
      - uses: github/codeql-action/analyze@v3
        with:
          category: "/language:javascript"
          # Upload results but don't block PR on scheduled runs
          upload: true
          checkout_path: ${{ github.workspace }}

Secret Scanning Revealing Embarrassing Legacy Issues

Problem: Enabling secret scanning exposes years of accumulated secrets in commit history, creating a massive cleanup effort.

Solution:

Use GitHub's secret scanning push protection to prevent new secrets immediately
Prioritize active secrets over historical ones (check if tokens still work)
Use git-filter-repo or BFG Repo-Cleaner to rewrite history for critical secrets
Accept that some historical secrets may need to remain (if rotated/inactive) rather than rewriting years of history
Focus remediation efforts on secrets exposed in the last 90 days first

Quick check if a token is still active:

# For GitHub tokens
curl -H "Authorization: token ghp_xxxxx" https://api.github.com/user

# For AWS keys
aws sts get-caller-identity --profile compromised-key

Licensing Costs vs. Security Value

Problem: Justifying the per-user cost of GHAS to leadership when ROI isn't immediately visible.

Solution:

Start with a pilot on critical repositories to demonstrate value with concrete metrics
Calculate cost of a breach ($4.45M average) vs. GHAS investment (~$49/user/month = $588/year)
Track time saved by preventing vulnerabilities from reaching production
Measure reduction in post-production security incidents
Document compliance benefits (SOC 2, ISO 27001 require security scanning)

ROI Calculation Example:

Team of 50 developers: 50 × $49/month = $2,450/month = $29,400/year
One prevented breach (MTTR from 48 hours to 4 hours saves $183K in incident response)
Prevented vulnerabilities reaching production: 12 critical issues caught in PR = $500K+ saved
Compliance audit time reduced: 40 hours saved = $8K
Net benefit: $661K/year vs. $29K investment = 22x ROI

Dependency Scanning Overhead on Large Monorepos

Problem: Dependency Review on monorepos with 50+ manifest files takes too long and creates noise.

Solution:

Use paths filters in workflows to only scan changed directories
Implement matrix strategies to scan different ecosystems in parallel
Configure allow-licenses to reduce license violation noise
Use Dependabot groups to batch related updates rather than individual PRs

Integrating GHAS with Your Broader Security Ecosystem

GHAS shouldn't operate in isolation. Modern security requires a layered approach where multiple tools complement each other. Here's how GHAS fits into your broader security strategy:

Complementing Commercial SAST/SCA Tools

If you already use tools like Snyk, Aqua Security, or Checkmarx, GHAS doesn't replace them-it complements them:

CodeQL (GHAS) strengths:

Deep semantic analysis of first-party code
Native GitHub integration with no third-party API dependencies
Customizable queries for organization-specific patterns
Free for public repositories

Commercial tool strengths:

Broader language support (Snyk supports 10+ more languages)
Container and infrastructure-as-code scanning
Advanced license compliance management
Dedicated support and consulting

Best practice: Use GHAS as your primary gate in the CI/CD pipeline for fast feedback, and run commercial tools on a nightly schedule for comprehensive coverage. Configure both to write to your centralized security dashboard.

Exporting to SIEM and Analytics Platforms

Send GHAS alert data to your Security Information and Event Management (SIEM) system for centralized monitoring:

Example: Export to Splunk

#!/bin/bash
# Export GHAS alerts to Splunk HEC endpoint

ORG="your-org"
SPLUNK_HEC_TOKEN="your-token"
SPLUNK_URL="https://splunk.company.com:8088/services/collector"

# Fetch all code scanning alerts
gh api "/orgs/$ORG/code-scanning/alerts" --paginate | \
jq -c '.[] | {
  time: .created_at,
  source: "github_ghas",
  sourcetype: "code_scanning",
  event: {
    repo: .repository.full_name,
    severity: .rule.severity,
    rule_id: .rule.id,
    state: .state,
    url: .html_url
  }
}' | \
while read -r event; do
  curl -k "$SPLUNK_URL" \
    -H "Authorization: Splunk $SPLUNK_HEC_TOKEN" \
    -d "$event"
done

Building Custom Dashboards with GitHub API

GHAS provides robust REST and GraphQL APIs for building custom security dashboards:

Example: GraphQL query for organization-wide security posture

query OrgSecurityPosture($org: String!) {
  organization(login: $org) {
    repositories(first: 100) {
      nodes {
        name
        vulnerabilityAlerts(first: 10, states: OPEN) {
          totalCount
          nodes {
            securityVulnerability {
              severity
              package { name }
            }
          }
        }
      }
    }
  }
}

Use this data to create real-time dashboards in Grafana, Datadog, or your internal portal showing:

Alert trends over time
Repository risk scores
Remediation velocity by team
Compliance coverage metrics

Integrating with Policy-as-Code Frameworks

Combine GHAS with Open Policy Agent (OPA) or Conftest to enforce security policies:

Example: OPA policy requiring zero critical vulnerabilities

package github.security

deny[msg] {
  input.code_scanning_alerts[_].severity == "critical"
  input.code_scanning_alerts[_].state == "open"
  msg := "Deployment blocked: Critical security vulnerabilities must be resolved"
}

deny[msg] {
  input.secret_scanning_alerts[_].state == "open"
  msg := "Deployment blocked: Active secrets detected"
}

Enforce this policy in your deployment pipeline before promoting to production.

Final Thoughts

If you're serious about DevSecOps, GitHub Advanced Security is a must-have. It empowers developers to take ownership of security without sacrificing speed. Start small by enabling Secret Scanning on a few repositories, experiment with Dependency Review, and explore Security Overview. As you gain confidence, scale these practices across your organization.

Security isn’t a destination; it’s a journey. With GHAS, you have the tools to make that journey smoother, safer, and more efficient.

Need help on your GitHub Journey? Ask me!

steve.kaschimer@slalom.com

GitHub Actions: Reusable Workflows vs. Composite Actions - Know the Difference

2026-03-13T00:00:00Z

Every team that grows past a handful of GitHub Actions workflows eventually hits the same wall: duplicated YAML, copy-pasted step sequences, a deploy job that lives in six repositories. The solution is obvious - abstract the common pieces. GitHub gives you two tools to do that: reusable workflows and composite actions. The docs present them as siblings. They're not. They operate at different levels of the execution model, enforce different scoping rules, and fail in different ways when you use them outside their intended purpose.

Most of the bugs I've seen come from one pattern: a developer reads about both abstractions, picks the one that looks right, and discovers the hard way that secrets don't arrive, matrix values vanish, or a branch protection rule silently stops enforcing. This post walks through three concrete failure scenarios - real YAML, real error behavior - and ends with a decision framework you can apply without rereading the docs.

What Each One Actually Is

Before the failure scenarios, a precise definition of each mechanism. The marketing framing ("reuse your workflows!") is accurate but useless for debugging.

Reusable Workflows

A reusable workflow is a complete workflow file that runs as its own job (or set of jobs) inside the calling workflow run. It is invoked at the jobs: level using uses:.

# caller.yml
jobs:
  test:
    uses: ./.github/workflows/run-tests.yml
    with:
      node-version: "20"
    secrets: inherit

The called file must declare on: workflow_call:. It runs on its own runner, in its own environment, with its own job context. From GitHub's perspective - and from branch protection's perspective - it appears as a separate job in the workflow run, with its own status check named <calling-job> / <reusable-job>.

Composite Actions

A composite action is a reusable sequence of steps that runs inside the calling job. It is invoked at the steps: level using uses:, just like any other action.

# caller.yml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: ./.github/actions/setup-node
        with:
          node-version: "20"
      - run: npm test

The called file is an action.yml that declares runs.using: composite. Its steps execute inside the calling job, sharing the runner, the workspace, environment variables, and the job context. It is not a separate job. It has no separate status check.

That structural difference - job vs. steps - is the source of every failure scenario below.

The Three Failure Scenarios

1. The Disappearing Secret

This is the most common gotcha. A team moves their deployment logic into a composite action and discovers that the secret they need is silently empty at runtime.

The broken setup:

# .github/actions/deploy/action.yml
name: Deploy
description: Deploy to production
runs:
  using: composite
  steps:
    - name: Call deployment API
      shell: bash
      run: |
        curl -sf -X POST \
          -H "Authorization: Bearer ${{ secrets.DEPLOY_TOKEN }}" \
          https://api.example.com/deploy

# .github/workflows/release.yml
jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/deploy

The curl command sends an empty Authorization header. The API returns a 401. Nothing in the logs explains why - ${{ secrets.DEPLOY_TOKEN }} just evaluates to an empty string inside the composite action because the secrets context is not available inside composite action YAML. Composite actions run within the calling job's environment, but they don't inherit the calling job's secrets context automatically. GitHub explicitly scopes secrets away from composite action definitions to prevent accidental secret forwarding into third-party actions.

The fix - pass it as an input:

# .github/actions/deploy/action.yml
name: Deploy
description: Deploy to production
inputs:
  deploy-token:
    description: API token for the deployment endpoint
    required: true
runs:
  using: composite
  steps:
    - name: Call deployment API
      shell: bash
      env:
        DEPLOY_TOKEN: ${{ inputs.deploy-token }}
      run: |
        curl -sf -X POST \
          -H "Authorization: Bearer ${DEPLOY_TOKEN}" \
          https://api.example.com/deploy

# .github/workflows/release.yml
jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/deploy
        with:
          deploy-token: ${{ secrets.DEPLOY_TOKEN }}

Two things changed. First, the composite action declares a deploy-token input and reads it via inputs.deploy-token. Second, the calling workflow explicitly passes ${{ secrets.DEPLOY_TOKEN }} via with:. The secret is now in scope at the call site, where the secrets context is available, and forwarded as an opaque input value.

Notice the env: block in the step definition. Referencing secrets (including values derived from inputs that originally came from secrets) via environment variables rather than inline ${{ }} interpolation is a defense-in-depth practice - it prevents the value from appearing in runner debug logs when step debug logging is enabled.

If your composite action needs many secrets, the with: list can get long fast. When that happens, it's often a signal that a reusable workflow is actually the right tool - it supports secrets: inherit, which passes all secrets from the calling workflow automatically.

2. The Matrix That Won't Cooperate

A developer wants to test their library against three Node.js versions. They already have a reusable workflow for running tests. The natural move seems to be: loop the matrix, call the reusable workflow for each combination. They write this:

# .github/workflows/ci.yml
jobs:
  test:
    strategy:
      matrix:
        node: [18, 20, 22]
    uses: ./.github/workflows/run-tests.yml
    with:
      node-version: ${{ matrix.node }}

This actually works syntactically - matrix.* is available in the with: block of a reusable workflow call when the calling job has a strategy.matrix defined. Each matrix combination triggers a separate invocation of the reusable workflow. So far so good.

The problem appears in the GitHub Actions UI and in branch protection rules. Each matrix combination produces a set of jobs named:

test (18) / lint
test (18) / unit-tests
test (20) / lint
test (20) / unit-tests
test (22) / lint
test (22) / unit-tests

If you had a required status check configured as lint or unit-tests, it no longer matches anything. The check names now include the calling job name AND the matrix suffix. Your branch protection rule passes vacuously - no check with that name exists, so GitHub considers it satisfied - and you've accidentally disabled your quality gate.

The fix: Update your required status checks to match the full generated names, or restructure so the matrix lives inside the reusable workflow rather than at the call site:

# .github/workflows/run-tests.yml
on:
  workflow_call:

jobs:
  unit-tests:
    strategy:
      matrix:
        node: [18, 20, 22]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
      - run: npm ci && npm test

# .github/workflows/ci.yml
jobs:
  test:
    uses: ./.github/workflows/run-tests.yml

Now the generated job names are test / unit-tests (18), test / unit-tests (20), and test / unit-tests (22). The required status check test / unit-tests (18) is predictable and won't shift when the caller changes. Better yet: you can require just test / unit-tests and GitHub will wait for all matrix variants to pass.

A composite action sidesteps this entirely - its steps appear within the parent job, and the job name in branch protection is just the job name. No suffix, no nesting. If you're not sharing the workflow cross-repo and don't need secrets isolation, a composite action plus a matrix on the calling job is cleaner.

3. The Status Check That Lies

This one is the most dangerous because it doesn't cause a visible failure. It causes a missing failure - a gate you thought was enforcing stops enforcing.

Suppose you have a workflow with a build job that your branch protection rules require to pass before merging. The team refactors build to call a reusable workflow:

# Before refactor
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build

# After refactor
jobs:
  build:
    uses: ./.github/workflows/build.yml

The reusable workflow file contains a job named compile:

# .github/workflows/build.yml
on:
  workflow_call:

jobs:
  compile:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build

After the refactor, the workflow run produces a check named build / compile. The old check named build no longer exists. GitHub's required status check for build now matches nothing, so it's considered satisfied automatically. Every PR merges regardless of whether the build passes.

Nobody notices until a broken build ships to production.

The fix has two parts:

First, update the required status check in branch protection from build to build / compile to match the new job name structure:

# GitHub API - update required status check
# PATCH /repos/{owner}/{repo}/branches/{branch}/protection
{
  "required_status_checks": {
    "strict": true,
    "contexts": ["build / compile"]
  }
}

Second, make this explicit in your reusable workflow by naming the job clearly:

# .github/workflows/build.yml
on:
  workflow_call:

jobs:
  build:          # ← name this to match what branch protection expects
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build

If the reusable workflow job is also named build, the required check becomes build / build - redundant but unambiguous. Some teams prefix reusable workflow jobs with rw- to make it obvious which job names come from reusable workflows.

Composite actions don't have this problem. Their steps roll up into the parent job's status. If you refactor steps into a composite action, the job name in branch protection doesn't change. This is one of the strongest arguments for composite actions when cross-repo sharing isn't needed.

Decision Framework

Use these rules. They're opinionated because ambiguity is what causes the bugs described above.

Reach for a reusable workflow when:

You need secrets to be available inside the abstraction without explicitly passing each one (use secrets: inherit)
You want the abstraction to appear as its own named job in the workflow UI and in status checks
The workflow needs to run on a different runner type than the caller (separate runs-on)
You're sharing the automation across repositories
The logic involves multiple jobs with dependencies between them

Reach for a composite action when:

You're sharing a sequence of steps within the same repository (or same workflow)
The steps need access to the calling job's workspace, environment variables, or matrix context
You want the steps to appear inline in the calling job - same status check, same log view
You're building a reusable action you'll publish to the GitHub Marketplace
Keeping the calling workflow's total job count low matters for readability

One rule of thumb that holds up: if you're thinking "I want this to look like a step," use a composite action. If you're thinking "I want this to look like a job," use a reusable workflow.

Side-by-Side Reference

Capability	Reusable Workflow	Composite Action
Invoked at	`jobs:` level (`uses:`)	`steps:` level (`uses:`)
Runs on	Its own runner	Calling job's runner
Appears in UI as	Separate job(s)	Steps within calling job
Status check name	`<caller-job> / <rw-job>`	Same as calling job
Secrets access	Via `secrets:` or `secrets: inherit`	Must pass via `with:` inputs
Calling job's env vars	Not inherited	Inherited
Calling job's workspace	Not shared	Shared
Matrix context	Not inherited; pass via `inputs`	Inherited (`${{ matrix.* }}` works)
Cross-repo use	Yes	Yes (if published or referenced by path)
`outputs` support	Yes (workflow-level outputs)	Yes (action-level outputs)
Multiple jobs	Yes, with `needs:` chains	No (steps only)
`strategy.matrix`	Definable inside the workflow	N/A - runs within calling job

Closing Thoughts

Reusable workflows and composite actions are not interchangeable. The GitHub documentation groups them under "reusing workflows" in a way that makes them look like two flavors of the same thing. They're not. One is a job abstraction; the other is a step abstraction. That difference determines everything: how secrets flow, how status checks are named, how matrix strategies compose, and where logs appear.

The three failure scenarios in this post - the disappearing secret, the matrix naming problem, and the missing status check - don't show up as actionable errors. They show up as empty strings, confusing UI, and security gates that quietly stop working. The fix is always the same: understand which layer you're operating at and choose the abstraction that matches.

If you're auditing existing workflows for these issues, start with branch protection. Pull your required status check names, run a recent workflow, and verify every required check name appears somewhere in the checks list. If anything is missing, you've found a silent bypass. That's the one worth fixing first.

Have questions about structuring your GitHub Actions pipelines, or want help auditing your branch protection rules? Reach out.

steve.kaschimer@slalom.com

Deploying to GitHub Pages with GitHub Actions: Beyond the Defaults

2026-03-18T00:00:00Z

Most tutorials for deploying to GitHub Pages start with peaceiris/actions-gh-pages or the GitHub UI's auto-generated workflow. Both work. Neither is production-grade. The problems are predictable: every run reinstalls all npm packages from scratch, build artifacts persist indefinitely against your storage quota, and the site goes live on every push to main with no human gate between "CI passed" and "it's in front of users."

The official actions/deploy-pages action - introduced in 2022 and now the GitHub-recommended approach - solves most of this. But using it correctly means understanding OIDC token authentication, the artifact lifecycle, and how GitHub Environments create a reviewable deployment gate. This post builds the full production pipeline, step by step, for an Eleventy + Tailwind CSS site.

What the Default Workflow Gets Wrong

Before the fix, the failure list:

No caching: every run reinstalls all npm packages from scratch, adding 60-90 seconds to every deploy
Broad token permissions: classic GITHUB_TOKEN-based deploys grant write access to the entire repository context; OIDC-based deployment scopes that to the Pages deployment specifically
No environment protection: the site deploys directly on every push to main - no reviewer gate, no way to stop a bad deploy before it goes live
Artifact leakage: actions/upload-pages-artifact defaults to a 90-day retention window; a blog with daily publishing accumulates artifacts fast against your GitHub storage quota
gh-pages branch pollution: the peaceiris approach writes a separate gh-pages branch - another moving part to maintain, rebase on, and reason about when something goes wrong

The Build This Pipeline Serves

This blog - and the workflow in this post - runs on a specific stack. If you're on the same one, you can drop this directly into your repo.

Eleventy v2 (@11ty/eleventy) - static site generator, outputs to _site/
Tailwind CSS v3 (tailwindcss) - utility-first CSS, built as a separate step
npm-run-all - used to run Eleventy and Tailwind in parallel during development (npm run dev), sequentially for production

The relevant scripts from package.json:

{
  "scripts": {
    "build": "npx @11ty/eleventy",
    "build:css": "npx tailwindcss -i ./src/styles/input.css -o ./_site/styles/output.css --minify",
    "deploy": "npm run build && npm run build:css"
  }
}

The deploy script runs build first, then build:css. Order matters here: Eleventy creates the _site/ directory, and build:css writes its output directly into _site/styles/. Running them in parallel with npm-run-all --parallel risks a race condition where Tailwind tries to write before _site/ exists. The deploy script gets this right - use it instead of calling the steps individually.

Step 1: Configure GitHub Pages to Use the Actions Source

Before any workflow will work, GitHub Pages must be configured to deploy from GitHub Actions rather than from a branch. The default is branch-based (gh-pages), and actions/deploy-pages silently does nothing if you've left it there.

Go to Repository Settings → Pages → Build and deployment → Source and select GitHub Actions.

That's the only UI change required. Everything else is workflow config.

Step 2: OIDC Authentication - What It Is and Why It Matters

The deployment permissions block that shows up in every deploy-pages example deserves an explanation, not just a copy-paste:

permissions:
  contents: read    # Read the repo to build it
  pages: write      # Write to GitHub Pages
  id-token: write   # Request an OIDC token for deployment authentication

OIDC (OpenID Connect) is the mechanism GitHub Actions uses to issue short-lived, scoped tokens at runtime. When actions/deploy-pages runs, it requests an OIDC token from GitHub's identity provider - a token that is scoped specifically to a Pages deployment for this workflow run, on this repository, in this environment. The token expires when the run completes.

The alternative - using a static GITHUB_TOKEN or a Personal Access Token stored as a repository secret - grants broader permissions that persist indefinitely, require rotation, and are exposed in your secrets store. With OIDC there is nothing to rotate, nothing to store, and nothing to leak. The id-token: write permission is what allows the workflow to request this token.

Step 3: Dependency Caching

The single change with the highest return on effort. actions/setup-node supports built-in npm caching:

- uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'

With cache: 'npm', the action manages a cache keyed on the hash of your package-lock.json. When the lockfile hasn't changed - which is true for the vast majority of content-only commits on a blog - the cache is hit and the npm ci install step takes seconds instead of a minute. When you do update dependencies, the lockfile changes, the cache key changes, and a fresh install populates the new cache.

For teams with monorepos or custom cache locations, the manual actions/cache@v4 approach gives you full control:

- name: Cache npm dependencies
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-

For a single-package repo like this one, cache: 'npm' in setup-node is equivalent and cleaner.

Step 4: Building the Site

The build job checks out the code, installs dependencies with npm ci (not npm install - ci respects the lockfile exactly and fails if it's out of sync), runs the production build, and uploads the artifact:

build:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4

    - uses: actions/setup-node@v4
      with:
        node-version: '20'
        cache: 'npm'

    - name: Install dependencies
      run: npm ci

    - name: Build Eleventy site and Tailwind CSS
      run: npm run deploy

    - name: Upload Pages artifact
      uses: actions/upload-pages-artifact@v3
      with:
        path: _site
        retention-days: 1

The retention-days: 1 on the artifact upload is the cleanup fix. The artifact only needs to survive long enough for the deploy job to consume it in the same workflow run - typically minutes. After that it has no value. The default is 90 days. For a blog with regular publishing, that accumulates fast against your GitHub storage quota. One day is the right number here.

Step 5: Deploying with Environment Protection

The deploy job is where the environment gate comes in:

deploy:
  needs: build
  runs-on: ubuntu-latest
  environment:
    name: github-pages
    url: ${{ steps.deployment.outputs.page_url }}
  permissions:
    pages: write
    id-token: write
  steps:
    - name: Deploy to GitHub Pages
      id: deployment
      uses: actions/deploy-pages@v4

The environment: block does two things. First, it connects this job to a GitHub Environment - a named deployment target that can be configured with protection rules. Second, the url: output from actions/deploy-pages is automatically surfaced in the GitHub UI, linked from the deployment entry in the Actions run.

The permissions here are scoped to this job only: pages: write and id-token: write. The top-level permissions for the workflow are set to contents: read. The build job never gets write access to Pages; the deploy job never gets more than it needs. This is the principle of least privilege applied where it's cheapest - YAML.

Configuring the GitHub Environment

The environment protection rules live in the GitHub UI, not the workflow file. Navigate to Repository Settings → Environments → New environment and name it github-pages.

From there, the two most useful controls:

Required reviewers: add one or more people who must approve the deployment before the job proceeds. When a deployment is pending approval, the deploy job pauses and GitHub sends a notification to the reviewers. The workflow waits - your site doesn't go live until someone explicitly approves it.
Deployment branch filter: restrict deployments to the main branch. This prevents accidental deploys from feature branches even if someone triggers a workflow_dispatch from the wrong ref.

For a personal site or solo project, required reviewers may be more friction than value. The deployment branch filter alone is a meaningful improvement - it eliminates the category of "I accidentally ran this from a branch that wasn't ready."

The Complete Workflow

All of it assembled:

name: Deploy to GitHub Pages

on:
  push:
    branches:
      - main
  workflow_dispatch:

permissions:
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Build Eleventy site and Tailwind CSS
        run: npm run deploy

      - name: Upload Pages artifact
        uses: actions/upload-pages-artifact@v3
        with:
          path: _site
          retention-days: 1

  deploy:
    needs: build
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    permissions:
      pages: write
      id-token: write
    steps:
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v4

A few design decisions worth calling out explicitly:

Two-job structure. build produces the artifact; deploy consumes it. If build fails, deploy never runs - there is no path from a broken build to a live deployment. The jobs are cleanly separated and could run on different runner types if needed.

workflow_dispatch. Allows manual triggering from the GitHub Actions UI, useful for redeploying after a config change, an environment tweak, or any situation where you want to redeploy without committing a change to main.

Top-level permissions: contents: read. This is the floor. Every job in this workflow inherits it unless they declare their own permissions block. The deploy job adds pages: write and id-token: write at the job level - those permissions exist for that job only, not for build.

npm ci not npm install. Reproducible installs, lockfile-enforcing. If package-lock.json diverges from package.json, npm ci fails loudly instead of silently mutating the install.

PR Preview Deployments

GitHub Pages doesn't natively support per-PR preview URLs. If that's a requirement, two options:

Cloudflare Pages or Netlify: connect your repository and they handle PR preview URLs automatically, with zero workflow changes on your end. Each PR gets its own preview URL, and it tears down when the PR closes. For most teams, this is the right answer.

Custom approach within GitHub Pages: deploy to a path-prefixed URL per PR number on a separate branch, managed through workflow logic. More engineering work, stays entirely within GitHub, no third-party dependency. Worth it if GitHub Pages is a hard constraint; not worth it otherwise.

GitHub Pages Deployment Checklist

[ ] Set Pages source to GitHub Actions in Repository Settings - not a branch
[ ] Use actions/setup-node with cache: 'npm' - eliminates 60-90 seconds of install time on unchanged deps
[ ] Run npm ci not npm install - reproducible, lockfile-respecting installs; fails loudly on lockfile drift
[ ] Use npm run deploy (not parallel dev scripts) - Eleventy must build _site/ before build:css can write into it
[ ] Set retention-days: 1 on the Pages artifact - it only needs to survive until the deploy job runs in the same workflow
[ ] Set top-level permissions: contents: read; add pages: write + id-token: write only in the deploy job
[ ] Create a github-pages Environment with a deployment branch filter set to main
[ ] Add required reviewers to the Environment if the site is anything beyond a personal project
[ ] Add workflow_dispatch - allows redeployment without a code change

The gap between "it works" and "it's production-grade" for GitHub Pages is surprisingly small. Caching, least-privilege permissions, a one-day artifact lifecycle, and a deployment environment that can be gated - none of these are complex changes. Together they cut deploy time noticeably, close the OIDC security gap, and give you the ability to stop a bad deploy before it reaches users. For a personal blog or a small team site, this workflow is the right baseline - not over-engineered, but not leaving the obvious improvements on the table either.

Questions about GitHub Actions deployment pipelines, or want help adapting this for a monorepo or a different static site generator? Reach out.

steve.kaschimer@slalom.com

Trunk-Based Development in Practice: What They Don't Tell You

2026-03-20T00:00:00Z

The internet has no shortage of "trunk-based development is better than GitFlow" hot takes. They're not wrong, but they're not useful either. Teams read the post, nod along, rename their develop branch to main, and wonder two sprints later why nothing has changed. The abstract argument isn't the hard part. The hard part is the prerequisites - the tooling and cultural wiring that has to be in place before TBD actually works. Nobody writes about those.

So let's do that instead.

Why the Research Points Here

Trunk-based development (TBD) is the practice of integrating code to a shared mainline frequently - at minimum daily, ideally multiple times a day - rather than maintaining long-lived feature or release branches. It sounds simple. The implications are not.

In Accelerate (Nicole Forsgren, Jez Humble, Gene Kim), the authors analyzed four years of DORA survey data spanning thousands of organizations and found that trunk-based development is one of a small cluster of technical practices that statistically separates elite software delivery performers from everyone else. Elite performers - the cohort deploying on demand, with lead times under an hour and change failure rates under 15% - almost universally practice TBD. It shows up alongside continuous integration, comprehensive test automation, and loosely coupled architecture as a predictor of both delivery throughput and stability.

"High performers were more likely to practice trunk-based development, have fewer than three active branches, and merge to trunk daily." - Accelerate, Forsgren et al.

The data point that tends to surprise people: TBD is correlated with both speed and reliability. The instinct is to assume that committing often to a shared branch increases instability. The research says the opposite. Long-lived branches accumulate integration debt that gets paid - with interest - at merge time. The longer you wait to integrate, the more expensive it gets.

That's the theory. Here's what it takes to actually do it.

What TBD Actually Requires

Feature Flags as a First-Class Citizen

The most common objection to TBD is: "What do we do with work that isn't ready for production?" The answer is feature flags, and if you don't have them, you don't have TBD - you have wishful thinking.

The model is simple: code that isn't ready for users still ships to production. It just ships behind a flag that keeps it dark. This decouples deployment (getting code onto servers) from release (exposing it to users). Once that mental model clicks, a lot of the fear around TBD dissolves.

Not all flags are the same. There are three types worth distinguishing:

Release toggles are long-lived flags that gate an unreleased feature. They're the most common, and the most abused.
Ops toggles are runtime switches - circuit breakers, kill switches for expensive features under load. These have a legitimate long lifespan.
Experiment toggles are A/B test controls. They're tied to a hypothesis with a defined end date.

A minimal flag pattern doesn't require LaunchDarkly or a feature management platform. A config value or environment variable will do for early-stage work:

// config.ts
export const flags = {
  newCheckoutFlow: process.env.FEATURE_NEW_CHECKOUT === "true",
};

// checkout.ts
import { flags } from "./config";

function renderCheckout(user: User) {
  if (flags.newCheckoutFlow) {
    return renderNewCheckout(user);
  }
  return renderLegacyCheckout(user);
}

The pattern is trivial. The discipline is not. Flag lifecycle is where teams get into trouble. Flags accumulate. Developers ship behind a flag, the feature launches, and the flag never gets removed. Six months later you have 40 flags controlling behavior that shipped a year ago, and nobody is confident about what happens if you toggle one. Treat flags like debt: every flag you create should have a removal ticket filed the day it ships to production. Make "remove old flags" a recurring part of your sprint.

Database Migrations Without Long-Lived Branches

Schema changes are the hardest part of TBD to get right, and the one most tutorials skip. The problem is classic: you need to rename a column, but the current production code still reads the old column name. If you deploy the migration before the application code, production breaks. If you merge the application code first, it breaks because the column doesn't exist yet. Long-lived branches "solve" this by bundling both changes together - and that solution is exactly what TBD rules out.

The answer is the expand/contract pattern, also called parallel change. Instead of making a breaking schema change in one step, you split it into three phases deployed across separate releases:

Phase 1 - Expand: Add the new column alongside the old one. Deploy application code that writes to both and reads from the old column. At this point, both versions of the code are compatible with the schema.

-- Migration 001: Add the new column (non-breaking)
ALTER TABLE orders ADD COLUMN customer_reference VARCHAR(255);

Phase 2 - Migrate and cut over: Deploy application code that reads from the new column. Run a backfill to populate the new column for existing rows. Both the old and new column still exist - a rollback is still safe.

-- Migration 002: Backfill existing rows
UPDATE orders SET customer_reference = order_ref WHERE customer_reference IS NULL;

Phase 3 - Contract: Once you're confident the new column is correct and the old column is no longer read anywhere in production code, drop it.

-- Migration 003: Drop the old column (safe to run after code is fully deployed)
ALTER TABLE orders DROP COLUMN order_ref;

This is not glamorous, but it is safe. It also means you can deploy at any of these phases independently, which is exactly what TBD demands.

The Minimum CI Gate

TBD has exactly one non-negotiable: trunk is always deployable. If you can't guarantee that, the whole model breaks down. The mechanism that enforces it is your CI pipeline.

Every commit to main must run your test suite and block merge on failure. That's table stakes. The less obvious constraint is speed. The target is under 10 minutes. This is not arbitrary. When a pipeline takes 30 minutes, developers stop waiting for it. They queue up another change, or they start multitasking, or they just merge and hope. The feedback loop breaks. Small batches accumulate. You're back to GitFlow behavior with a different branch name.

Here's a minimal GitHub Actions workflow that enforces this:

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4

      - name: Set up Node
        uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test

      - name: Run lint
        run: npm run lint

The timeout-minutes: 10 is doing real work here - it enforces the discipline in code, not just policy. If your test suite is already over 10 minutes, parallelizing test execution and aggressively culling slow integration tests is the first investment you need to make before TBD is viable.

Short-Lived Branches (If You Use Branches at All)

TBD does not require that every developer commits directly to main. Short-lived feature branches with pull requests are fine - and for most teams, preferable. The rule is: a branch that lives longer than one day is a risk. A branch that lives longer than a week is a problem.

The target is branches that represent a few hours of work, get reviewed, and merge the same day. When a task is genuinely larger than that, the skill to develop is decomposition - breaking the work into independently mergeable slices, each behind a feature flag if needed.

Stacked PRs are a technique worth knowing here. Instead of one massive PR that touches the data layer, API layer, and UI, you create three PRs where each one builds on the previous. PR 1 merges first. PR 2 is rebased on top of it. PR 3 is rebased on PR 2. Each is small and reviewable. The stack merges in order over the course of a day. This is how you do large changes without long-lived branches.

How to Talk Your Team Out of GitFlow

Don't argue abstractions. "Trunk-based development has better research support" will not move anyone who has spent three years on a team where GitFlow worked fine. Argue consequences.

Long-lived branches create merge conflicts. Merge conflicts are not a technical nuisance - they are lost time, and they compound. A branch that was one day of work becomes two days when you factor in the merge and the re-testing.

GitFlow's release branch is solving the wrong problem. The release/2.4.1 branch exists to stabilize code before it ships. TBD solves the same problem differently: with a CI pipeline that keeps main stable, and feature flags that let you exclude unready work. The stabilization is continuous, not batch.

The hotfix question. Teams always ask this one: "What about hotfixes? We need a way to patch production without shipping everything in develop." This is a legitimate scenario. TBD handles it better, not worse. If main is always deployable, a hotfix is just: commit the fix to main, deploy. There's no hotfix/ branch to create, no cherry-pick into develop, no cherry-pick into main. The ceremony GitFlow adds for hotfixes is ceremony that only exists because GitFlow made the process complicated in the first place.

The migration path. Don't try to flip a team from GitFlow to TBD overnight. Start with one metric: branch lifetime. Track how long the average branch lives from creation to merge. Make it visible. Set a goal. Start pushing toward same-day merges. That single habit change will surface all the tooling gaps - missing feature flags, slow pipelines, large PRs - and give you a concrete agenda for fixing them. Branch lifetime is the leading indicator for everything else.

The Minimum GitHub Setup for TBD

The tooling that enforces TBD practices in GitHub is branch protection rules (or the newer rulesets for organizations). Here's the minimum configuration that makes the model work:

# Equivalent repository ruleset (GitHub API / terraform-github-provider)
ruleset:
  name: "Trunk Protection"
  target: branch
  enforcement: active
  conditions:
    ref_include: ["~DEFAULT_BRANCH"]
  rules:
    - type: required_status_checks
      parameters:
        strict_required_status_checks_policy: true  # branch must be up to date
        required_status_checks:
          - context: "CI / test"
          - context: "CI / lint"
    - type: pull_request
      parameters:
        required_approving_review_count: 1
        dismiss_stale_reviews_on_push: true
    - type: non_fast_forward          # no force-pushes to main
    - type: deletion                  # can't delete main

If you prefer GitHub UI, the key settings are: Require status checks to pass before merging, Require branches to be up to date before merging, and Require a pull request before merging. Enable Automatically delete head branches at the repository level to keep the branch list clean.

One opinion worth taking: include administrators in the restriction. The "bypass for admins" escape hatch gets used. When it does, it undermines the trust the CI gate is supposed to build. If the trunk is always deployable, there's no reason admins need to bypass it.

TBD Readiness Checklist

Use this to assess whether your team has the prerequisites in place before making the switch:

[ ] CI pipeline completes in under 10 minutes - if not, parallelization is the first project
[ ] Feature flags exist for in-progress or unreleased work - code ships dark
[ ] Database migrations follow the expand/contract pattern - no single-step breaking changes
[ ] Branches are deleted within 24 hours of creation - track this as a team metric
[ ] Every merge to main triggers a deployment (to at least a staging environment)
[ ] Developers are comfortable committing incomplete work behind a flag - this is the cultural shift

If more than two of these are unchecked, start there before changing your branching strategy. The tools have to be in place before the practice is safe.

The One Thing to Do First

TBD isn't hard because of Git. Git is fine. It's hard because it exposes every gap in your delivery pipeline and makes every cultural shortcut visible. Teams that succeed treat it as an engineering practice with prerequisites - not a branching strategy you adopt by announcing it in a team meeting.

If you're starting from GitFlow, the single change with the most leverage is this: stop creating branches that last more than a day. Not as a rule you enforce immediately, but as a target you start measuring toward. That one constraint will surface the flag infrastructure you need, the pipeline speed you're missing, and the decomposition skills your team hasn't had to develop yet. Fix those, and the rest follows.

The research is clear on where this leads. The path there is less a strategy swap and more an engineering discipline you build one merged PR at a time.

Want to talk through a TBD migration for your team, or figure out where to start? Reach out.

steve.kaschimer@slalom.com

The GitHub Actions `permissions` Block: Principle of Least Privilege for Workflows

2026-03-25T00:00:00Z

Every time a GitHub Actions workflow runs, GitHub provisions a GITHUB_TOKEN automatically - a short-lived credential scoped to the repository. You don't create it, rotate it, or store it as a secret. It just appears. What most developers don't realize is what that token can do by default: write to repository contents, open and merge pull requests, push packages, create deployments, manage releases, and more. All of it, unless you say otherwise. The default exists because GitHub designed it for ease of adoption - get a workflow running without thinking about permissions. That's reasonable for a first prototype. It's a real problem for anything that runs in production.

The attack surface is concrete. A compromised dependency in a build step. A malicious action injected through a supply-chain attack. A command injection vulnerability in an untrusted PR title. Any of these can use the workflow's default GITHUB_TOKEN to read secrets, push code, or overwrite a release. Not because the workflow was misconfigured. Because the default is permissive and nobody added the permissions block.

The fix is three to six lines of YAML. The return on investment is not subtle.

What the Default Permissions Actually Are

By default, when the permissions key is absent from a workflow, GitHub Actions grants write access to most token scopes when the workflow is triggered by an event on the default branch. Workflows triggered by pull requests from forks get read-only by default - but that's a different default, and it applies only to that specific case.

Here are the actual scopes that GITHUB_TOKEN receives when you don't specify a permissions block:

Scope	Default (non-fork)	Description
`actions`	write	Manage workflow runs
`checks`	write	Create and update check runs
`contents`	write	Read/write repo contents, create commits and branches
`deployments`	write	Create deployments
`id-token`	none	Request OIDC tokens - must be explicitly opted in
`issues`	write	Create and update issues
`packages`	write	Push packages to GitHub Packages
`pages`	write	Manage GitHub Pages
`pull-requests`	write	Open, edit, and merge pull requests
`repository-projects`	write	Manage projects
`security-events`	write	Upload SARIF results, manage Dependabot alerts
`statuses`	write	Set commit statuses

Notice id-token: it is the one scope that is not granted by default. Everything else in this table is write-enabled unless you turn it off. A workflow that runs unit tests needs one of these - checks: write to post test results, or sometimes nothing at all. It has all of them.

The practical implication: if your test workflow checks out code, installs dependencies from npm or PyPI, and runs tests, every package in your transitive dependency tree is running code inside a process that holds a token with write access to your repository. That's the blast radius. It exists whether or not anyone intended it.

Workflow-Level vs. Job-Level Permissions

The permissions block can appear at two places in a workflow file. Understanding both is necessary to use it correctly.

Workflow-level permissions sit at the top of the file, under the on: block. They establish a baseline that every job in the workflow inherits unless a job explicitly overrides them:

name: CI
on: [push]

permissions:
  contents: read
  checks: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm test

Job-level permissions sit inside a specific job and override the workflow baseline for that job only. This lets different jobs in the same workflow operate with different scopes:

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      checks: write
    steps:
      - uses: actions/checkout@v4
      - run: npm test

  deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pages: write
      id-token: write
    steps:
      - uses: actions/deploy-pages@v4

The correct pattern for any workflow with more than one job - or any workflow where you care about security at all - is to set permissions: {} at the workflow level and then declare exactly what each job needs at the job level:

name: CI
on: [push]

permissions: {}  # zero baseline - every job must declare what it needs

jobs:
  test:
    permissions:
      contents: read
      checks: write
    runs-on: ubuntu-latest
    steps:
      ...

  deploy:
    permissions:
      pages: write
      id-token: write
    runs-on: ubuntu-latest
    steps:
      ...

The empty object {} grants zero permissions. Any job added later starts with nothing and will fail visibly in CI if it uses a token operation it hasn't been granted. That failure in CI is strictly preferable to silently holding permissions that were never intended.

Three Real Workflow Scenarios

Scenario 1: Run Tests and Post Results

A test workflow needs two things: to read the repository code (contents: read) and to post check results (checks: write). That's the complete list.

name: Test
on: [push, pull_request]

permissions: {}

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      checks: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm test

What this workflow does not have: write access to repository contents, issues, pull requests, packages, or anything else. A compromised dependency in npm ci or npm test cannot push a commit, open a PR, or modify a release with this configuration. The blast radius is contained to the job's declared scope.

Scenario 2: Comment on a Pull Request

A workflow that posts a comment on a PR - a code coverage summary, a preview URL, a diff report - needs pull-requests: write. It still does not need contents: write.

name: Coverage Report
on: pull_request

permissions: {}

jobs:
  coverage:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run test:coverage
      - uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: '## Coverage: 94.2%'
            })

One thing worth calling out explicitly: commenting on a pull request uses pull-requests: write, not issues: write. Pull requests and issues share an API in GitHub - a PR is technically an issue - but they are separate GITHUB_TOKEN scopes. Grant only pull-requests: write; issues: write gives the workflow access to create and modify issues across the repository.

Scenario 3: Deploy to GitHub Pages with OIDC

This scenario requires the most permissions, which makes it the most important one to scope correctly. A misconfigured deploy workflow with an overly broad GITHUB_TOKEN can modify branches, overwrite releases, or interact with packages - none of which a Pages deployment needs.

The correct approach splits build and deploy into separate jobs, each with only what it needs:

name: Deploy
on:
  push:
    branches: [main]

permissions: {}

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run build
      - uses: actions/upload-pages-artifact@v3
        with:
          path: _site/

  deploy:
    needs: build
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deploy.outputs.page_url }}
    permissions:
      pages: write
      id-token: write
    steps:
      - uses: actions/deploy-pages@v4
        id: deploy

The id-token: write scope deserves special attention here. It is the one scope in the permissions table that is not granted by default and must be explicitly declared. It authorizes the workflow to request an OIDC token from GitHub - the short-lived, keyless credential used for authentication with GitHub Pages and cloud providers. Without id-token: write, OIDC-based deployments fail. The error messages are not always clear about why. When a Pages or cloud deploy workflow silently fails to authenticate, the missing id-token: write permission is the first thing to check.

The `permissions: {}` Pattern - Zero Baseline

There are three ways to handle the workflow-level permissions block, and they are not equivalent:

# Inherits GitHub's permissive defaults - write access to almost everything
name: Dangerous Workflow
on: [push]
# no permissions key

---

# Better - grants read access to all scopes; still broader than necessary
name: Less Dangerous Workflow
on: [push]
permissions: read-all

---

# Correct - jobs declare exactly what they need, nothing is inherited
name: Correct Workflow
on: [push]
permissions: {}

The read-all shorthand is a common stopping point for teams that know they should restrict permissions but aren't ready to audit each job. It meaningfully reduces the write blast radius. But read access to contents still means any step in the workflow can read the full repository source, read secrets exposed as environment variables via env:, and exfiltrate data to an external endpoint. Read-only is not zero. permissions: {} is zero.

The other reason the zero baseline matters: it makes security visible in code review. When a developer adds a new job that calls softprops/action-gh-release to create a release, and the workflow has permissions: {} at the top, the CI run will fail immediately with a 403. The review conversation becomes "this job needs contents: write to create a release - is that the right tool for this workflow?" instead of "the release job works, ship it." The failure surface in CI is the faster feedback loop.

Organization-Level Defaults

Individual workflow permissions blocks are the most important control - but GitHub also allows setting a default permissions policy at the organization level. Navigate to Settings → Actions → Workflow permissions at the org level and you'll find two options:

"Read and write permissions" - the default for most organizations, grants write access to most scopes
"Read repository contents and packages permissions" - grants read-only by default

Set the org default to read-only. This doesn't replace per-workflow permissions blocks - those override the org default and should still be explicit - but it reduces the blast radius for any workflow file in any repository in the org that is missing its permissions block entirely. In a large organization with dozens of repositories and workflows, that gap is not hypothetical.

For organizations using GitHub Enterprise or GitHub Advanced Security, this setting is often the fastest compliance win available: one checkbox that immediately restricts the default token scope across the entire org, with no workflow changes required.

Auditing Existing Workflows

Before adding permissions blocks to new workflows, it's worth knowing which existing workflows don't have them.

Manual scan - find workflow files with no permissions key:

grep -rL "^permissions:" .github/workflows/

This outputs every workflow file in .github/workflows/ that has no permissions declaration at all. Each result is a workflow running on GitHub's permissive defaults.

Using step-security/harden-runner - for determining what permissions a workflow actually uses before committing to a minimal set:

- uses: step-security/harden-runner@v2
  with:
    egress-policy: audit

Harden-runner logs all outbound network calls and the permissions the workflow actually exercises during a run. Run it in audit mode for a few cycles before adding a permissions block - it tells you the minimal set you need rather than requiring you to read every action's documentation to figure it out.

Using actionlint - static analysis for GitHub Actions workflows:

# Install and run actionlint
brew install actionlint
actionlint .github/workflows/*.yml

actionlint catches a broad range of workflow issues including type mismatches, invalid expressions, and - with the right configuration - jobs without explicit permission declarations. It's the fastest way to get a baseline audit across all workflows in a repository.

Permissions Quick Reference

Use case	Minimum permissions needed
Checkout and build	`contents: read`
Run tests, post check results	`contents: read`, `checks: write`
Comment on a PR	`contents: read`, `pull-requests: write`
Create a release	`contents: write`
Push to GitHub Packages	`packages: write`
Deploy to GitHub Pages (OIDC)	`pages: write`, `id-token: write`
Upload SARIF to code scanning	`security-events: write`
Request OIDC token (cloud deploy)	`id-token: write`

Key rules:

Set permissions: {} at the workflow level as a zero baseline
Grant only what each job needs, declared at the job level
Set "Read repository contents" as the org-level default in Actions settings
id-token: write is never granted by default - always declare it explicitly
Add step-security/harden-runner in audit mode to discover actual permissions used before writing your permissions block
Run grep -rL "^permissions:" .github/workflows/ to find workflows still on GitHub defaults

The permissions block is three lines of YAML that meaningfully reduces the attack surface of every workflow that includes it. It doesn't require a security team, a policy review, or a platform migration. It requires looking at what each job actually does, mapping that to the minimum set of scopes, and writing it down. GitHub's defaults were designed for ease of adoption - get something running without friction. The permissions block is how you opt out of that tradeoff once the workflow is running in production. That's the right time to do it, which means the right time is now.

Want to talk through permissions strategy for your workflows, or work through a permissions audit for your GitHub Actions setup? Reach out.

steve.kaschimer@slalom.com

Dependabot Advanced: Getting Past the Noise

2026-03-27T00:00:00Z

Here's how most Dependabot stories end: the team enables it, a flood of PRs appears, nobody has time to review 40 dependency bumps, the PRs age into staleness, and eventually someone closes them all in bulk and adds Dependabot to the list of things that sounded good in theory. Sometimes they disable it outright. Sometimes they just stop looking.

The tool isn't broken. The configuration is. Dependabot out of the box is optimized for coverage - it will find every update and open a PR for it. What it is not optimized for is human attention. The default config fires daily, creates one PR per package per version bump, treats a patch bump to a dev-only type package the same as a major version change to your HTTP client, and sets a low cap on open PRs that triggers a silent failure mode most teams don't even know exists. Every one of those choices is tunable. Two hours of configuration work will cut your PR volume by 70% while keeping security updates fast and individual. This post walks through exactly how to do that.

What You Get by Default

A repo with npm, Docker, and GitHub Actions dependencies needs exactly three lines of configuration to enable Dependabot:

# .github/dependabot.yml (default)
version: 2
updates:
  - package-ecosystem: "npm"
    directory: "/"
    schedule:
      interval: "daily"
  - package-ecosystem: "docker"
    directory: "/"
    schedule:
      interval: "daily"
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "daily"

For a medium-sized project - 50 npm dependencies, two or three Docker base images, a handful of GitHub Actions - the first week will produce somewhere between 20 and 50 PRs. If you haven't updated dependencies in a few months, that number can spike higher. Each PR is a single package bump, unreviewed, with a title like Bump @types/node from 20.11.0 to 20.11.5 that carries no signal about whether it matters.

The problem compounds. A daily schedule means any upstream package that releases a new version today will generate a new PR tomorrow. For active ecosystems like npm, that's not occasional - it's continuous. eslint, typescript, @types/*, React ecosystem packages - they release constantly. Without grouping or scheduling discipline, you're running a low-grade interrupt loop that trains your team to ignore the PRs entirely.

That's the configuration problem. Here's how to fix it.

The Tuning Levers

Scheduling - Weekly Is the Right Default

Daily updates are wrong for most teams. Not because freshness doesn't matter - it does - but because daily creates a pace of review that no team actually sustains. The right default is weekly, and the right day is Monday morning.

Monday gives your team a clean start with a predictable batch of updates. Friday is actively bad - Dependabot PRs that open on a Friday sit over the weekend and nobody is happy about merging an untested dependency bump before heading out. Monday morning also means the team can merge, run CI, and have time in the same week to deal with anything unexpected.

One important caveat: security updates bypass the schedule entirely. When Dependabot detects a vulnerability in a dependency, it opens a PR immediately, regardless of what you've set for interval. Switching to weekly does not slow down your response to known CVEs. This distinction matters and it's one of the most common misconceptions about tuning the schedule.

schedule:
  interval: "weekly"
  day: "monday"
  time: "09:00"
  timezone: "UTC"

Grouping - The Single Highest-Impact Change

Without grouping, every package gets its own PR. With grouping, related packages are bundled into a single PR. This is the lever that most dramatically reduces PR volume.

The syntax is straightforward. You define named groups with a pattern that matches package names, and optionally constrain them to specific update types:

groups:
  dev-dependencies:
    dependency-type: "development"
    update-types:
      - "minor"
      - "patch"
  aws-sdk:
    patterns:
      - "@aws-sdk/*"
      - "aws-cdk*"
  eslint-plugins:
    patterns:
      - "eslint*"
      - "@typescript-eslint/*"

With this configuration, all your dev dependencies (patch and minor) arrive in a single PR. All your @aws-sdk/* packages - and there can be dozens - arrive in one PR. All your ESLint toolchain packages arrive together. Instead of 15 PRs for dev tooling updates, you get one.

Two things worth knowing about how grouping interacts with security updates. First: security updates are excluded from groups by default. When Dependabot opens a PR for a known vulnerability, it opens it as an individual PR regardless of whether the package matches a group. This is the correct behavior - you want security PRs to be fast, individual, and easy to track. Don't fight this. Second: ungrouped packages still get individual PRs, so you're not forced to bucket everything - you can be selective about which families you group.

For GitHub Actions, grouping by patch updates keeps your action version noise low while letting major version changes - which sometimes involve breaking API changes - surface individually:

# For the github-actions ecosystem
groups:
  actions-minor-patch:
    update-types:
      - "minor"
      - "patch"

Versioning Strategy

Dependabot offers three versioning strategies for npm. The difference matters for lockfile-only projects versus projects that also manage package.json version ranges:

lockfile-only: Only updates package-lock.json or yarn.lock. Does not change version ranges in package.json. Useful if you want strict control over what you've declared, but it means Dependabot can't update packages that don't already satisfy the current range.
increase-if-necessary: Updates the version range in package.json only when the new version falls outside the current range. This is the right default for most projects - it keeps your declared ranges honest without aggressively bumping them.
widen: Widens the version range to include both the old and new version. Creates permissive ranges that can hide what version is actually running.

For most npm projects, increase-if-necessary is the right call. For GitHub Actions, it's also the right default - though if you're serious about supply chain security, pinning Actions to full commit SHAs and using Dependabot to update those pins is a stronger posture (that's worth its own post).

versioning-strategy: increase-if-necessary

Allowed and Ignored Updates

Sometimes you explicitly don't want Dependabot touching a specific package. Maybe you're in the middle of migrating away from it. Maybe a known-broken major version exists and you're not ready to upgrade. The ignore directive handles this:

ignore:
  # Hold on major version bumps for webpack until we migrate config
  - dependency-name: "webpack"
    update-types: ["version-update:semver-major"]
  # This package has a broken v3.x release; skip it entirely for now
  - dependency-name: "some-library"
    versions: ["3.x"]

For monorepos with a mix of internal and external packages, allow lets you whitelist just the external dependencies you actually want Dependabot to manage, which prevents it from opening PRs for internal workspace packages:

# Option A: only direct dependencies (production + dev, no transitive)
allow:
  - dependency-type: "direct"

# Option B: only production dependencies (direct + transitive, no dev)
allow:
  - dependency-type: "production"

The PR Limit Silent Failure

This one deserves special emphasis because it creates a failure mode most teams don't know about until they're already affected.

Dependabot has an open-pull-requests-limit that defaults to 5. Once 5 Dependabot PRs are open in a repo, Dependabot stops opening new ones - silently. No notification, no warning, no dashboard indicator. If you have 6 open PRs and a new vulnerability is discovered in one of your dependencies, Dependabot will not open a security PR until you reduce your open count below the limit.

This is the exact opposite of what you want from a security tool.

The fix is to set the limit explicitly and high enough to accommodate your grouping strategy:

open-pull-requests-limit: 10

If you've enabled grouping, your actual PR count should stay low enough that 10 is comfortable. But set it explicitly regardless - relying on the default means accepting a silent ceiling you might not notice until it matters.

Auto-merge for Low-Risk Updates

Grouping and scheduling reduce review volume, but the further optimization is auto-merge for updates that are genuinely low-risk. A companion GitHub Actions workflow can merge Dependabot patch and minor PRs automatically once CI passes:

# .github/workflows/dependabot-automerge.yml
name: Dependabot Auto-merge
on: pull_request

permissions:
  contents: write
  pull-requests: write

jobs:
  auto-merge:
    runs-on: ubuntu-latest
    if: github.actor == 'dependabot[bot]'
    steps:
      - name: Fetch Dependabot metadata
        id: metadata
        uses: dependabot/fetch-metadata@v2
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
      - name: Auto-merge patch and minor updates
        if: |
          steps.metadata.outputs.update-type == 'version-update:semver-patch' ||
          steps.metadata.outputs.update-type == 'version-update:semver-minor'
        run: gh pr merge --auto --squash "$PR_URL"
        env:
          PR_URL: ${{ github.event.pull_request.html_url }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

gh pr merge --auto queues the merge but does not bypass branch protection. The PR still needs to pass all required status checks before it merges. If CI fails, nothing merges - auto-merge just removes the human step of pressing the button on PRs that would obviously have been approved anyway.

For teams with strict review requirements, you can scope auto-merge to patch-only and require human review for minor bumps. The key decision is comfort level: patches are usually safe to auto-merge; minor versions occasionally introduce behavioral changes that warrant a look.

The Complete Tuned Configuration

Here's the full dependabot.yml that incorporates all of the above. This is the config I'd start with for a real project and adjust from there:

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "npm"
    directory: "/"
    schedule:
      interval: "weekly"
      day: "monday"
      time: "09:00"
      timezone: "UTC"
    versioning-strategy: increase-if-necessary
    open-pull-requests-limit: 10
    groups:
      dev-dependencies:
        dependency-type: "development"
        update-types:
          - "minor"
          - "patch"
      aws-sdk:
        patterns:
          - "@aws-sdk/*"
          - "aws-cdk*"
      eslint-plugins:
        patterns:
          - "eslint*"
          - "@typescript-eslint/*"
    ignore:
      # Hold major version bumps on webpack pending config migration
      - dependency-name: "webpack"
        update-types: ["version-update:semver-major"]

  - package-ecosystem: "docker"
    directory: "/"
    schedule:
      interval: "weekly"
      day: "monday"
      time: "09:00"
      timezone: "UTC"
    open-pull-requests-limit: 10

  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"
      day: "monday"
      time: "09:00"
      timezone: "UTC"
    open-pull-requests-limit: 10
    groups:
      actions-minor-patch:
        update-types:
          - "minor"
          - "patch"

What this produces in practice: one grouped PR per week for dev dependencies, one for AWS SDK packages (if applicable), one for ESLint plugins, individual PRs for production dependency minor and major bumps, a grouped PR for action patches and minor versions, individual PRs for action major versions, and immediate individual PRs for any security advisory. The average team running this configuration sees 3-5 Dependabot PRs per week instead of 15-50.

When to Reach for Renovate Instead

Dependabot is the right default. It's zero-config to enable, deeply integrated with GitHub's security features, and handles most repos perfectly well with the tuning above. But it has real limitations worth knowing about.

No monorepo workspace awareness. Dependabot doesn't understand npm workspaces natively. It will open PRs for the root package.json and for each workspace's package.json independently, without understanding that some of those packages are internal workspace references that shouldn't be bumped. Renovate handles workspace topology and won't create PRs for internal packages.

No custom regex versioning. Renovate can extract version strings from arbitrary files - a Dockerfile with a custom ARG VERSION=1.2.3 pattern, a .tool-versions file, a Makefile constant. Dependabot is limited to the ecosystems it officially supports. If your infrastructure tooling version lives somewhere outside those ecosystems, Dependabot can't see it.

No Dependency Dashboard. Renovate creates a single "Dependency Dashboard" issue in the repo - a living document that shows every pending update, every pending decision, every ignored package, and every rate-limited PR in one place. For large repos, this is dramatically better UX than navigating a list of PRs in varying states of staleness. Dependabot has no equivalent.

More flexible grouping. Dependabot's grouping handles the common cases well, but Renovate's grouping rules are more expressive - you can group across ecosystems, apply regex to version strings, and build more complex rules for large monorepos.

The signal for switching: if you find yourself writing complicated ignore chains and still fighting the tool, or if your repo is a multi-package workspace, try Renovate. If Dependabot's grouping handles your repo's structure and you're not managing version strings outside supported ecosystems, stay - it's one YAML file and no additional setup.

Dependabot Tuning Checklist

Apply these today to cut PR volume without slowing down security response:

[ ] Switch schedule from daily to weekly, targeting Monday morning
[ ] Add groups for dev dependencies and any major SDK families (AWS SDK, testing frameworks, ESLint)
[ ] Set open-pull-requests-limit explicitly to 10 or higher - the default 5 creates a silent failure
[ ] Add the auto-merge workflow for patch and minor updates (gated on CI)
[ ] Add ignore rules for any known-broken version ranges or packages under active migration
[ ] Verify that security updates are not in groups - they shouldn't be by default, but confirm it
[ ] Merge, close, or label all existing stale Dependabot PRs before the new config takes effect
[ ] Review open Dependabot PRs monthly - a backlog is a signal, not a to-do list

Closing Thoughts

The teams that ignore Dependabot PRs aren't lazy. They're dealing with a configuration problem that the default setup actively creates. A flood of low-signal PRs trains teams to stop looking, and once that habit forms it takes real effort to undo.

The tuning described here - weekly schedule, dependency grouping, explicit PR limits, auto-merge for low-risk updates - converts Dependabot from a PR flood into a low-maintenance practice that actually runs in the background of your team's week. The security updates still arrive immediately. The housekeeping updates arrive in manageable batches. Auto-merge handles the ones that don't need eyes. The remaining PRs that reach your review queue are the ones that actually warrant attention.

That's what a correctly configured Dependabot looks like. It takes about two hours to set up, and it changes the relationship from "thing we're ignoring" to "thing that quietly keeps our dependencies current."

Have questions about Dependabot configuration, supply chain security, or whether Renovate is the right call for your repo? Reach out.

steve.kaschimer@slalom.com

Tailwind CSS v4: What Actually Changed and How to Migrate

2026-04-01T00:00:00Z

Tailwind v4 isn't a config syntax refresh with a migration codemod attached. It's a rewritten engine - Oxide, built in Rust - that changes how configuration works, how CSS is generated, how plugins are authored, and how the CLI operates. The headline benchmarks (full builds 5× faster, incremental builds 100×+ faster) are real, but the migration isn't purely mechanical. For developers with custom color palettes, class-based dark mode, or typography plugin overrides, there are breaking changes the codemod doesn't handle.

This post walks through what actually changed, migrates this blog's real v3 tailwind.config.js to v4 line by line, and flags the three breaking changes most likely to catch you off-guard. The migration is manageable - under an hour for a typical Eleventy blog - but you need to know what you're walking into.

What the Engine Change Means

The first thing to understand is that tailwind.config.js isn't just changing syntax - it's going away. Configuration moves into CSS using @theme, @utility, and @variant directives. The JS file is replaced by a CSS entry point that becomes the single source of truth for everything previously split between the config file and your CSS.

Four changes that affect every project:

No more tailwind.config.js: everything moves to CSS. The @tailwindcss/upgrade codemod generates a starter @theme block from your existing config, but complex customizations need manual migration.
No more content array: v4 uses automatic content detection. It scans your project for Nunjucks, HTML, Markdown, and JS files automatically. The explicit content: ['./src/**/*.{html,md,njk,js}'] entry is no longer needed - though if you have non-standard locations or extensions, @source provides an explicit escape hatch.
@tailwindcss/cli replaces tailwindcss for CLI invocations: any npx tailwindcss call in your build scripts becomes npx @tailwindcss/cli.
@tailwindcss/postcss replaces tailwindcss as the PostCSS plugin package name, if you're using PostCSS.

The Oxide engine is written in Rust. The 5× full build and 100×+ incremental build improvements are from the Tailwind team's own benchmarks. For an Eleventy site running Tailwind as a separate build step, the incremental build gain is what you'll feel on every file save during local development.

The v3 Config: What We're Starting From

This blog's tailwind.config.js is a representative v3 config - a custom color scale, class-based dark mode, the typography plugin, and prose variable overrides:

/** @type {import('tailwindcss').Config} */
module.exports = {
  content: [
    "./src/**/*.{html,md,njk,js}",
  ],
  darkMode: 'class',
  theme: {
    extend: {
      colors: {
        primary: {
          50: '#f0f9ff',
          100: '#e0f2fe',
          200: '#bae6fd',
          300: '#7dd3fc',
          400: '#38bdf8',
          500: '#0ea5e9',
          600: '#0284c7',
          700: '#0369a1',
          800: '#075985',
          900: '#0c4a6e',
        },
      },
      typography: ({ theme }) => ({
        DEFAULT: {
          css: {
            '--tw-prose-body': theme('colors.gray[700]'),
            '--tw-prose-headings': theme('colors.gray[900]'),
            '--tw-prose-links': theme('colors.primary[600]'),
            '--tw-prose-bold': theme('colors.gray[900]'),
            '--tw-prose-code': theme('colors.gray[900]'),
            '--tw-prose-pre-bg': theme('colors.gray[100]'),
          },
        },
        invert: {
          css: {
            '--tw-prose-body': theme('colors.gray[300]'),
            '--tw-prose-headings': theme('colors.white'),
            '--tw-prose-links': theme('colors.primary[400]'),
            '--tw-prose-bold': theme('colors.white'),
            '--tw-prose-code': theme('colors.white'),
            '--tw-prose-pre-bg': theme('colors.gray[800]'),
          },
        },
      }),
    },
  },
  plugins: [
    require('@tailwindcss/typography'),
  ],
}

And the current src/styles/input.css entry point opens with:

@tailwind base;
@tailwind components;
@tailwind utilities;

Those three directives are the first thing to replace. Everything else follows from there.

Migrating to v4: Section by Section

The `content` array → gone

// v3 - delete this block entirely
content: [
  "./src/**/*.{html,md,njk,js}",
],

Auto-detection in v4 covers Nunjucks, HTML, Markdown, and JS without configuration. For a standard Eleventy project with templates in src/, nothing else is needed.

The `theme.extend.colors` block → `@theme`

The custom primary color scale moves from a JavaScript object to CSS custom properties in an @theme block inside input.css:

@import "tailwindcss";

@theme {
  --color-primary-50: #f0f9ff;
  --color-primary-100: #e0f2fe;
  --color-primary-200: #bae6fd;
  --color-primary-300: #7dd3fc;
  --color-primary-400: #38bdf8;
  --color-primary-500: #0ea5e9;
  --color-primary-600: #0284c7;
  --color-primary-700: #0369a1;
  --color-primary-800: #075985;
  --color-primary-900: #0c4a6e;
}

The naming convention is direct: theme.extend.colors.primary[500] becomes --color-primary-500. Every bg-primary-600, text-primary-400, and border-primary-600 in the templates continues to work without touching a single template file.

The `plugins` array → `@plugin`

// v3 - remove this
plugins: [
  require('@tailwindcss/typography'),
],

/* v4 - add to input.css */
@plugin "@tailwindcss/typography";

The require() call is replaced by a @plugin directive. The prose class, prose-sm, prose-lg, prose-invert - all work identically on the consuming side.

Typography theme overrides → direct CSS variables

The typography section of the v3 config is the most nuanced part of this migration. Those --tw-prose-* overrides were resolved at build time using Tailwind's theme() function. In v4, the same variables are still supported by @tailwindcss/typography, but you set them directly in CSS with the resolved hex values:

/* v4: resolved prose color overrides */
.prose {
  --tw-prose-body: #374151;      /* gray-700 */
  --tw-prose-headings: #111827;  /* gray-900 */
  --tw-prose-links: #0284c7;     /* primary-600 */
  --tw-prose-bold: #111827;
  --tw-prose-code: #111827;
  --tw-prose-pre-bg: #f3f4f6;    /* gray-100 */
}

.prose-invert {
  --tw-prose-body: #d1d5db;      /* gray-300 */
  --tw-prose-headings: #ffffff;
  --tw-prose-links: #38bdf8;     /* primary-400 */
  --tw-prose-bold: #ffffff;
  --tw-prose-code: #ffffff;
  --tw-prose-pre-bg: #1f2937;    /* gray-800 */
}

You lose the theme() indirection, but you gain direct CSS that a browser can read without a build tool.

`darkMode: 'class'` → `@variant dark`

This is the breaking change with the most teeth, and the one the codemod silently misses. The darkMode: 'class' option tells v3 to apply dark utilities when a .dark class is present on a parent element. In v4, that moves to CSS:

@variant dark (&:where(.dark, .dark *));

Without this line, all the dark: prefixed classes in the templates - dark:bg-gray-900, dark:text-gray-100, dark:prose-invert - will silently fall back to media-query behavior instead of responding to the .dark class toggled by JavaScript. The pages will still look fine in a system-level dark mode setting. The bug is invisible unless you test with the actual JS toggle.

The migrated `input.css`

Putting it together, the v3 entry point's three directives collapse into a single @import, and all theme configuration moves in:

@import "tailwindcss";
@plugin "@tailwindcss/typography";

@variant dark (&:where(.dark, .dark *));

@theme {
  --color-primary-50: #f0f9ff;
  --color-primary-100: #e0f2fe;
  --color-primary-200: #bae6fd;
  --color-primary-300: #7dd3fc;
  --color-primary-400: #38bdf8;
  --color-primary-500: #0ea5e9;
  --color-primary-600: #0284c7;
  --color-primary-700: #0369a1;
  --color-primary-800: #075985;
  --color-primary-900: #0c4a6e;
}

.prose {
  --tw-prose-body: #374151;
  --tw-prose-headings: #111827;
  --tw-prose-links: #0284c7;
  --tw-prose-bold: #111827;
  --tw-prose-code: #111827;
  --tw-prose-pre-bg: #f3f4f6;
}

.prose-invert {
  --tw-prose-body: #d1d5db;
  --tw-prose-headings: #ffffff;
  --tw-prose-links: #38bdf8;
  --tw-prose-bold: #ffffff;
  --tw-prose-code: #ffffff;
  --tw-prose-pre-bg: #1f2937;
}

@layer base {
  /* ... unchanged ... */
}

@layer components {
  /* ... unchanged ... */
}

tailwind.config.js gets deleted. One fewer JavaScript file in your project root.

The Three Breaking Changes Most Likely to Burn You

1. Dark mode configuration

Already covered above, but worth stating plainly: darkMode: 'class' has no automatic equivalent in v4, and the upgrade codemod does not emit the @variant dark line. If you skip it, your dark mode silently switches from class-based to media-query-based - a behavior change that's invisible in automated tests and only obvious when you manually click the dark mode toggle.

The fix is one line:

@variant dark (&:where(.dark, .dark *));

Put it at the top of input.css, immediately after the @import.

2. Arbitrary value syntax for CSS variables

v4 tightens the arbitrary value parser. The bracket syntax for inline CSS variable references changes:

<!-- v3 -->
<div class="bg-(--color-brand)">

<!-- v4: CSS variable references use parenthesis syntax -->
<div class="bg-(--color-brand)">

The (--variable) syntax replaces [var(--variable)] everywhere. If your templates reference CSS variables inline in Tailwind classes - common for dynamic theming or per-component tokens - this is a targeted find-and-replace across your template files. Run a grep for [var(-- before considering the migration done.

3. Custom screen breakpoints

If your config extends theme.screens, the breakpoints move to @theme:

// v3 tailwind.config.js
theme: {
  extend: {
    screens: { '3xl': '1920px' },
  },
},

/* v4 input.css */
@theme {
  --breakpoint-3xl: 1920px;
}

The subtler issue: v4 adjusts the default breakpoint values slightly. The sm, md, lg, xl, and 2xl values are close to their v3 equivalents but not identical. If your layout uses responsive utilities like md:grid-cols-2 at precise breakpoints and you care about exact pixel boundaries, check the v4 defaults before declaring the migration complete.

The Migration Path

Step 1: Run the codemod

npx @tailwindcss/upgrade

Handles: renaming deprecated utilities, generating a starter @theme block, updating PostCSS config. Does not handle: darkMode: 'class', typography theme() overrides, or arbitrary variable syntax.

Step 2: Install v4 packages

npm install tailwindcss @tailwindcss/cli @tailwindcss/typography

Step 3: Update package.json build scripts

The CLI package name changes from tailwindcss to @tailwindcss/cli. For this blog, that's two script entries:

{
  "scripts": {
    "build:css": "npx @tailwindcss/cli -i ./src/styles/input.css -o ./_site/styles/output.css --minify",
    "watch:css": "npx @tailwindcss/cli -i ./src/styles/input.css -o ./_site/styles/output.css --watch"
  }
}

The build, start, dev, and deploy scripts are unchanged - only the two that invoke the Tailwind CLI directly need updating.

Step 4: Update input.css

Replace the three @tailwind directives with @import "tailwindcss", add @plugin "@tailwindcss/typography", move the theme config in, and add the @variant dark line.

Step 5: Check for the three breaking changes

Confirm @variant dark (&:where(.dark, .dark *)); is present
Grep for [var(-- and update to (-- parenthesis syntax
Verify any custom breakpoint values against v4 defaults

Step 6: Verify the build

npm run build:css

Check the output file size - v4's dead-code elimination is more aggressive, so the output should be at least as small as v3, typically smaller. If you see deprecation warnings, address those before calling it done.

Build Time: What to Expect

For this blog's stack - Eleventy v2 with a moderate number of Tailwind utility classes - the Rust engine should drop cold build time from roughly 2-4 seconds to under a second, and reduce watch mode latency to something effectively instant.

The practical impact on the npm run dev script - which uses npm-run-all --parallel start watch:css to run Eleventy and Tailwind side by side - is that the watch:css process stops being something you wait for. The bottleneck shifts fully to Eleventy's templating and data cascade. That's exactly where you want it; the CSS layer should be invisible overhead, not a noticeable pause.

v4 Migration Checklist

[ ] Run npx @tailwindcss/upgrade first - handles the mechanical parts
[ ] npm install tailwindcss@next @tailwindcss/cli@next @tailwindcss/typography@next
[ ] Replace @tailwind base/components/utilities with @import "tailwindcss" in input.css
[ ] Add @plugin "@tailwindcss/typography" to input.css (replaces the plugins array)
[ ] Add @variant dark (&:where(.dark, .dark *)); - the codemod does not emit this
[ ] Move theme.extend.colors to @theme CSS custom properties
[ ] Resolve typography overrides to actual hex values in .prose and .prose-invert
[ ] Update build scripts: npx tailwindcss → npx @tailwindcss/cli
[ ] Grep for [var(-- and update to (-- parenthesis syntax
[ ] Check any custom breakpoint values against v4 defaults
[ ] Delete tailwind.config.js - if the build passes, you're done

v4 is a better tool. The Rust engine is genuinely faster, the CSS-native config is more coherent than a JavaScript object that mirrors CSS concepts, and automatic content detection eliminates the whole category of "why aren't my classes generating?" debugging sessions. The migration has real rough edges - the darkMode: 'class' gap and the arbitrary value syntax change are both things the codemod won't catch for you. But for an Eleventy blog like this one, the full migration runs under an hour. The codemod handles 80% of it; the remaining 20% is a focused search-and-replace and one line of CSS. The result is a faster build, less config to maintain, and one fewer JavaScript file in your project root.

Working through a v4 migration and hitting something this post didn't cover? Reach out.

steve.kaschimer@slalom.com

Understanding CVSS Scores: A Practical Guide for Developers

2026-04-03T00:00:00Z

Dependabot fires an alert. It says Critical 9.8. The developer drops everything, merges the patch PR, and marks it done - without reading the advisory, without checking whether the vulnerable package is even reachable in their deployment, without asking whether an exploit exists in the wild. The fire drill takes two hours and disrupts the sprint. Or the opposite happens: after the fifteenth Critical alert this month, the developer dismisses it without reading, and a genuinely exploitable vulnerability sits open in a public-facing API for six weeks. Both failures trace back to the same root cause - treating a CVSS score as a verdict rather than a starting point.

The score is not a triage decision. It's a standardized severity estimate calculated against an imaginary worst-case deployment. The number tells you how bad the vulnerability could be in ideal attack conditions. It says nothing about your infrastructure, your network topology, your authentication requirements, or whether a working exploit even exists. Once you understand how the score is constructed, you stop panic-patching on every 9.8 and stop dismissing alerts because you're fatigued. You read the vector string, check your context, and make a call in two minutes instead of two hours.

What CVSS Actually Is

CVSS - the Common Vulnerability Scoring System - is a framework maintained by FIRST (Forum of Incident Response and Security Teams) for communicating the characteristics and severity of software vulnerabilities in a standardized, vendor-neutral way. The current version you'll encounter in practice is CVSS v3.1. A v4.0 spec exists, but the GitHub Advisory Database, NVD (National Vulnerability Database), and most security tooling including Dependabot still report v3.1 scores. That's what this post covers.

CVSS defines three metric groups:

Base Metrics - the intrinsic characteristics of the vulnerability: how it's exploited, what it affects, and how severely. This is the number your tooling shows you. It's static - it doesn't change based on time, patches, or your environment.
Temporal Metrics - how the threat landscape has evolved since disclosure: whether exploit code exists publicly, whether a patch or workaround is available. These change over time and can be applied on top of the Base Score to get a more current picture.
Environmental Metrics - your organization's specific context: whether the affected component is internet-facing, how much you actually care about confidentiality of that data, what compensating controls you have in place.

The Base Score answers: "How bad could this be in the worst possible context?" It does not answer: "How bad is this for my application?"

Most tools show only the Base Score because it's universal - it requires no knowledge of your environment. Environmental and Temporal scores require input your tooling doesn't have. That makes the Base Score useful for comparison across vulnerabilities and useless as a standalone triage signal. It's the beginning of the analysis, not the end.

Decoding the Vector String

Every CVSS score is accompanied by a vector string - a compact, human-readable encoding of all the metrics that produced the score. If you only take one thing from this post, take this: the vector string is where the real information lives. The number is a summary. The string is the data.

Here's a real-world example of a Critical score:

CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H

Score: 10.0 Critical. This is the ceiling - every metric is at its worst. Here's what each component means:

Metric	Code	Value	Meaning
Attack Vector	AV	N (Network)	Exploitable remotely over the network
Attack Complexity	AC	L (Low)	No special conditions required
Privileges Required	PR	N (None)	Attacker needs no authentication
User Interaction	UI	N (None)	No user action needed
Scope	S	C (Changed)	Exploit crosses security boundaries
Confidentiality	C	H (High)	Complete data disclosure possible
Integrity	I	H (High)	Complete data modification possible
Availability	A	H (High)	Complete service disruption possible

Every metric at its most severe produces a 10.0. Now look at what happens when three metrics shift:

CVSS:3.1/AV:N/AC:H/PR:H/UI:R/S:U/C:H/I:H/A:H

The Confidentiality, Integrity, and Availability impacts are identical. The potential damage ceiling is the same. But AC:H means the attacker needs specific, non-default conditions to land the exploit - a race condition, a particular configuration, a timing window. PR:H means they need admin-level credentials on the target system first. UI:R means a legitimate user has to take an action - click a link, open a file, trigger a specific code path.

The score on that second vector drops to 6.4 Medium, despite the same damage potential. That drop reflects how much harder the exploitation chain is in practice. The gap between a 10.0 and a 6.6 isn't about how bad the impact is - it's about how accessible the attack path is.

The two metrics that do the most work in changing real-world exploitability are AC (does this require unusual conditions?) and PR (does the attacker need existing access?). Learn to read those two first.

Why the Base Score Lies About Your Risk

This is the most important section. The Base Score is calculated against a theoretical target with no defenses and maximum exposure. Your deployment is not that target. The delta between the two is where triage happens.

Example A: The Network-Reachable API

A Critical 9.8 in an npm package used by your public-facing REST API. The vector shows AV:N/AC:L/PR:N - network exploitable, no special conditions, no authentication required. Your API is reachable from the internet. The package handles request parsing and runs on every inbound request. The Base Score is accurate here: this is a genuine fire drill. Patch immediately. The theoretical worst case and your actual case are close to the same thing.

Example B: The Same CVE in a Build Tool

The exact same CVE - same package, same vector string - but this time the package only runs during your local npm run build step or inside a CI job with no external network exposure. AV:N in the vector means "network" is the attack vector under ideal conditions. If the package never processes data from an untrusted network source and the machine running it isn't exposed to one, that attack vector doesn't apply to your deployment. The 9.8 is still on the advisory. Your actual risk is dramatically lower. This belongs in the next sprint, not in an emergency change window tonight.

Example C: The "Critical" Without a Public Exploit

A 9.8 Base Score with no known public proof-of-concept. The Temporal metric Exploit Code Maturity would show E:U (Unproven) if applied - but most tooling doesn't apply Temporal metrics, so you only see the Base Score. Check the advisory References section manually. If no PoC exists, the window of realistic exploitation is much narrower. This doesn't mean "ignore it" - it means "don't drop everything at 5pm on a Friday to merge an untested patch."

The Environmental Score: The Fix Nobody Uses

CVSS provides Environmental Metrics precisely for this problem. Your organization can configure values for Modified Attack Vector, Modified Confidentiality, and others to reflect the actual deployment context, producing an adjusted score that accurately represents your exposure. An AV:N vulnerability running inside a network segment with no external access can have its Attack Vector modified to AV:L in the environmental calculation, producing a score that reflects reality.

Almost no teams do this because tooling support is inconsistent and the process isn't automated. Understanding that it exists changes how you read advisories - you know you can mentally apply the same logic even when the tool doesn't do it for you.

Reading a Real GitHub Advisory

When a Dependabot alert fires, the advisory it links to contains more useful information than the score. Here's how to read it efficiently.

The CVSS section shows the full vector string, not just the number. Click through to it. The vector string is the data; the number is just a summary. Read the metrics directly rather than trying to reverse-engineer them from the score.

The Weaknesses field (CWE) tells you the type of vulnerability - CWE-79 (Cross-Site Scripting), CWE-89 (SQL Injection), CWE-400 (Uncontrolled Resource Consumption). CVSS tells you how severe; CWE tells you what it actually is. This matters for assessing whether your code actually exercises the vulnerable path. A CWE-79 in a server-side rendering library matters a lot if you're rendering user-supplied content and nothing if you're using the library in a static site generator that never processes external input.

Affected versions and Patched versions are more immediately useful than the score for deciding urgency. If a patched version exists, the question becomes "how hard is this upgrade?" - often the answer is "trivially easy," and you should just do it regardless of score. If no patched version exists, you need mitigations and monitoring, and that's true whether the score is 4.0 or 9.8.

The References section is where exploit signal lives. Look for links to GitHub repositories, exploit-db entries, proof-of-concept write-ups, or Metasploit modules. A published PoC changes the urgency calculation immediately - regardless of Base Score, the barrier to exploitation just dropped to near-zero for anyone with basic skills.

A Triage Framework

Apply this as a decision sequence, not a scoring rubric. Work through it in order and stop when you have enough signal.

Is the vulnerable package reachable from an untrusted network in production? Check your deployment: does this package process data from external sources? If no → deprioritize, schedule for next sprint or next maintenance window. If yes → continue.
Does Attack Complexity require conditions you don't have? An AC:H vulnerability requires non-default configuration or specific runtime conditions. If your deployment doesn't match those conditions → reduce urgency. If AC:L → continue.
Does it require privileges your attack surface doesn't expose? PR:H means an admin-level authenticated attacker. If your vulnerable endpoint requires authentication and your threat model doesn't include compromised admin accounts → reduce urgency. If PR:N → continue.
Is there a known public exploit? Check the advisory References section and the CVE detail pages on NVD and Mitre. A published proof-of-concept means treat it as immediate regardless of score. An E:U Temporal rating (no public exploit) means you have more runway.
Is a patched version available? If yes → patch now. Even for lower-urgency vulnerabilities, if the upgrade path is straightforward, just do it. The cost is low and the future you will be grateful. If no → document a mitigation (firewall rule, input validation layer, feature flag) and monitor for patch availability.

CVSS Quick Reference

The metrics that most change real-world exploitability:

AV:N = network-exploitable (worst for server apps) - ask whether the package actually processes network input in your deployment
AV:L = local access required - much lower risk for any server-side or cloud-hosted workload
AC:L = no special conditions needed (worst) - the attack path is straightforward
AC:H = requires specific configuration or conditions - assess whether your deployment matches
PR:N = no authentication required (worst) - unauthenticated remote exploitation
PR:H = admin credentials required - material reduction in exploitability
S:C (Scope Changed) = the exploit crosses security boundaries - container escapes, privilege escalation, cross-tenant impact - always serious regardless of other metrics
Base Score alone is not a triage decision - always check: is the package reachable in production? Is there a public exploit? Is a patch available?

Closing

CVSS scores are a standardized starting point for a conversation, not the end of one. The number exists to make vulnerabilities comparable across software and vendors. It was never designed to replace context - it was designed to communicate in the absence of it.

A 9.8 in a package your public API depends on is a fire drill. The same 9.8 in a build-time tool that never processes network input is a scheduled maintenance item. Both are real vulnerabilities. Only one of them should interrupt your day.

Teams that treat every Critical as a five-alarm emergency burn out and start ignoring alerts. Teams that read the vector string, check their deployment context, and apply the five-step triage sequence above make better decisions faster - and build the kind of judgment that means the actual emergencies get the response they deserve.

The vector string is eight components. It takes sixty seconds to read. Start there.

Have questions about vulnerability triage, CVSS environmental scoring, or building a security response process that doesn't burn out your team? Reach out.

steve.kaschimer@slalom.com

Generating and Using SBOMs with GitHub Actions

2026-04-10T00:00:00Z

The SBOM requirement showed up in a procurement questionnaire. Someone on the team generated one, attached it to a Confluence page, checked the box, and moved on. Six months later a new CVE dropped for a package nobody had heard of. It turned out to be a transitive dependency - the dependency of a dependency - that had been in every release for two years. The Confluence document, already stale the day it was created, couldn't answer the question that mattered: was the vulnerable version in the build that shipped last week, or the one that shipped the week before? The audit trail was blank. The compliance checkbox was green.

This is the gap between compliance theater and an actually useful SBOM - Software Bill of Materials. A document filed in a wiki tells you roughly what was on a developer's machine the day someone decided to run a scan. An SBOM attached to a specific release commit, generated automatically by your CI pipeline, cryptographically signed, and queryable on demand tells you exactly what shipped and when. The difference isn't philosophical. One is evidence; the other is paperwork. GitHub Actions - specifically anchore/sbom-action and GitHub's artifact attestation - makes producing the real version take about fifteen lines of YAML.

What an SBOM Actually Is

An SBOM is a machine-readable inventory of every component in your software - direct dependencies, transitive dependencies, their versions, licenses, and known vulnerabilities at the time of build.

The two dominant formats are SPDX (Linux Foundation, widely used in government and enterprise procurement) and CycloneDX (OWASP, richer vulnerability data, better tooling ecosystem). The NTIA minimum elements guidance and Executive Order 14028 are format-agnostic, but in practice CycloneDX has better tooling support for querying and analysis. This post uses CycloneDX JSON.

A CycloneDX SBOM contains, per component:

Package name and version - exactly what was resolved and installed, not what was specified
PURL - a Package URL in the form pkg:npm/lodash@4.17.21 that uniquely identifies the component across ecosystems
License - often the thing legal is actually asking about
Supplier - the entity that published the package
Hashes - SHA-256 and SHA-512 digests of the component at the time of inclusion

The version and hash fields are what make the SBOM meaningful for security response. When a CVE drops, you don't ask "do we use this package?" - you ask "which of our releases included version X, and is that version still deployed?" The SBOM answers both questions directly.

The reason transitive dependencies matter more than most developers realize: the majority of documented supply chain attacks target transitive dependencies, not the packages a team explicitly installs. Your package.json might list twenty direct dependencies. Your resolved dependency tree likely contains several hundred packages. Most of your team can't name ten of them. The SBOM names all of them.

An SBOM is a snapshot of your software's supply chain at a specific point in time. Its value degrades as soon as a dependency changes - which is why generating it at build time, not manually, is the only approach that scales.

The GitHub Tooling Stack

Four components do the work in this post:

anchore/sbom-action generates CycloneDX or SPDX SBOMs from a source repository, a compiled artifact, or a container image. Under the hood it wraps Syft, Anchore's open-source SBOM generator. The action handles ecosystem detection automatically - npm, Maven, Go modules, Python, NuGet, and others are all supported without configuration.

actions/attest creates a sigstore-based attestation that cryptographically binds your SBOM file to the specific GitHub Actions workflow run and commit that produced it. The attestation is stored in GitHub's attestation API, not as a file in your repo. It uses the workflow's OIDC identity - a short-lived token issued to the specific run - as the signing key, so there's no long-lived secret to manage and no key rotation story to write.

GitHub Releases is where the SBOM gets attached as a named asset. Consumers - security teams, procurement reviewers, downstream pipelines - can retrieve it without cloning the repository.

gh attestation verify is how any consumer, including your own audit workflow, validates that an SBOM file was produced by the claimed workflow run and hasn't been tampered with since.

Generating the SBOM: Step by Step

Step 1: Basic SBOM Generation on Release

This workflow triggers on any tag matching v*, generates a CycloneDX JSON SBOM, attests it, and attaches it to the GitHub Release created by the tag push.

name: Release

on:
  push:
    tags:
      - 'v*'

permissions:
  contents: write
  id-token: write
  attestations: write

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Generate SBOM
        uses: anchore/sbom-action@v0
        with:
          format: cyclonedx-json
          output-file: sbom.cyclonedx.json

      - name: Attest SBOM
        uses: actions/attest@v1
        with:
          subject-path: sbom.cyclonedx.json

      - name: Attach SBOM to release
        uses: softprops/action-gh-release@v2
        with:
          files: sbom.cyclonedx.json

The three permissions are not interchangeable defaults - each one does specific work:

contents: write allows the workflow to create and upload assets to the GitHub Release created by the tag push
id-token: write allows the workflow to request an OIDC token from GitHub, which is the signing identity that sigstore uses for the attestation - without this, the attest step fails silently
attestations: write allows the workflow to write the attestation record to GitHub's attestation API

If you're building a container image alongside the release, anchore/sbom-action can generate an image SBOM instead by setting image instead of scanning the source tree:

- name: Generate image SBOM
  uses: anchore/sbom-action@v0
  with:
    image: ghcr.io/your-org/your-image:${{ github.ref_name }}
    format: cyclonedx-json
    output-file: sbom.cyclonedx.json

Source-tree and image SBOMs answer different questions. The source-tree SBOM reflects what your build process consumed. The image SBOM reflects what ended up in the container, including any OS-level packages installed in the base image. For a complete supply chain picture you want both, attached as separate release assets.

Step 2: Validating the Attestation Downstream

After the release is created, any consumer can verify the SBOM's provenance:

gh attestation verify sbom.cyclonedx.json \
  --owner your-org \
  --repo your-repo

What this checks: that the file was signed by a GitHub Actions workflow running in the specified org and repo, using the OIDC identity of the specific workflow run. The attestation record includes the git commit SHA, the workflow file path, and the ref that triggered the run. If the file has been modified since it was attested - even a single byte - verification fails.

You can add this as a gate in a downstream audit workflow, or run it manually in an incident response scenario to confirm that the SBOM you're looking at is the one that was produced at release time and hasn't been manipulated:

# Output shows the signer identity, workflow ref, and commit
gh attestation verify sbom.cyclonedx.json \
  --owner your-org \
  --repo your-repo \
  --format json | jq '.verificationResult.statement.predicate'

The SBOM as a Debugging Tool

Compliance is the reason most teams generate an SBOM. Debugging transitive dependency surprises is the reason you'll be glad you did. Three concrete scenarios:

Scenario A: The Mystery Vulnerability

Dependabot fires an alert for a package you don't recognize. You search your package.json - it's not there. It's a transitive dependency. Without the SBOM you trace the tree manually: npm ls <package>, follow the chain, work out which of your direct dependencies pulled it in, decide whether you can bump that direct dep or need a resolution override.

With the SBOM, you query it:

# Find the affected component and its PURL
jq '.components[] | select(.name == "vulnerable-package") | {name, version, purl}' \
  sbom.cyclonedx.json

The PURL tells you the ecosystem, the package registry, the name, and the exact version. From there you know immediately whether the version in the release matches the affected range in the CVE. You're not guessing based on what's currently installed - you're looking at the resolved state at the moment the build ran.

Scenario B: License Audit

Legal asks whether any GPL-licensed dependencies made it into the product. Without an SBOM this is a manual audit of every package in the tree, opening each one's LICENSE file or checking the registry. With one:

# List all components with GPL licenses
jq '.components[] | select(.licenses[]?.license.id | test("GPL"; "i")) | {name, version, licenses}' \
  sbom.cyclonedx.json

This runs in seconds and produces an exhaustive list including transitive dependencies that almost certainly weren't reviewed during the original dependency selection. License compliance failures are disproportionately found in transitive deps - packages that seemed safe because nobody chose them.

Scenario C: Point-in-Time Comparison

A new CVE drops on a Tuesday. Your current codebase has already been patched - the vulnerable package was bumped in a PR three weeks ago. But you need to know whether the release that's currently in production, tagged v2.4.1 two months ago, was affected. The SBOM attached to that release tag is the authoritative answer. No guessing from git history, no reconstructing lock files, no hoping that the package manager's lock file actually reflects what was installed in CI.

This is the scenario that makes the "attach to every release, don't let it be ephemeral" rule non-negotiable. An SBOM that lives only in a workflow artifact expires in 90 days by default. One attached to a GitHub Release lives as long as the release does.

SBOM in the PR Pipeline

Generating on release is the baseline. Generating on every PR and diffing the result is the level-up. The goal is to catch unexpected changes in the transitive dependency tree before they merge - the scenario where a direct dependency bump quietly pulls in a new version of a shared transitive dep that nobody reviewed.

name: SBOM Diff

on:
  pull_request:

permissions:
  contents: read

jobs:
  sbom-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Generate SBOM for PR branch
        uses: anchore/sbom-action@v0
        with:
          format: cyclonedx-json
          output-file: sbom-pr.cyclonedx.json

      - name: Checkout base branch
        uses: actions/checkout@v4
        with:
          ref: ${{ github.base_ref }}
          path: base

      - name: Generate SBOM for base branch
        uses: anchore/sbom-action@v0
        with:
          path: base
          format: cyclonedx-json
          output-file: sbom-base.cyclonedx.json

      - name: Diff transitive dependency count
        run: |
          base_count=$(jq '.components | length' sbom-base.cyclonedx.json)
          pr_count=$(jq '.components | length' sbom-pr.cyclonedx.json)
          echo "Base: $base_count components | PR: $pr_count components"
          if [ "$pr_count" -gt "$base_count" ]; then
            echo "::warning::Transitive dependency count increased by $((pr_count - base_count))"
          fi

This won't block PRs by default - it surfaces the signal as a warning annotation. Whether that warning should block merges is a policy call for your team. The point is making the change visible before it ships, not after someone queries the release SBOM in response to an incident.

SBOM Implementation Checklist

Generate on every tagged release - not manually, not on demand
Use CycloneDX JSON format for best tooling and jq compatibility
Attest with actions/attest@v1 for cryptographic provenance tied to the specific workflow run
Attach to GitHub Releases as a named asset (sbom.cyclonedx.json) so it survives past artifact expiry
Set id-token: write and attestations: write permissions - without both, attestation silently fails
Verify attestation in your audit workflow with gh attestation verify
Archive SBOMs alongside release artifacts - an SBOM that expires in 90 days can't answer questions about a release from last year
Know your transitive dependency count: if you don't know it, run jq '.components | length' sbom.cyclonedx.json on your last release

Closing

The compliance requirement is a forcing function, but treat it as the floor rather than the ceiling. A manually generated SBOM filed in Confluence is compliance theater - it's a document that describes a state that no longer exists, signed by nobody, attached to nothing. The workflow in this post runs in under two minutes, produces a cryptographically attestable artifact tied to a specific git commit and workflow run, and gives your security team something they can query against a real CVE in a real incident.

The SBOM is only as useful as it is current. "Current" means generated at build time, on every release, automatically - not whenever someone on the team remembers to run a scanner. The attestation is only as useful as your ability to verify it. The debugging value is only as real as your willingness to actually query the artifact instead of filing it and forgetting it.

Your transitive dependency tree almost certainly contains packages you've never evaluated. The SBOM tells you their names, their versions, and their licenses. It takes one jq command to find out how many there are. Start there.

Have questions about supply chain security, SBOM tooling, or wiring attestation into your release pipeline? Reach out.

steve.kaschimer@slalom.com

GitHub CLI Power User: 10 `gh` Commands That Replace Browser Tabs

2026-04-17T00:00:00Z

Most developers have GitHub open in a browser tab permanently. They switch to it to check a PR status, review a diff, watch a failing run, paste in a secret, or find the branch name for an issue. Each of those trips is 30 seconds of context-switching that breaks whatever thread of thought was running in the background. The gh CLI eliminates most of them - not because it's clever, but because it puts GitHub's full API surface in the terminal, where you already are.

The problem isn't that people don't know gh exists. Most developers have it installed. The problem is that they used gh pr create once, found it fine, and never went deeper. This post covers the commands that actually change how you work: the ones that replace complete browser workflows rather than just wrapping a single API call.

The 10 Commands

1. `gh pr checkout`

Replaces: Copying a branch name off the PR page, running git fetch, running git checkout.

# Check out by PR number
gh pr checkout 342

# Check out by URL - works from any directory
gh pr checkout https://github.com/org/repo/pull/342

The underrated behavior: gh pr checkout sets up remote tracking automatically. git push works immediately after without a --set-upstream. It also handles PRs from forks - no manual remote setup, no fetching from the contributor's fork URL. If you've ever spent three minutes getting a forked PR's branch onto your machine, this is the command that eliminates that entirely.

2. `gh pr review`

Replaces: Opening the PR in a browser, navigating to the Files tab, writing a review.

# Approve with a note
gh pr review 342 --approve --body "LGTM, tested locally"

# Request changes
gh pr review 342 --request-changes --body "See inline comments"

# Leave a comment without a decision
gh pr review 342 --comment --body "One question before I approve"

Approve, request-changes, and comment on the whole PR are fully terminal-native. The one thing that still requires the browser: inline comments on specific file lines. For anything else - including the daily "LGTM" on a PR you've reviewed locally - this is faster than a browser tab.

3. `gh run watch`

Replaces: Refreshing the Actions tab to monitor a workflow run in progress.

# Watch the most recent run interactively
gh run watch

# Watch a specific run by ID
gh run watch 1234567890

# Exit with the run's exit code (the flag most people miss)
gh run watch --exit-status

The --exit-status flag is the one worth knowing: it returns a non-zero exit code when the run fails. That makes gh run watch composable in scripts:

gh workflow run deploy.yml && gh run watch --exit-status && echo "deployed successfully"

Without --exit-status, the command exits 0 regardless of whether the run passed or failed - which makes it useless in automation. With it, you get a blocking, scriptable workflow monitor.

4. `gh run rerun`

Replaces: Opening a failed run in the browser and clicking "Re-run failed jobs".

# Rerun only the failed jobs - not the entire workflow
gh run rerun 1234567890 --failed

# Rerun with step debug logging enabled
gh run rerun 1234567890 --debug

The --debug flag is the behavior most people don't know exists. It enables step-level debug logging for the rerun - equivalent to setting ACTIONS_STEP_DEBUG=true as a repository secret, but without touching your repo settings and without affecting other runs. When a job fails intermittently and you need visibility into exactly what happened, --debug is the first thing to reach for.

5. `gh issue develop`

Replaces: Manually creating a branch, remembering to include the issue number in the name, hoping you remember it later for the PR description.

# Create a branch linked to issue 88 and check it out immediately
gh issue develop 88 --checkout

The branch name is generated from the issue title and number - something like 88-fix-authentication-redirect-loop. The branch is automatically linked to the issue in the GitHub UI, and when you open a PR from it, the issue is referenced and closed automatically on merge. Use --base to target a non-default branch. This eliminates an entire class of "I forgot to link the issue" PR comments.

6. `gh secret set`

Replaces: Opening Repository Settings → Secrets and variables → Actions → New repository secret, pasting a value into a browser form field.

# Set from a file - value never touches shell history
gh secret set MY_API_KEY < secret.txt

# Pipe directly from a secret manager
aws secretsmanager get-secret-value --secret-id prod/api-key \
  --query SecretString --output text | gh secret set PROD_API_KEY

# Set an environment-scoped secret (not repo-level)
gh secret set DEPLOY_TOKEN --env production

# Set an org-level secret visible to all repos
gh secret set SHARED_TOKEN --org my-org --visibility all

Never pass the secret value directly as a flag: gh secret set MY_KEY --body "actual-value" writes the plaintext value into your shell history. The stdin approach (< secret.txt or a pipe) keeps the value out of history entirely. This is the default you should build muscle memory around.

7. `gh repo create --template`

Replaces: Navigating to a template repository, clicking "Use this template", waiting for the GitHub UI to create the repository, then cloning it.

# Create a private repo from a template and clone it locally in one step
gh repo create my-new-service \
  --template org/service-template \
  --private \
  --clone

Combine with a wrapper script in ~/scripts/new-service.sh that pre-fills the standard options for your organization - private visibility, team access, your naming convention. No more clicking through four browser screens for every new repository.

8. `gh api` with `--jq`

Replaces: Looking up the GitHub API endpoint, constructing a curl command, piping to a separate JSON parser.

# List open PRs with review status
gh api /repos/{owner}/{repo}/pulls \
  --jq '.[] | {number, title, user: .user.login, draft: .draft}'

# List org repos sorted by last push, handling pagination automatically
gh api /orgs/my-org/repos \
  --paginate \
  --jq 'sort_by(.pushed_at) | reverse | .[] | {name, pushed_at}'

Two things worth knowing: the {owner} and {repo} placeholders are filled automatically from the current directory's git remote - no hardcoding needed. And --paginate handles multi-page responses transparently, fetching all pages and concatenating the results before piping to --jq. Any GitHub REST endpoint is reachable this way, which means gh api is the escape hatch for anything the purpose-built commands don't cover.

9. `gh search`

Replaces: GitHub's web search interface, which requires a browser and returns results buried in a UI.

# Your open issues across all repos
gh search issues --assignee @me --state open --json number,title,repository

# Open Dependabot PRs in a specific repo
gh search prs "dependabot" --repo org/repo --state open

# Find hardcoded tokens in YAML files
gh search code "GITHUB_TOKEN" --language yaml --repo org/repo

The --json flag outputs machine-readable results composable with jq. The @me shorthand resolves to your authenticated GitHub user automatically. For cross-repo issue triage or security audits across an organization, gh search is considerably faster than assembling a GraphQL query by hand.

10. The Standup Script

Replaces: Mentally reconstructing what you worked on yesterday before a standup.

Save this as ~/scripts/standup.sh:

#!/bin/bash
# standup.sh - what did I do yesterday?
YESTERDAY=$(date -d "yesterday" +%Y-%m-%dT%H:%M:%SZ 2>/dev/null \
  || date -v-1d +%Y-%m-%dT%H:%M:%SZ)

echo "=== PRs you reviewed ==="
gh search prs --reviewed-by @me --updated ">$YESTERDAY" \
  --json number,title,repository \
  --jq '.[] | "  #\(.number) \(.title) [\(.repository.name)]"'

echo ""
echo "=== PRs you opened or updated ==="
gh search prs --author @me --updated ">$YESTERDAY" \
  --json number,title,state,repository \
  --jq '.[] | "  #\(.number) [\(.state)] \(.title) [\(.repository.name)]"'

echo ""
echo "=== Issues you were involved in ==="
gh search issues --involves @me --updated ">$YESTERDAY" \
  --json number,title,repository \
  --jq '.[] | "  #\(.number) \(.title) [\(.repository.name)]"'

The date syntax differs between GNU date (Linux) and BSD date (macOS) - the 2>/dev/null || fallback handles both. Run this every morning before standup: it pulls the previous day's PR reviews, authored PRs, and issue activity across all your repos without touching a browser.

Shell Aliases Worth Adding

A small set of aliases for .bashrc or .zshrc that make the most common workflows single-keystrokes:

# Check out a PR by number
alias prco='gh pr checkout'

# Watch the latest run on the current branch
alias runwatch='gh run watch $(gh run list \
  --branch $(git branch --show-current) \
  --limit 1 --json databaseId \
  --jq ".[0].databaseId")'

# Open the current repo in the browser (for the things that do need the browser)
alias ghopen='gh repo view --web'

# Create a PR for the current branch, pre-filled from commit messages
alias ghpr='gh pr create --fill --web'

The runwatch alias is the most useful: it resolves the latest run ID for the current branch automatically, so you can push a commit and immediately run runwatch without knowing or caring about run IDs.

Getting Started: Install and Auth

Install:

macOS: brew install gh
Windows: winget install GitHub.cli
Debian/Ubuntu: sudo apt install gh

Authenticate:

gh auth login          # browser flow or token
gh auth switch         # switch between accounts or GitHub Enterprise hosts
gh auth status         # check who you're authenticated as

One rule that matters: all gh commands resolve context from the current directory's git remote. Run them from inside the repository you want to act on. If you run gh pr list in the wrong directory, you'll get the wrong repo's PRs - and wonder why until you check gh repo view.

Closing

The payoff isn't any single command. It's the accumulated effect of eliminating ten context-switches a day - ten times you didn't reach for the browser, ten times you stayed in the terminal and kept the thread of thought intact. Over a workday that compounds into real, measurable concentration time. The standup script alone saves five minutes of mental reconstruction every morning before you've had coffee.

Start with gh pr checkout and gh run watch. Those two commands cover the majority of daily GitHub back-and-forth for most developers. The rest follows naturally once you've built the reflex to reach for gh before reaching for the browser.

Working on developer tooling at your organization, or want to talk through GitHub CLI adoption with your team? Reach out.

steve.kaschimer@slalom.com

Writing Commit Messages That Make Code Review Faster

2026-04-24T00:00:00Z

You open a PR for review. It has twelve commits. The messages read: "fix", "wip", "update", "more fixes", "actually fix", "pr feedback". There is no narrative, no context, no explanation of what was tried and discarded. To understand why any particular line changed, you have to reverse-engineer intent from the diff alone - which is exactly what the commit messages were supposed to make unnecessary. This is a communication failure, and it compounds: bad commit messages make code review slower, make git bisect a guessing game, make git blame useless for anything except finding who to ask, and make onboarding new teammates onto a codebase a puzzle instead of a story.

The fix takes about 60 seconds per commit. Most developers just haven't been taught the format.

The Anatomy of a Good Commit Message

Start with a concrete example of the finished product, then take it apart:

feat(auth): replace session tokens with JWTs

Cookie-based sessions were hitting a scaling wall - the session store
was becoming a bottleneck at ~5k concurrent users. JWTs eliminate the
server-side session lookup entirely.

Considered Redis cluster as an alternative but rejected it: adds
infrastructure complexity and the session store problem recurs at
higher scale. JWTs shift the complexity to token validation, which
is stateless and horizontally scalable.

Breaking change: clients must handle 401 responses by re-authenticating.
Existing sessions are invalidated on deploy.

Closes #412
Co-authored-by: Jamie Lee <jamie@example.com>

Five distinct structural elements. Each one is doing specific work.

The subject line

50 characters or fewer - hard limit is 72. If your editor shows a ruler, put it there.
Imperative mood: "add", "fix", "remove" - not "added", "fixed", "removes". The convention is to complete the sentence "If applied, this commit will..." - the rest of that sentence is your subject line.
Type prefix + scope: feat(auth):, fix(api):, chore(deps): - this is Conventional Commits, covered in full below.
No period at the end. The subject line is a title, not a sentence.
If you can't write it in 50 characters, the commit is probably doing too much. That's information worth acting on.

The blank line

Required. Without it, many git tools - git log --oneline, git shortlog, GitHub's PR commit list - treat the entire message as a single subject. The blank line is not optional punctuation. It is structural.

The body

This is the part most developers skip and the part that pays the most dividends over time. The body explains why, not what - the diff already shows what changed. Three questions the body should answer:

Why was this change necessary?
What alternatives were considered and why were they rejected?
What constraints or tradeoffs shaped the approach?

Wrap at 72 characters. git log outputs body text at full width in a terminal - unwrapped lines that run past 80 characters make the output unreadable without horizontal scrolling.

The footer

Issue references: Closes #412, Fixes #88, Resolves #200, Refs #101
Co-authors: Co-authored-by: Name <email> - GitHub parses this trailer and credits the contributor in the commit view and contribution graph
Breaking changes: BREAKING CHANGE: - the Conventional Commits spec; triggers a major version bump in semantic-release and release-please

The footer is where metadata lives. Putting Closes #412 in the body instead of the footer works syntactically, but it survives squash-merge and PR description edits more reliably as a footer trailer.

Conventional Commits - The Spec Worth Adopting

Conventional Commits is a specification for commit message format that makes history machine-parseable: <type>(<scope>): <subject>.

The common types, and what they mean:

Type	Use it for
`feat`	New capability or behavior
`fix`	Bug fix
`docs`	Documentation only
`style`	Formatting, whitespace - no logic change
`refactor`	Code restructuring, no behavior change
`test`	Adding or updating tests
`chore`	Maintenance, config, tooling
`ci`	CI/CD pipeline changes
`perf`	Performance improvement
`revert`	Reverting a previous commit

The scope in parentheses is optional but useful: feat(auth), fix(api), chore(deps). It narrows where the change lives and makes filtered log queries (git log --grep="^feat(auth)") actually useful.

Why this matters beyond aesthetics: Conventional Commits is machine-parseable. Tools like semantic-release, conventional-changelog, and release-please read your commit history to determine version bumps and generate changelogs automatically. A feat commit triggers a minor version bump. A fix triggers a patch. A commit with BREAKING CHANGE: in the footer triggers a major. That automation is only possible because the commit messages follow a predictable structure.

Before:

fix stuff
update deps
more work on auth
fix tests

After:

feat(auth): add JWT refresh token rotation
fix(api): handle null user on profile endpoint
chore(deps): bump axios from 1.6.0 to 1.7.2
test(auth): add coverage for token expiry edge case

From the "after" log, conventional-changelog generates:

## [2.1.0] - 2026-04-24

### Features
- **auth:** add JWT refresh token rotation

### Bug Fixes
- **api:** handle null user on profile endpoint

Zero manual changelog writing. The history is the changelog, because the commit messages are structured well enough to read programmatically.

Writing the Body - The Why, Not the What

The body is where most developers have the most room to improve and the most to gain. Here is the pattern to avoid:

# Bad - describes what the diff already shows
refactor(db): extract query builder

Moved query building logic from UserRepository into a new
QueryBuilder class. Added methods for filtering and sorting.

That body is worse than no body. It repeats what the diff shows, adds no context, and will tell a future reader nothing they couldn't have learned from running git diff. Compare:

# Good - explains why and what was considered
refactor(db): extract query builder

UserRepository had grown to 400 lines, 60% of which was
query construction logic unrelated to repository concerns.
Extracting QueryBuilder makes each class testable in isolation
and unblocks the planned migration to a read replica (tracked
in #388).

Considered an ORM (Prisma) but deferred: migration cost is
high and the current query patterns don't justify the
abstraction. Revisit if the read replica migration expands
the query surface significantly.

The test for whether a body is done: could someone who wasn't in the room understand why this change was made, six months from now, with only this message and the diff? If not, the body isn't done.

That test is particularly important for decisions that look arbitrary without context. The rejected Redis cluster alternative in the opening example isn't there to show off the author's research - it's there because the next engineer to touch that code will have the same idea, and they deserve to know it was already considered and why it was rejected. Without that note, the investigation happens again. Bad commit messages bill future engineers for decisions that were already paid for.

Linking Issues and PRs Correctly

GitHub parses specific closing keywords in commit messages (and PR descriptions) and acts on them when code lands on the default branch:

Closes #123 - closes the issue on merge
Fixes #123 - closes the issue (alias for Closes)
Resolves #123 - closes the issue (alias for Closes)
Refs #123 - links without closing, for partial work or related issues

The recommendation: put these in the commit message footer, not the PR description. Here's why.

If you use a squash-merge strategy, GitHub uses the PR description as the squash commit message by default. But PR descriptions get edited - the final state of the description may not match what was in the original. Issue references in individual commit messages survive this, and they're visible in the git history independent of GitHub's UI.

For Refs specifically: use it when a commit is related to an issue but doesn't fully resolve it. A multi-PR epic might have three commits that each Refs #88 and one final commit that Closes #88. That gives a clean audit trail of every commit that touched the work.

Enforcing Format with a Commit-Msg Hook

A commit message standard that lives only in a team wiki is not a standard. Enforcement needs to be automatic.

The first layer is a commit-msg hook that runs locally before the commit is accepted:

#!/bin/bash
# .git/hooks/commit-msg
# Enforce Conventional Commits format

commit_regex='^(feat|fix|docs|style|refactor|test|chore|ci|perf|revert)(\(.+\))?: .{1,72}'

if ! grep -qE "$commit_regex" "$1"; then
  echo "ERROR: Commit message does not follow Conventional Commits format."
  echo "Expected: <type>(<scope>): <subject>"
  echo "Example:  feat(auth): add JWT refresh token rotation"
  exit 1
fi

Install it:

chmod +x .git/hooks/commit-msg

The problem with a raw .git/hooks/ file: it isn't committed to the repository and doesn't automatically apply for new clones. The team-scale solution is commitlint with Husky:

npm install --save-dev husky @commitlint/cli @commitlint/config-conventional
npx husky init

{
  "scripts": {
    "prepare": "husky"
  }
}

# .husky/commit-msg
npx --no -- commitlint --edit $1

// commitlint.config.js (ESM - requires "type": "module" in package.json)
export default {
  extends: ['@commitlint/config-conventional']
};

// CommonJS alternative: rename to commitlint.config.cjs and use:
// module.exports = { extends: ['@commitlint/config-conventional'] };

The prepare script runs on npm install, so every developer who clones the repository and installs dependencies gets the hook automatically.

CI Enforcement

The local hook can be bypassed with git commit --no-verify. For teams where that matters - or for open-source projects where contributors control their own environments - add a CI check that runs on pull requests:

name: Lint Commits
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  commitlint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npx commitlint --from ${{ github.event.pull_request.base.sha }} --to ${{ github.event.pull_request.head.sha }} --verbose

The fetch-depth: 0 is required - without it, the shallow clone won't have the base commit in history, and commitlint can't compute the range. This catches any commit that bypassed the local hook, and it gives contributors clear feedback in CI before the PR goes to review.

`git notes` - Post-Merge Context Without Rewriting History

Sometimes you learn something after a commit merges - a production incident reveals the real cause, a follow-up investigation changes your understanding of a decision. git notes lets you attach context to an existing commit without amending or rewriting history:

# Add a note to the most recent commit
git notes add -m "This introduced a subtle race condition under high load. See incident-2024-11-14 in the runbook."

# Add a note to a specific commit
git notes add -m "Root cause confirmed in #512. The fix is in abc9876." abc1234

# View notes in git log
git log --show-notes

The limitation worth knowing upfront: git notes don't sync automatically. You have to push and fetch them explicitly:

# Push notes to the remote
git push origin refs/notes/commits

# Fetch notes from the remote
git fetch origin refs/notes/commits

That friction makes git notes most useful for team-internal context in repositories where the note-fetching step can be scripted into onboarding. For open-source projects where contributors won't have the notes configured, a linked issue comment is a more reliable place for post-merge context. Use git notes where you control the team's git workflow; use issue/PR references everywhere else.

Commit Message Checklist

Before every commit:

[ ] Subject line is ≤ 50 characters (hard limit: 72), imperative mood, no trailing period
[ ] Type prefix matches what changed - feat for new capability, fix for bug, chore for maintenance
[ ] Body explains why, not what the diff already shows
[ ] Tradeoffs and rejected alternatives are documented if the decision wasn't obvious
[ ] Issue reference is in the footer (Closes #N, Refs #N) - not buried in the body
[ ] If it's a breaking change: BREAKING CHANGE: is in the footer
[ ] If you couldn't fit the change in one subject line, consider whether the commit should be split

The Asymmetry of the Investment

Writing a good commit message costs 60 seconds. Reading a bad one during code review, a git bisect session, or an incident postmortem costs multiples of that - multiplied by every person who reads it, every time the codebase is touched for as long as it exists. A codebase with good commit messages is a codebase with a searchable, human-readable record of every decision ever made: why the architecture looks the way it does, what was tried and rejected, what constraints shaped each choice.

That's useful for reviewers. It's useful for the new engineer trying to understand a module they've never touched. It's especially useful for the person who wrote the commits six months from now, staring at a line they no longer remember writing, asking themselves why they made a choice they can't explain.

The format is learnable in an afternoon. The discipline is a habit built commit by commit. Start with the subject line - type prefix, imperative mood, under 72 characters. Add a body the next time you make a decision that future-you will need to understand. The rest follows.

Working on developer tooling or engineering practices at your organization? Reach out.

steve.kaschimer@slalom.com

Architecture Decision Records: The 30-Minute Investment That Pays Off for Years

2026-05-01T00:00:00Z

Six months into a project, a new engineer asks why the codebase uses library X instead of the obvious choice Y. Nobody remembers. The original decision-maker has left. The Slack thread is gone. The PR description says "initial implementation." The team spends 45 minutes reconstructing a decision that took 20 minutes to make - and they still aren't sure they got it right.

This happens constantly. It is entirely avoidable.

An Architecture Decision Record (ADR) is a Markdown file that captures a decision, its context, the alternatives considered, and the reasoning. One file. Thirty minutes. Permanent record. A codebase with 20 ADRs is a codebase whose entire architectural history is readable in a docs/ folder without needing to interrogate anyone, reconstruct anything, or trust that the person who made the call is still at the company.

What an ADR Is (and Isn't)

An ADR records a single architectural decision at the moment it was made. It is not a design document, not a post-mortem, not a wiki page that gets updated as the system evolves. The distinction matters because it determines how you use the record.

The two properties that make ADRs useful are also the two that teams instinctively resist. First: one decision per file. Not "the architecture of the authentication system" - that's a design document. An ADR is "use JWTs instead of server-side sessions." Specific, bounded, answerable. Second: immutable once accepted. You do not edit an old ADR to reflect a change in direction. You write a new ADR that supersedes it, and the old one stays in the repo with its status updated. The history is the value.

The format was coined by Michael Nygard in a 2011 blog post and later popularized by the adr-tools CLI project. The exact template has evolved, but the principle hasn't moved.

What counts as an architectural decision: anything that affects the structure of the system, is expensive to reverse, or that future maintainers will need to understand in order to make sensible choices. Template engine selection, database schema approach, authentication strategy, monorepo vs. polyrepo, API versioning policy. What doesn't warrant an ADR: bug fixes, routine implementation choices, minor refactors that don't change structural constraints.

An ADR is a snapshot of a decision as it was understood at the time it was made. Its value isn't that it's always right - it's that it's honest about what was known, what was considered, and what was chosen, so future teams can evaluate whether those conditions still hold.

The Template

The maximalist ADR templates floating around the internet have twelve sections and take longer to fill out than it took to make the decision. This is the version that covers what actually matters:

# ADR-{number}: {Title}

**Date:** YYYY-MM-DD  
**Status:** Proposed | Accepted | Deprecated | Superseded by ADR-{N}  
**Deciders:** {Names or roles of people involved in the decision}

## Context

What is the situation that requires a decision? What constraints or forces are at play?
Describe the problem, not the solution.

## Decision

What was decided? State it clearly in one or two sentences.

## Alternatives Considered

| Option | Pros | Cons |
|--------|------|------|
| Option A | ... | ... |
| Option B | ... | ... |
| Option C | ... | ... |

## Consequences

What becomes easier or harder as a result of this decision?
What follow-up decisions does this enable or require?
What is the cost of reversing this decision if it proves wrong?

## References

- Link to relevant PRs, issues, discussions, external docs

Four substantive sections. Context describes the situation and its constraints - it is about the problem, not the solution. If you skip this section, the decision loses its meaning the moment the original conditions change. Decision is one or two sentences stating what was chosen. Alternatives Considered is the table most teams fill out in their heads and never write down - it is the section that prevents the same research from being done twice. Consequences is the section people skip most often and future maintainers value most. It answers the questions that actually come up during maintenance: Is this easy to reverse? What follow-up choices did this lock in? What got harder?

A Real-World Example

This blog runs on Eleventy, and the template engine choice is exactly the kind of decision that looks arbitrary without context. Here is what ADR-001 for this project would look like:

# ADR-001: Use Nunjucks as the Eleventy Template Engine

**Date:** 2025-10-15  
**Status:** Accepted  
**Deciders:** Steve Kaschimer

## Context

Eleventy supports multiple template languages: Nunjucks, Liquid, Handlebars,
EJS, and plain HTML. The project needs a template language that supports
layouts, includes, macros/partials, and conditional logic. The choice affects
every template file in the project and is expensive to reverse.

## Decision

Use Nunjucks (`.njk`) as the primary template language for all layouts and pages.

## Alternatives Considered

| Option | Pros | Cons |
|--------|------|------|
| Nunjucks | Full-featured (macros, filters, inheritance), mature Eleventy support, familiar to Jinja2 users | Slightly more syntax to learn than Liquid |
| Liquid | Simpler syntax, default in Jekyll (familiar to many) | Fewer features, no macro support, less expressive for complex layouts |
| Handlebars | Familiar to JS developers | Limited built-in helpers, logic-less by design (a constraint here, not a feature) |
| EJS | Pure JavaScript in templates | Mixing logic and markup leads to unmaintainable templates at scale |

## Consequences

- All layout and page files use `.njk` extension
- Eleventy filters and shortcodes are written to work with Nunjucks syntax
- New contributors familiar with Liquid/Jekyll will need a brief orientation
- Migration cost if we switch: high - every template file would need rewriting
- Enables: complex layout inheritance, custom filters, macro-based component patterns

## References

- [Eleventy template language docs](https://www.11ty.dev/docs/languages/)
- PR #3: Initial project scaffold

Every new engineer who touches a template now gets this answer in under two minutes instead of in a 45-minute archaeology session. And when Eleventy ships a compelling new template format - say, WebC - the question "should we reconsider this?" is grounded in the documented reasons the original choice was made, not in whoever happens to be in the room.

Where to Store ADRs

ADRs belong in the repository, not in Confluence, not in Notion, not in a separate wiki. When the ADR lives next to the code it governs, it's reviewable in pull requests, findable from the same search that surfaces source files, and it survives tool migrations. Documentation that drifts away from the code it documents becomes archaeology at a different URL.

The convention:

docs/decisions/ at the repository root
Filenames: zero-padded number + kebab-case title - 001-use-nunjucks-as-template-engine.md
An index at docs/decisions/README.md with one-line summaries and status

The index format:

# Architecture Decision Records

| # | Title | Status | Date |
|---|-------|--------|------|
| 001 | Use Nunjucks as the Eleventy template engine | Accepted | 2025-10-15 |
| 002 | Deploy to GitHub Pages via GitHub Actions | Accepted | 2025-10-20 |
| 003 | Use Tailwind CSS for styling | Accepted | 2025-10-20 |

The index is the entry point for any engineer who wants to understand why the system looks the way it does. It should be readable top-to-bottom in five minutes. Keep it current as part of the PR that adds or supersedes an ADR.

The Deliberation Workflow with GitHub Discussions

The ADR template handles the record. The deliberation - the conversation before a decision is made - belongs somewhere else. Mixing the two in the same file produces ADRs that are half-deliberation, half-decision, and useful as neither. GitHub Discussions is the right tool for the deliberation phase.

The workflow:

Engineer opens a Discussion in an Architecture category (create it if it doesn't exist) with the draft ADR as the body
Team comments with concerns, alternative options, data, prior art
Engineer updates the draft as the conversation converges
Once consensus is reached, a PR is opened: docs/decisions/005-adopt-jwt-auth.md
PR description includes: Closes discussion #42
PR merges; Discussion closes; the decision is now permanent and co-located with code

This creates a two-layer record. The Discussion holds the deliberation - the messy, non-linear conversation where options were surfaced and rejected. The ADR holds the distilled outcome. Both are searchable in GitHub. Neither requires a separate tool. And critically: the Discussion captures the voices of people who raised concerns that were ultimately rejected, which is often the most valuable thing to know when you revisit the decision two years later.

Linking ADRs from PRs and Commits

An ADR sitting in docs/decisions/ and never referenced from the code it governs is a document that will be forgotten. The connections have to be explicit.

PR descriptions: when a PR implements a decision, reference the ADR directly. "Implements ADR-005. See docs/decisions/005-adopt-jwt-auth.md." This makes the PR self-contained - reviewers know where to find the rationale without asking.

Commit message footers: for commits that land architectural changes, add Refs docs/decisions/005-adopt-jwt-auth.md in the trailer block. This connects git blame output to the ADR. The combination is the complete picture: git blame tells you who changed the line; the ADR tells you why the approach was chosen in the first place.

Code comments: for non-obvious implementation choices, a single-line comment is enough. // See ADR-003 - chosen over alternatives for reasons in docs/decisions/. Not a comment that explains what the code does - the code does that. A comment that explains why the code is structured this way and where to find the full reasoning.

The goal is a web of references tight enough that any engineer starting from either the ADR or the code can reach the other within one click.

How Decisions Evolve - Superseding ADRs

Architectural decisions change. The correct response is not to edit the original ADR. It is to write a new one that supersedes it and update the old one's status field.

The old ADR: status becomes Superseded by ADR-007. The new ADR: references the old one in its Context section, explaining what has changed since the original decision was made. Here is what that looks like:

# ADR-007: Migrate from Nunjucks to WebC for Component-Based Templates

**Date:** 2026-06-01  
**Status:** Accepted  
**Supersedes:** ADR-001

## Context

ADR-001 chose Nunjucks for its maturity and layout inheritance support.
Since that decision, Eleventy introduced WebC - a single-file component
format that eliminates the need for macros and provides scoped CSS and JS
bundling. The project has grown to 15+ reusable components where Nunjucks
macros are showing maintenance friction. The original concern about reversal
cost still applies; this decision should not be made lightly.

This creates a decision changelog. You can trace how the team's thinking evolved over time, what changed in the environment, and what the cost of each reversal was judged to be. That history is only available because the earlier ADR was never edited - it captured what was true and what was known at the time it was written. The moment you start retroactively updating ADRs to reflect where you ended up, you lose the record of how you got there.

ADR Quick-Start Checklist

To start using ADRs today:

[ ] Create docs/decisions/ in your repository root
[ ] Add docs/decisions/README.md - even if the index starts empty
[ ] Write your first ADR for the most recent significant decision you made - don't reconstruct the entire project history, start from now
[ ] Add an ADR pull request template at .github/PULL_REQUEST_TEMPLATE/adr.md with the four-section structure
[ ] Establish the norm: any PR that introduces or changes a foundational pattern either references an existing ADR or creates a new one
[ ] Reference ADRs from PR descriptions and commit footers - the link from code to reasoning is what makes the record useful
[ ] When a decision changes: update the old ADR's status field, write a new ADR that supersedes it - never edit the original

You don't need a tool to start. adr-tools (CLI) is useful at scale but not required. A folder and a template are enough.

The Asymmetry

ADRs have almost no cost at the time of writing and asymmetric value over time. The 30 minutes you spend on ADR-001 pays back the first time a new engineer asks "why are we using Nunjucks?" and gets a two-minute answer instead of a 45-minute archaeology session. The payback compounds: a codebase with 20 ADRs is a codebase whose architectural history is readable, searchable, and honest about uncertainty. Not just "what did we decide" but "what did we consider," "what did we know at the time," and "what would it cost to change this."

That's not documentation for its own sake. That's a team that respects the time of every engineer who comes after them - including themselves, six months from now, staring at a decision they no longer remember making.

The first ADR is the hardest. Write it this week for the last significant decision your team made. Everything after that is just the habit.

Want to talk through documenting architectural decisions at your organization, or building a decision-record practice from scratch? Reach out.

steve.kaschimer@slalom.com

GitHub Branch Protection Rules vs. Rulesets: The New Way to Enforce Standards

2026-05-08T00:00:00Z

Most teams set up branch protection rules once, years ago, and haven't touched them since. That's understandable - once it's configured, it's invisible infrastructure. What's less visible is the hole in it. Classic branch protection has a default behavior that's documented but easy to miss: repository admins bypass all rules. Require pull request reviews? An admin can push directly to main. Require status checks? An admin can merge without them. For most small and medium teams - where the admin is also a developer - the protection they think they have has a gap large enough to drive a production incident through.

GitHub Rulesets close that gap. They also add organization-level enforcement, tag protection, named bypass actors, and an evaluation mode that lets you audit what would be blocked before you enforce anything. This post maps what changed between the two systems, walks through a production-ready Ruleset configuration, and includes an audit workflow that checks Ruleset coverage across every repo in your org.

What Classic Branch Protection Actually Does - and Where It Breaks Down

Classic branch protection gives you the fundamentals most teams need:

Require pull request reviews before merging (with configurable reviewer count and stale review dismissal)
Require status checks to pass before merging
Require branches to be up to date before merging
Restrict who can push to the branch
Require signed commits
Require linear history

That list covers a lot. For a single repo with a small team, it's often enough. The limitations become visible as teams grow or when something goes wrong.

The Admin Bypass Problem

By default, repository admins are exempt from all classic branch protection rules. There is a checkbox - "Include administrators" - that removes the exemption, but it is not enabled by default, and in practice many teams never enable it. This means that on most repos, the people most likely to push directly to main under pressure (the people with admin access) are the people for whom all those protections are silently inactive.

This isn't a fringe edge case. It's the default behavior.

Everything Else the Classic System Can't Do

Beyond admin bypass, the classic model has structural limitations:

No tag protection: classic branch protection is branches-only. Tags have a separate, weaker protection mechanism that most teams don't configure at all. Your v1.2.3 release tags are likely unprotected.
No organization-level enforcement: branch protection is configured per-repo. If your organization has 50 repositories, you need 50 separate configurations. There's no single source of truth.
No bypass actors: you can't grant a specific team or GitHub App the ability to bypass rules without making them full admins on the repo. The access model is binary.
No evaluation mode: you can't test what a new protection would block before you enable it. You enforce or you don't.

What Rulesets Are and How They Differ

A Ruleset is GitHub's next-generation enforcement layer - it can target branches and tags, applies at the repo or organization level, supports named bypass actors, and can be exported and version-controlled as JSON.

Rulesets were introduced for GitHub Enterprise and are now available on all plan tiers. They don't replace the classic system immediately - you can run both simultaneously - but they are strictly more capable in every dimension that matters for compliance and security.

Capability	Classic Branch Protection	Rulesets
Applies to branches	✅	✅
Applies to tags	❌	✅
Organization-level enforcement	❌	✅
Bypass actors (non-admin)	❌	✅
Admin bypass (default)	✅ (admins bypass by default)	Configurable - admins can be included or excluded
Multiple rulesets per repo	❌	✅
Exportable as JSON	❌	✅
Evaluation mode (audit without enforcing)	❌	✅
Targets by branch name pattern	✅	✅
Full fnmatch pattern support	Limited	✅

The Capabilities Worth Understanding Before You Migrate

Bypass Actors

This is the most important capability Rulesets add. Instead of the binary admin/non-admin split, Rulesets let you define specific bypass actors - entities that are permitted to bypass rules under defined conditions:

A specific team - your platform engineering team can push hotfixes directly to main without a PR; no one else can
A specific GitHub App - your release automation app can create and delete version tags; human engineers cannot
Repository roles - Maintainer role can bypass; Contributor role cannot

The bypass_mode field is particularly useful. Setting bypass_mode: "pull_request" means the bypass actor can still only merge via a pull request - they bypass the status check or review requirements, but not the PR itself. This lets you grant trusted actors flexibility without removing the audit trail that comes with PR history.

Evaluation Mode

Before enforcing a new Ruleset, set its enforcement to evaluate. In evaluation mode, GitHub runs all the checks and logs what would have been blocked - without actually blocking anything. This is indispensable for organizations rolling out standards across many repos: you see the blast radius before anyone's work is interrupted.

Run a Ruleset in evaluate mode for one to two weeks. If nothing surprising surfaces in the audit log, switch to active. If something does surface, you've caught it before it becomes an incident.

Tag Protection

Classic branch protection has no equivalent for tags. Rulesets close this. A tag-targeting Ruleset prevents deletion, non-fast-forward updates, and unauthorized creation of version tags:

{
  "name": "Protect release tags",
  "target": "tag",
  "enforcement": "active",
  "conditions": {
    "ref_name": {
      "include": ["refs/tags/v*"],
      "exclude": []
    }
  },
  "rules": [
    { "type": "deletion" },
    { "type": "non_fast_forward" },
    { "type": "creation" }
  ]
}

The creation rule blocks all creation of matching refs by default - only bypass actors can create v* tags. If your release process creates tags through a GitHub App or Actions bot, add that actor as a bypass actor on this Ruleset. Human engineers - including admins - are blocked by default.

Organization-Level Rulesets

A single Ruleset defined at the organization level applies to all repos in that org, or to a filtered subset by repo name pattern. This is the answer to "how do we enforce our branching standards across all 200 repositories" - one Ruleset, not 200 individual configuration changes. Repos can layer additional repo-level Rulesets on top of the org baseline; the most restrictive rule wins when rules conflict.

A Complete Ruleset for a Typical Project

The following is a production-ready Ruleset for protecting the main branch of a typical open-source or team project. You can import it directly through the GitHub UI (Repository → Settings → Rules → Rulesets → Import) or apply it via the API.

{
  "name": "Protect main branch",
  "target": "branch",
  "enforcement": "active",
  "conditions": {
    "ref_name": {
      "include": ["refs/heads/main"],
      "exclude": []
    }
  },
  "bypass_actors": [
    {
      "actor_id": 1,
      "actor_type": "OrganizationAdmin",
      "bypass_mode": "pull_request"
    }
  ],
  "rules": [
    {
      "type": "deletion"
    },
    {
      "type": "non_fast_forward"
    },
    {
      "type": "pull_request",
      "parameters": {
        "required_approving_review_count": 1,
        "dismiss_stale_reviews_on_push": true,
        "require_code_owner_review": false,
        "require_last_push_approval": true,
        "allowed_merge_methods": ["squash", "merge"]
      }
    },
    {
      "type": "required_status_checks",
      "parameters": {
        "strict_required_status_checks_policy": true,
        "required_status_checks": [
          {
            "context": "build / compile",
            "integration_id": null
          },
          {
            "context": "test / unit-tests",
            "integration_id": null
          }
        ]
      }
    },
    {
      "type": "required_signatures"
    }
  ]
}

A few choices worth explaining:

bypass_mode: "pull_request" on the OrganizationAdmin actor: org admins can still bypass review and status check requirements, but they can't push directly to main - they still have to open a PR. The audit trail stays intact.
require_last_push_approval: true: the person who made the last push to a PR branch cannot be the one who approves the merge. This prevents a single developer from self-approving their own changes by pushing a trivial amendment to reset the review state.
strict_required_status_checks_policy: true: the branch must be up to date with main before merging. Disabling this allows a PR to merge even if its base has drifted in ways that would break the combined result.
allowed_merge_methods: restricting to squash and merge (excluding rebase) is a project-specific choice - squash keeps main history linear and readable; including merge accommodates workflows that want to preserve PR structure. Adjust to match your conventions.

Replace build / compile and test / unit-tests with the actual check names from your Actions workflows. The names in required_status_checks must match exactly - including the <job-name> / <step-name> format that Actions generates.

Auditing Ruleset Coverage Across an Org

Rulesets are only useful if they're actually configured. As your organization grows, repos get created without anyone ensuring the baseline standards are applied. The following GitHub Actions workflow runs weekly and fails visibly if any repo in the org has no active Rulesets:

name: Audit Ruleset Coverage
on:
  schedule:
    - cron: '0 9 * * 1'  # Every Monday at 9am
  workflow_dispatch:

permissions:
  contents: read

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - name: Find repos without active Rulesets
        env:
          GH_TOKEN: ${{ secrets.ORG_READ_TOKEN }}
          ORG: ${{ vars.ORG_NAME }}
        run: |
          echo "Checking Ruleset coverage for org: $ORG"

          # Get all repos in the org
          repos=$(gh api /orgs/$ORG/repos --paginate \
            --jq '.[].name')

          uncovered=()

          while IFS= read -r repo; do
            ruleset_count=$(gh api /repos/$ORG/$repo/rulesets \
              --jq '[.[] | select(.enforcement == "active")] | length' \
              2>/dev/null || echo "0")

            if [ "$ruleset_count" -eq "0" ]; then
              uncovered+=("$repo")
            fi
          done <<< "$repos"

          if [ ${#uncovered[@]} -eq 0 ]; then
            echo "✅ All repos have active Rulesets configured."
          else
            echo "⚠️  Repos missing active Rulesets:"
            printf '  - %s\n' "${uncovered[@]}"
            exit 1
          fi

Two things to know about running this:

ORG_READ_TOKEN needs repo scope to read private repository metadata, or read:org if you're working with org-level Rulesets. Store it as a repository secret on wherever this workflow lives - a dedicated platform-engineering repo works well. ORG_NAME is a repository variable (not a secret) set to your GitHub organization name.

The workflow exits with code 1 when uncovered repos are found. That means it fails visibly in the Actions UI and can trigger notifications. You can extend it to open a GitHub Issue automatically or post to Slack, but the exit code alone is enough to make the gap impossible to ignore in a weekly check-in workflow.

Note that this audit only detects repos with no active Rulesets at all - it doesn't validate that the Rulesets that exist are correctly configured. For more granular compliance checking, extend the inner loop to inspect specific rule types against your organization's baseline requirements.

The Migration Path

This doesn't need to be a big-bang migration. Here's a sequence that keeps risk low.

1. Enable Rulesets in parallel. Create a Ruleset that mirrors your existing branch protection rules and set enforcement to evaluate. Run it for two weeks. Check the Insights tab under Repository → Settings → Rules - it shows every rule evaluation and whether it would have been blocked. Confirm nothing unexpected surfaces.

2. Map your bypass actors. Who on your team legitimately needs to bypass rules? Your release automation bot? A platform team doing emergency hotfixes? Write that list down and map each actor to a Ruleset bypass actor. Stop relying on admin status as a proxy for "trusted to bypass."

3. Add tag protection immediately. If you use version tags (v1.2.3, v2.0.0-rc.1), you almost certainly have no protection on them right now. Add a tag-targeting Ruleset today - this is the change with the best risk-to-effort ratio in this entire post.

4. Check your admin bypass exposure. In your existing classic branch protection, is "Include administrators" checked? If not, every repo admin bypasses every rule. Fix this in the Ruleset (the bypass_mode: "pull_request" pattern shown above), or add it to the classic rules as an immediate stopgap while you migrate.

5. For orgs with many repos: define one org-level Ruleset for baseline standards. Individual repos can add repo-level Rulesets on top for project-specific requirements.

6. Once confident, disable classic branch protection. Running both simultaneously isn't dangerous - the stricter rule always wins - but it is confusing. When a developer asks "why can't I merge this?" and the answer requires knowing which system is blocking them, you've created an unnecessary support burden. Once your Rulesets are active and validated, remove the classic rules.

Migration Checklist

[ ] Check existing branch protection: is "Include administrators" enabled on every protected branch? If not, fix it first - this is your current exposure.
[ ] Create a mirror Ruleset in evaluate mode and run it for 1-2 weeks; review the Insights log for unexpected evaluations
[ ] Map your bypass needs: list who legitimately needs to bypass rules and map each to a named bypass actor (team, app, or role)
[ ] Add tag protection for release tags (v*) - classic branch protection offers nothing here
[ ] For multi-repo orgs: define an org-level baseline Ruleset that applies to all repositories
[ ] Set the audit workflow to run on a weekly schedule
[ ] Once Rulesets are active and validated: disable classic branch protection to eliminate confusion about which system is enforcing what

Classic branch protection did the job for years, but it was designed for a simpler model - one repo, one team, admin-or-not access control. Rulesets are designed for the actual complexity of modern engineering organizations: multiple repos, mixed access models, automated actors, and the need to audit compliance across all of it. The migration isn't urgent. But the admin bypass exposure - the protection that silently disappears for the people most likely to push directly to main under pressure - is reason enough to start this week. That's not a theoretical gap. It's the default configuration.

Want to talk through Ruleset strategy for your organization, or get help designing a bypass actor model that matches your team's actual access needs? Reach out.

steve.kaschimer@slalom.com

Shift Right: Why Production Observability Is a Security Practice

2026-05-15T00:00:00Z

Shift left is right. Catch vulnerabilities in code review, in CI, in dependency scanning - before they ship. The DevSecOps community has invested heavily here and the tooling is genuinely good. But shift left has a blind spot: it only covers the attack surface you anticipated. Attackers don't respect your threat model. They operate in production, against live systems, exploiting behavior that passed every scan because it wasn't a vulnerability until it was combined with a specific runtime condition, a specific data pattern, or a specific sequence of requests. The answer isn't to abandon shift left - it's to complement it with shift right: instrumentation in production that makes anomalous behavior visible before it becomes an incident.

The reason this gets skipped is understandable. Shift-left tooling is largely automatic - you plug in Dependabot, add a CodeQL workflow, and the scans run without anyone thinking about them. Shift-right requires decisions: what to instrument, what thresholds to set, where to route alerts, how to avoid drowning your team in noise. Those decisions aren't hard, but they require intention. This post makes the argument for doing it, shows how to instrument a Node.js service with OpenTelemetry for security-relevant signals, and routes those signals into GitHub Security alerts - where your team is already reviewing Dependabot and CodeQL findings - so the operational overhead stays close to zero.

Observability Data and Security Data Are the Same Data

Here is the core argument: everything your security team wants to see in a SIEM is already flowing through your observability stack. The difference is framing and threshold, not source.

A spike in 4xx errors might be a bug. It might be credential stuffing.
Unusual outbound connections might be a misconfigured integration. They might be C2 beacon traffic.
A process spawning a child process might be a legitimate job runner. It might be a shell injection exploit in progress.
Authentication failures at 3am from a single IP might be a developer locked out. It might be a brute-force attempt.

None of these interpretations require different data. They require different context and different thresholds applied to the same stream. Most organizations run separate tooling for observability (Datadog, Grafana, OpenTelemetry collectors) and security (SIEM, GitHub Security alerts, Dependabot). They are looking at the same signals twice through different products, paying twice, and - critically - the integration between the two stacks is usually manual, asynchronous, and owned by nobody. The better model is to define security-relevant thresholds inside your observability instrumentation and route high-severity signals to security tooling directly.

Observability asks: "What is my system doing right now?" Security asks: "Is my system being abused right now?" At the production layer, these questions share an answer.

The Three Signal Categories Worth Instrumenting for Security

Not everything in your telemetry is security-relevant. The goal is not to route all traces into a SIEM - it's to identify the narrow set of signals where an anomaly is evidence of abuse rather than just a bug. Three categories cover most of what matters for a typical web application.

Authentication and Authorization Signals

Failed logins are the most observable leading indicator of account-based attacks. What to track:

Failed authentication attempts - rate, source IP, target account
Successful auth after N prior failures - credential stuffing success signal; this is the event that matters most
Privilege escalation events - role changes, permission grants, admin flag toggles
Token anomalies - same token presented from multiple distinct IPs, token used after recorded revocation

The trick with auth signals is baselining. A handful of failed logins per hour is normal. Fifty from a single IP over five minutes is not. Establish a 30-day rolling baseline for failed auth rate per hour, alert when the current rate exceeds 3× baseline sustained for five or more minutes. This single threshold eliminates the noise from developers mistyping their passwords while catching the early ramp of a stuffing attack.

Outbound Connection Patterns

Server-side applications have a predictable set of outbound destinations. Your API calls GitHub, Slack, maybe a payment processor. That list changes infrequently and is known in advance. An unexpected outbound connection is anomalous by definition - it doesn't require a baseline, because the baseline is your allowlist.

What to track:

Outbound connections to new or unknown hosts - first-time destinations not on the allowlist
DNS resolution of domains outside the allowlist - catches connections that haven't been made yet
Data volume spikes on established outbound connections - potential exfiltration even to a known destination

This pattern is particularly high-value because it catches post-exploitation activity. An attacker who has achieved code execution in your application will eventually try to beacon home or exfiltrate data. The outbound connection is often the first observable evidence.

Process and Runtime Behavior

For Node.js specifically, child_process.exec and child_process.spawn with user-controlled input are a common injection vector. Logging every process spawn with its full argument list is cheap - the events are rare in a healthy application - and the signal-to-noise ratio when something anomalous appears is excellent.

What to watch for:

Unexpected child process spawns, especially shell interpreters: sh, bash, cmd.exe, powershell.exe
File system writes to locations outside expected application directories - binaries dropping to /tmp, writes to cron directories
Memory usage anomalies - some exploit payloads and deserialization attacks cause distinctive heap growth patterns

A Node.js web application should almost never spawn a shell. If one appears in your process spawn log, that is not a false positive you tune away - it is the beginning of an incident response.

Instrument spawns by wrapping child_process.spawn at a single callsite in your application:

// lib/safe-spawn.js
import { spawn as _spawn } from 'child_process';
import { trace } from '@opentelemetry/api';

const SHELL_INTERPRETERS = new Set(['sh', 'bash', 'zsh', 'cmd.exe', 'powershell.exe', 'pwsh']);

export function spawn(command, args = [], options = {}) {
  const fullCommand = [command, ...args].join(' ');
  const isShell = SHELL_INTERPRETERS.has(command);

  console.log(JSON.stringify({
    level: isShell ? 'error' : 'info',
    event: 'process.spawn',
    command,
    args: args.join(' '),
    is_shell: isShell,
    timestamp: new Date().toISOString(),
  }));

  const span = trace.getActiveSpan();
  if (span) {
    span.setAttributes({
      'process.spawn.command': command,
      'process.spawn.args': args.join(' '),
      ...(isShell && { 'security.anomaly': 'shell_spawn', 'security.severity': 'high' }),
    });
  }

  return _spawn(command, args, options);
}

Use lib/safe-spawn.js everywhere in your application instead of importing child_process directly - the wrapper is a eslint rule away from being enforced (no-restricted-imports).

Instrumenting a Node.js App with OpenTelemetry

OpenTelemetry's auto-instrumentation handles the baseline - HTTP spans, framework-level timing, error tracking - without code changes. The security-relevant work is adding custom attributes to existing spans and emitting structured logs alongside them.

Install the core packages:

npm install @opentelemetry/sdk-node \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/exporter-otlp-http

Bootstrap the SDK - load this before anything else using --require ./instrumentation.js. Using spanProcessors explicitly lets you compose the export pipeline with custom processors (the OutboundAnomalyProcessor defined below):

// instrumentation.js
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-otlp-http';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-node';
import { OutboundAnomalyProcessor } from './processors/outbound-anomaly.js';

const exporter = new OTLPTraceExporter({
  url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
});

const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(exporter),   // normal export pipeline
    new OutboundAnomalyProcessor(),     // security annotation layer
  ],
  instrumentations: [
    getNodeAutoInstrumentations({
      '@opentelemetry/instrumentation-http': { enabled: true },
      '@opentelemetry/instrumentation-express': { enabled: true },
    }),
  ],
});

sdk.start();

Add security-relevant attributes in your auth middleware:

// middleware/auth.js
import { trace, SpanStatusCode } from '@opentelemetry/api';

export function authMiddleware(req, res, next) {
  const span = trace.getActiveSpan();

  if (span) {
    span.setAttributes({
      'auth.method': req.headers['authorization'] ? 'bearer' : 'none',
      'auth.ip': req.ip,
      'auth.user_agent': req.headers['user-agent'] ?? 'unknown',
    });
  }

  // ... auth logic
  if (authFailed) {
    span?.setAttributes({ 'auth.failure_reason': 'invalid_credentials' });
    span?.setStatus({ code: SpanStatusCode.ERROR, message: 'auth_failure' });

    // Structured log alongside the span - queryable by your log aggregator
    console.log(JSON.stringify({
      level: 'warn',
      event: 'auth.failure',
      ip: req.ip,
      target: req.body?.username ?? 'unknown',
      timestamp: new Date().toISOString(),
    }));
  }

  next();
}

Flag unknown outbound destinations using a custom span processor. OpenTelemetry's HTTP auto-instrumentation captures outbound calls automatically; this processor adds a security annotation when the destination isn't recognized:

// processors/outbound-anomaly.js
const KNOWN_DESTINATIONS = new Set([
  'api.github.com',
  'hooks.slack.com',
  // ... your known outbound hosts
]);

export class OutboundAnomalyProcessor {
  onStart(span, _parentContext) {
    const host = span.attributes['net.peer.name'];
    if (host && !KNOWN_DESTINATIONS.has(host)) {
      span.setAttributes({
        'security.anomaly': 'unknown_outbound_destination',
        'security.severity': 'medium',
      });
      console.log(JSON.stringify({
        level: 'warn',
        event: 'outbound.unknown_destination',
        host,
        timestamp: new Date().toISOString(),
      }));
    }
  }
  onEnd() {}
  shutdown() { return Promise.resolve(); }
  forceFlush() { return Promise.resolve(); }
}

Register the processor in your SDK config by passing it to spanProcessors alongside the exporter. The onStart hook fires before the span is exported, so the attribute is visible in both your trace backend and your log stream.

Routing Security Signals to GitHub Security Alerts

Your team already reviews Dependabot and CodeQL findings in the GitHub Security tab. Routing runtime signals to the same interface means one fewer tool to check and no new alerting channel to maintain. GitHub's Code Scanning API accepts SARIF (Static Analysis Results Interchange Format) - a JSON schema that was designed for static analysis but works equally well as a transport for runtime anomaly results.

A GitHub Actions workflow to ingest a batch of security signals:

name: Ingest Security Signals
on:
  workflow_dispatch:
    inputs:
      signals_file:
        description: 'Path to security signals JSON file'
        required: true

permissions:
  security-events: write
  contents: read

jobs:
  ingest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Convert signals to SARIF
        id: convert
        run: |
          node scripts/signals-to-sarif.js \
            --input "${{ inputs.signals_file }}" \
            --output security-signals.sarif

      - name: Upload SARIF to GitHub Security
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: security-signals.sarif
          category: runtime-security

The category: runtime-security field keeps runtime signals visually separated from your static analysis results in the Security tab. The SARIF your conversion script emits needs to be schema-valid - here is the minimal structure for an auth failure spike result:

{
  "version": "2.1.0",
  "runs": [{
    "tool": {
      "driver": {
        "name": "runtime-security",
        "rules": [{
          "id": "auth-failure-spike",
          "name": "AuthenticationFailureSpike",
          "shortDescription": { "text": "Elevated authentication failure rate" },
          "defaultConfiguration": { "level": "warning" }
        }]
      }
    },
    "results": [{
      "ruleId": "auth-failure-spike",
      "message": { "text": "Auth failure rate 4.2x baseline for 8 minutes from 203.0.113.42" },
      "level": "warning",
      "locations": [{
        "physicalLocation": {
          "artifactLocation": { "uri": "src/middleware/auth.js" }
        }
      }]
    }]
  }]
}

The physicalLocation.artifactLocation.uri maps the alert to the relevant source file - the auth middleware, the HTTP client wrapper, wherever the instrumented code lives. GitHub uses this to anchor the alert in code review context.

The signals-to-sarif.js conversion script reads a newline-delimited JSON file of signal objects emitted by your application (each line: { ruleId, message, level, uri }) and produces the SARIF document:

// scripts/signals-to-sarif.js
import { readFileSync, writeFileSync } from 'fs';
import { parseArgs } from 'util';

const { values } = parseArgs({
  options: {
    input:  { type: 'string' },
    output: { type: 'string' },
  },
});

const signals = readFileSync(values.input, 'utf8')
  .trim().split('\n').map(line => JSON.parse(line));

const ruleIds = [...new Set(signals.map(s => s.ruleId))];

const sarif = {
  version: '2.1.0',
  runs: [{
    tool: {
      driver: {
        name: 'runtime-security',
        rules: ruleIds.map(id => ({
          id,
          name: id.replace(/-([a-z])/g, (_, c) => c.toUpperCase()),
          shortDescription: { text: id },
          defaultConfiguration: { level: 'warning' },
        })),
      },
    },
    results: signals.map(s => ({
      ruleId: s.ruleId,
      message: { text: s.message },
      level: s.level ?? 'warning',
      locations: [{ physicalLocation: { artifactLocation: { uri: s.uri } } }],
    })),
  }],
};

writeFileSync(values.output, JSON.stringify(sarif, null, 2));
console.log(`Wrote ${signals.length} signal(s) to ${values.output}`);

Your runtime alerting code (rate-threshold logic, correlation checks) writes to the signals file; this script is the schema adapter. Keep them separate - the converter is stateless and easy to test; the threshold logic is where the real decisions live.

Defining Thresholds That Distinguish Abuse from Bugs

The hardest part of runtime security alerting is the signal-to-noise ratio. Alert on everything anomalous and the team stops reading alerts within a week. Alert on nothing and the whole exercise is a waste of instrumentation work. A practical framework has two layers.

Rate-based thresholds apply to signals with a predictable normal distribution - auth failures, error rates, request volume:

Establish a 30-day rolling baseline per metric
Alert at 3× baseline sustained for 5+ minutes - this filters transient spikes from deploys and traffic bursts
Page at 10× baseline sustained for 2+ minutes - this is almost certainly not normal traffic under any reasonable interpretation

Novelty-based thresholds apply to signals that should be near-zero in a healthy application - no baseline required:

First occurrence of a new outbound destination: medium alert
First occurrence of a process spawn with a shell interpreter: high alert immediately (this should not happen in a Node.js web server)
First successful auth from an IP that had 10+ prior failures in the preceding hour: high alert

Correlation is where the signal-to-noise math actually improves. Single anomalous events are noisy - each one has plausible innocent explanations. Two correlated anomalies from the same source within a short window have very few:

Auth failure spike from IP X and successful auth from IP X within 30 minutes → credential stuffing success, critical
Error rate spike and unknown outbound connection and new process spawn within the same 5-minute window → potential active exploit, critical

Correlation is the difference between alert fatigue and actionable detection. Build it into your threshold logic before you go live with alerting.

Shift-Right Implementation Checklist

[ ] Install OpenTelemetry with @opentelemetry/auto-instrumentations-node - baseline HTTP and framework spans are free
[ ] Add security-relevant attributes to auth spans: IP, user agent, failure reason
[ ] Log every auth failure as structured JSON (not free-text) - machines need to count these
[ ] Build a known-destinations allowlist and use OutboundAnomalyProcessor to flag first-time outbound connections
[ ] Log every child_process spawn with full argument list
[ ] Define 30-day rolling rate baselines for auth failures and error rates
[ ] Set alert thresholds at 3× baseline sustained for 5 minutes; page at 10×
[ ] Add correlation rules: auth spike + auth success from same IP; error spike + unknown outbound + process spawn
[ ] Route high-severity signals to GitHub Security via SARIF upload with category: runtime-security
[ ] Review security signal alerts in the same workflow as Dependabot and CodeQL - same interface, same team, same weekly review cadence

Shift left is necessary. It is not sufficient. Every security posture has a boundary where it stops - and for shift-left tooling, that boundary is the production environment. Attackers don't respect that boundary. They operate where your SAST scanner doesn't run, where your dependency auditor has no visibility, where the only evidence of compromise is a pattern in your logs that nobody was watching for.

Observability instrumentation doesn't require a dedicated security team or a SIEM contract. It requires structured logging, OpenTelemetry, a set of thresholds, and the discipline to review alerts in the same tool where you review everything else. The shift is small. The coverage gap it closes is not.

Questions about OpenTelemetry instrumentation strategy, threshold design, or wiring runtime signals into your existing GitHub security workflow? Reach out.

steve.kaschimer@slalom.com

Enforcing Code Quality with GitHub Actions Status Checks You Can Actually Trust

2026-05-29T00:00:00Z

Your required status checks are configured. CI is green. PRs merge. The team believes the gate is working.

Then someone asks: what happens if a developer pushes directly to main? What happens when the test suite takes 18 minutes and developers start merging before results are in - or, worse, start suppressing failures to keep velocity up? What happens when a job is renamed in the workflow file and silently stops matching the required check name, so every PR from that point forward merges without the check running at all?

Status checks are only a quality gate if they are fast, reliable, correctly named, and actually enforced. Most pipelines satisfy one or two of those criteria. This post covers all four failure modes and a concrete fix for each.

Failure Mode 1: Flaky Tests That Erode Trust

When tests fail non-deterministically, developers learn to re-run rather than investigate. That's rational behavior - if the test failed without a code change, the fix is to run it again. But it's catastrophic for a quality gate. Once re-running is the default response to a red check, the check stops functioning as a signal. It becomes a speed bump: an obstacle to clear, not a question to answer.

The erosion compounds. A developer re-runs a flaky test and it passes. The next time, a different developer re-runs a different flaky test. After enough repetitions, the implicit team norm becomes "if it's red, retry it." Actual failures get retried and missed.

Identifying Flaky Tests

The gh CLI makes it straightforward to spot patterns. Look for jobs that fail on one run and pass on a re-run without any code change between them:

# Re-run only the failed jobs from the last workflow run
gh run rerun --failed <run-id>

# View the history of runs on a workflow, filtering by conclusion
gh run list --workflow ci.yml --limit 50 \
  --json conclusion,headBranch,headSha \
  | jq '.[] | select(.conclusion == "failure")'

Compare the failing run's SHA against subsequent passing runs. If the same SHA both fails and passes across runs, those tests are flaky by definition.

The Quarantine Pattern

The immediate mitigation is to quarantine flaky tests into a separate job that does not block merge:

jobs:
  tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test -- --testPathIgnorePatterns="flaky"

  flaky-tests:
    runs-on: ubuntu-latest
    continue-on-error: true   # doesn't block merge
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test -- --testPathPattern="flaky"

continue-on-error: true means the flaky-tests job can fail without failing the workflow. It runs, it reports, but it does not gate. The tests job runs everything except the quarantined set and does gate.

This is a mitigation, not a fix. The real fix is to treat flaky tests as bugs with a defined SLA - they cost more in lost trust than they save in coverage. A quarantined test that doesn't get scheduled for repair inside a sprint or two should be deleted. Zero coverage is more honest than false coverage.

Failure Mode 2: Slow Pipelines That Developers Route Around

18-minute CI is a social engineering problem as much as a technical one. When developers know the check takes that long, some will merge "probably fine" work before results are in. Others will pressure reviewers to approve before CI completes. The pipeline hasn't broken - it's been socially bypassed. The 10-minute rule: if your full CI pipeline exceeds 10 minutes, you will observe developers routing around it within three months. Not because they're careless, but because the feedback loop is too slow to respect.

Parallelizing Linters and Tests

The most common source of unnecessary serial time is running linters, type checks, and tests sequentially in one job. These have no dependency on each other. They should run in parallel:

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run lint

  typecheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm run typecheck

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test

These three jobs run simultaneously. The workflow completes in max(lint, typecheck, test) time, not sum. If each takes three minutes, the total is three minutes instead of nine. cache: 'npm' in actions/setup-node handles dependency caching keyed on package-lock.json - on a cache hit, npm ci takes seconds.

Using `paths` Filters to Skip Irrelevant Checks

A PR that only changes README.md or docs/ does not need to run the full test suite. The paths trigger filter skips the workflow entirely for those PRs - zero runner minutes consumed:

on:
  pull_request:
    paths:
      - 'src/**'
      - 'package.json'
      - 'package-lock.json'
      - '.github/workflows/**'

There is an important caveat. If a required status check is configured and the workflow is skipped by a paths filter, GitHub marks the check as Expected rather than Success - and the PR is blocked. The check never ran, so GitHub doesn't know if it would have passed.

The fix is a ci-gate job that always runs and evaluates whether the real checks passed or were legitimately skipped. This is covered in detail in Failure Mode 3 below - the ci-gate pattern is the solution to both the bypass problem and the path-filter problem simultaneously.

The Path-Aware CI Pattern

The production-correct approach uses dorny/paths-filter to detect which paths changed and make downstream jobs conditional:

name: CI

on:
  pull_request:

jobs:
  changes:
    runs-on: ubuntu-latest
    outputs:
      src: ${{ steps.filter.outputs.src }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            src:
              - 'src/**'
              - 'package-lock.json'

  test:
    needs: changes
    if: needs.changes.outputs.src == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci && npm test

  # Always runs - this is the job configured as the required status check
  ci-gate:
    needs: [changes, test]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Check all required jobs passed or were skipped
        run: |
          if [[ "${{ needs.test.result }}" == "failure" ]]; then
            echo "Tests failed"
            exit 1
          fi
          echo "CI gate passed"

The ci-gate job runs on every PR without exception (if: always()). It checks the result of the test job: if test failed, ci-gate fails. If test was skipped because the paths didn't match, ci-gate passes - that's a legitimate skip. ci-gate is the one job name registered as a required status check. It is the stable contract between the workflow and branch protection.

Failure Mode 3: Required Checks That Can Be Bypassed

A vacuous pass occurs when a required status check is configured by name, but no check with that name runs on a given PR - GitHub treats an expected-but-absent check as satisfied and allows the merge to proceed.

This is the most insidious failure mode because it looks correct from the outside. Branch protection is enabled. Required checks are configured. PRs require checks to pass. But if the job was renamed - unit-tests became jest during a test runner migration, or a workflow file was restructured - the old required check name no longer matches any running check. Every PR from that point forward merges without running it.

This is a naming contract problem. The required check name in branch protection is a string. The job name in the workflow is a string. Nothing enforces that they match.

Auditing the Mismatch

Check what's configured as required against what actually ran on recent PRs:

# List configured required status checks for main
gh api /repos/{owner}/{repo}/branches/main/protection \
  --jq '.required_status_checks.contexts'

# List checks that actually ran on the last merged PR
gh api /repos/{owner}/{repo}/commits/$(gh api /repos/{owner}/{repo}/pulls \
  --jq 'map(select(.state=="closed")) | first | .merge_commit_sha')/check-runs \
  --jq '[.check_runs[] | {name, status, conclusion}]'

Compare the two lists. Any name in the first list that doesn't appear in the second list is a vacuous pass waiting to happen - or already happening.

Configuring Required Checks via the API

Clicking required checks in the UI works but isn't reproducible or auditable. Set them via the API instead:

# Set required status checks via API
# PUT replaces the full protection config - include null for unused fields
gh api --method PUT /repos/{owner}/{repo}/branches/main/protection \
  --input - <<'EOF'
{
  "required_status_checks": {
    "strict": true,
    "contexts": ["ci-gate"]
  },
  "enforce_admins": true,
  "required_pull_request_reviews": null,
  "restrictions": null
}
EOF

enforce_admins=true removes the admin bypass from classic branch protection - without it, repository admins skip all rules by default. (For a more complete treatment of admin bypass and why Rulesets are the better long-term answer, see the May 8 post on Rulesets.)

Using Rulesets instead of classic branch protection, the required check is expressed as JSON that can be version-controlled and applied programmatically:

{
  "type": "required_status_checks",
  "parameters": {
    "strict_required_status_checks_policy": true,
    "required_status_checks": [
      { "context": "ci-gate" }
    ]
  }
}

One required check name - ci-gate - that never changes. The workflow internals (job names, parallel vs. serial structure, which linter runs) can be refactored freely without ever touching branch protection configuration. The contract is between branch protection and ci-gate. Everything else is internal.

Failure Mode 4: Coverage Enforcement With No Floor

Tests pass. Coverage is tracked. The badge on the README shows 73%. A PR removes a feature and its tests together - coverage drops to 65%. It merges. Nothing blocked it.

Status checks can enforce a coverage floor, but most teams configure the check without one. Tracking coverage and enforcing a minimum are different things. The badge is decorative. The threshold is functional.

Enforcing a Floor in the Workflow

For Jest, threshold enforcement is built in - the --coverageThreshold flag causes the process to exit non-zero if the threshold isn't met, which fails the step, which fails the job, which blocks the merge:

- name: Run tests with coverage
  run: npm test -- --coverage --coverageThreshold='{"global":{"lines":80}}'

For other frameworks, read the coverage report and fail explicitly:

- name: Check coverage threshold
  run: |
    COVERAGE=$(cat coverage/coverage-summary.json | jq '.total.lines.pct')
    echo "Line coverage: $COVERAGE%"
    if (( $(echo "$COVERAGE < 80" | bc -l) )); then
      echo "Coverage $COVERAGE% is below the 80% threshold"
      exit 1
    fi

Threshold Philosophy

The floor should reflect your current coverage, not your aspirational coverage. Setting 80% when you're at 65% blocks every PR until someone dedicates a sprint to coverage work - which creates pressure to delete tests rather than write them, which is the opposite of the intent.

Set the threshold at current coverage minus a small buffer. If you're at 73%, set 70%. This prevents regression without creating an immediate blocker. Then raise it quarterly, by five points at a time, as part of normal engineering hygiene. A threshold that moves deliberately upward is more useful than one set aspirationally high and immediately disabled under pressure.

The Stable Required Check Pattern

These four failure modes share a common solution: a single ci-gate job that acts as the stable contract between workflow internals and branch protection configuration.

PR opens
  → changes job:  detect which paths changed
  → lint job:     runs only if src/ changed (skipped otherwise)
  → test job:     runs only if src/ changed (skipped otherwise; coverage threshold enforced inline)
  → ci-gate job:  always runs, fails if lint or test failed
        ↑
        Required status check in Ruleset

The properties that make this pattern reliable:

One stable check name configured in branch protection or Rulesets: ci-gate. This string never needs to change regardless of what happens inside the workflow.
ci-gate always runs - if: always() ensures it has a result on every PR, including PRs where the real checks were skipped by path filters.
ci-gate validates outcomes - it passes if the real checks passed or were legitimately skipped; it fails if any real check failed.
Workflow internals can change freely - add a linter, rename a job, restructure parallelism. None of those changes affect the string registered in branch protection.
Coverage threshold lives inside a job that ci-gate depends on - the test job exits non-zero if coverage falls below the threshold, which propagates through needs and surfaces as a ci-gate failure.

The complete wiring for a typical project:

name: CI

on:
  pull_request:

jobs:
  changes:
    runs-on: ubuntu-latest
    outputs:
      src: ${{ steps.filter.outputs.src }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            src:
              - 'src/**'
              - 'package-lock.json'

  lint:
    needs: changes
    if: needs.changes.outputs.src == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci && npm run lint

  test:
    needs: changes
    if: needs.changes.outputs.src == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: >
          npm ci && npm test -- --coverage
          --coverageThreshold='{"global":{"lines":80}}'

  ci-gate:
    needs: [changes, lint, test]
    if: always()
    runs-on: ubuntu-latest
    steps:
      - name: Check all required jobs passed or were skipped
        run: |
          for result in "${{ needs.lint.result }}" "${{ needs.test.result }}"; do
            if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
              echo "A required job did not pass: $result"
              exit 1
            fi
          done
          echo "CI gate passed"

Status Check Audit Checklist

Run this against your current setup before the next sprint planning:

[ ] List your required status check names (gh api /repos/{owner}/{repo}/branches/main/protection --jq '.required_status_checks.contexts') and verify they match actual job names running on recent PRs

[ ] Check whether "Include administrators" is enforced on classic branch protection - or migrate to Rulesets (see May 8 post) where admin bypass is explicitly configurable

[ ] Identify any test jobs that fail non-deterministically across the last 20 runs - quarantine them with continue-on-error: true and file them as bugs with an SLA

[ ] Measure CI wall-clock time on the last 10 PRs - is the median under 10 minutes? If not, identify which serial jobs can be parallelized

[ ] If using paths filters on the workflow trigger: verify you have a ci-gate job with if: always() that is the registered required check name

[ ] Set a coverage threshold at current coverage minus 2-3% - then raise it 5 points per quarter

[ ] Name the job registered as required status check something stable (ci-gate) so workflow refactoring never silently disables enforcement

Status checks are the enforcement layer between "code was reviewed" and "code meets quality standards." When they're slow, developers route around them. When they're flaky, developers stop trusting them. When they're misconfigured, they enforce nothing at all.

The fixes aren't complex - parallel jobs, a stable gate job, a coverage floor, path-aware skipping - but they require deliberate design. Default pipelines rarely have all four in place. A status check that developers respect and that actually enforces what it claims to enforce is one of the highest-leverage investments in code quality a team can make. It compounds: every PR, every merge, every release, automatically.

Questions about CI design, workflow architecture, or structuring required checks across an org? Reach out.

steve.kaschimer@slalom.com

GitHub Copilot in CI: Automating Code Review at Scale

2026-06-05T00:00:00Z

Most teams use GitHub Copilot the same way they use autocomplete: a single developer, in an editor, getting suggestions on the code in front of them. That's a real productivity gain. It's also a fraction of what Copilot can do in a CI context, where it can review every PR, on every push, without a human reviewer being available - and where the quality of its configuration determines whether it becomes a genuine signal or just another source of noise to dismiss.

This post covers how to wire up Copilot code review in a way that actually helps: what to configure, what to exclude, how to prevent it from becoming a gate that erodes trust, and how to read its output alongside human review rather than instead of it.

Wiring `github/copilot-code-review` in CI

The fastest way to operationalize Copilot review across every PR is the github/copilot-code-review action. It runs on PR events, analyzes the diff, and posts review comments directly on changed lines.

Start with a minimal workflow:

name: copilot-code-review

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]

jobs:
  review:
    if: github.event.pull_request.draft == false
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - uses: github/copilot-code-review@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

Three behavior details matter for rollout:

It never approves PRs. Treat Copilot as a comment stream, not an approver. Required reviewer policies should still be human-only.

It reviews the diff, not your whole architecture. It catches local issues in changed code; it does not replace design review.

It should be tuned before broad rollout. Scope and instruction quality determine whether teams see signal or noise.

Enabling It: Org Policy vs. Repo Workflow

Copilot review still requires Copilot Business or Enterprise, but delivery happens in two layers:

Organization or repository-level policy is the prerequisite that enables Copilot code review.
Repository workflow controls when and how github/copilot-code-review runs.

Org admins configure this in Settings → Copilot → Policies → Code review (or repository-level Copilot settings for per-repo control). If policy is disabled, the workflow will complete successfully in CI logs without posting any review comments on the PR.

This split is useful: platform admins enable the feature once, while each repo keeps execution details versioned, reviewable, and constrained to its own risk profile.

Configuring What Copilot Reviews

Out of the box, Copilot reviews everything in every diff. That is the wrong default for most real codebases. Machine-generated files, test fixtures, migration scripts, and vendored code will generate comments that nobody wants to read. Configuring scope before you roll this out to a team matters.

Path Filters

The action supports path_filters, so you can keep review focused on high-signal files:

- uses: github/copilot-code-review@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    path_filters: |
      - 'src/**'
      - '.github/workflows/**'

This should be conservative by default: include only high-signal source and workflow paths first, then expand as needed. Revisit filters as your repo evolves so Copilot keeps looking where humans want help.

Custom Review Instructions

Custom instructions tell Copilot what to focus on, what to ignore, and what your team's standards are. They live in .github/copilot-instructions.md alongside a code-review section, or in a dedicated .github/copilot-review-instructions.md file. The repository-level settings UI also accepts them directly under Settings → Copilot → Custom instructions.

A well-structured instruction file makes the difference between generic comments ("this could throw") and comments that are specific to your codebase conventions:

# Code Review Instructions

## Focus Areas

Review all diffs for:
- Unhandled promise rejections and error paths that silently swallow exceptions
- Hardcoded credentials, secrets, or environment-specific values that belong in config
- Direct SQL string concatenation that should use parameterized queries
- Missing input validation on any function that accepts external data
- N+1 query patterns in ORM usage

## Ignore

Do not comment on:
- Code style or formatting (handled by our linter - see .eslintrc)
- Test coverage percentage or lack of test files in this PR
- TODO comments (we track these separately)
- Performance micro-optimizations unless there is a clear O(n²) or worse pattern

## Context

This is a Node.js API service. Database access uses Prisma. Authentication uses JWT
tokens validated by the `verifyToken` middleware. Any route handler that does not
call `verifyToken` should be flagged unless it is explicitly in the public routes list
at `src/routes/public.ts`.

The instruction file is committed to the repository, versioned, and reviewable in PRs like any other configuration. That is a feature - when the team decides to add a new focus area or update a convention, the change goes through a PR, gets discussed, and lands with a clear commit history.

Custom instructions have a length limit (currently around 8,000 characters). Prioritize security-relevant patterns and team-specific conventions over general best practices; Copilot already knows general best practices. The value of custom instructions is encoding the knowledge that is specific to your codebase.

Preventing Copilot from "Approving Itself"

Copilot review comments are advisory, but teams often run separate automation that auto-approves bot PRs (for dependency updates or chore changes). Add an explicit Copilot denylist there so Copilot-authored changes cannot be auto-approved by bot logic:

jobs:
  auto-approve-bot-prs:
    if: >
      github.event.pull_request.user.login != 'github-copilot[bot]' &&
      github.event.pull_request.user.login != 'copilot-swe-agent[bot]'

github-copilot[bot] is the review bot identity, and copilot-swe-agent[bot] is a common bot identity for Copilot-authored PRs created by coding agents. Excluding both prevents bot-written changes from being auto-approved by bot-only logic.

Pair that with branch protection requiring at least one human approval. The practical rule: Copilot can suggest and comment, but a human still owns merge intent.

The Advisory vs. Hard Gate Decision

This is the most consequential configuration choice, and most teams get it wrong by defaulting to the wrong answer.

The wrong answer: make Copilot review a required gate that blocks merges when it posts REQUEST_CHANGES.

Why it's wrong: Copilot will flag real issues and non-issues with similar confidence. It will comment on patterns that are intentional in your codebase because it lacks the context to know they're intentional. Developers will encounter legitimate PRs blocked by a Copilot comment they disagree with and no clear path to dismiss it without either fixing something that doesn't need fixing or granting themselves an exception that undermines the gate's purpose.

Once a required gate generates enough friction that developers start working around it - requesting each other's approvals before Copilot finishes, or dismissing reviews as a reflex - it is no longer a quality gate. It's a ceremony. The damage is not just to velocity; it's to the team's belief that automated tooling can be trusted, which makes the next time someone proposes a real quality gate harder.

The right answer: run Copilot review as a non-blocking, advisory-only check. Copilot posts its comments. Developers see them. Human reviewers see them. The PR can merge with outstanding Copilot comments - that is fine. The comments are signal, not mandates.

The way to enforce this configuration is to ensure that Copilot is never configured as a required reviewer in branch protection rules or rulesets, and that your repository's review requirements (e.g., "require 1 approving review from code owners") are satisfied by humans only.

To make this explicit in documentation your team can refer to, add it to your CODEOWNERS file as a comment:

# Copilot code review is enabled for all paths and runs automatically.
# Copilot review is ADVISORY ONLY - it does not count toward required reviewer approval.
# Do not dismiss Copilot comments without reading them, but do not feel obligated
# to address every comment before merging. Use judgment.

*       @your-org/eng-team

When Blocking Does Make Sense

There is a narrow category of Copilot comment that warrants a different policy: security findings. If Copilot flags a potential credential exposure, SQL injection pattern, or authentication bypass, that comment deserves mandatory review before merge - but the mechanism should be a human, not an automated block.

A pragmatic approach: configure a GitHub Actions workflow that is triggered when Copilot posts a review containing specific keywords. The workflow notifies a security channel (or opens an issue in a security repo) but does not block the PR. It creates a paper trail and a human gets eyes on it; the PR continues.

name: Copilot Security Alert Triage

on:
  pull_request_review:
    types: [submitted]

jobs:
  triage:
    if: >
      github.event.review.user.login == 'github-copilot[bot]' &&
      github.event.review.state == 'changes_requested'
    runs-on: ubuntu-latest
    permissions:
      issues: write
      pull-requests: read
    steps:
      - name: Check for security-related comments
        env:
          REVIEW_BODY: ${{ github.event.review.body }}
        run: |
          KEYWORDS="credential|injection|hardcoded|secret|auth bypass|XSS|CSRF"
          if echo "$REVIEW_BODY" | grep -qiE "$KEYWORDS"; then
            echo "security_flag=true" >> $GITHUB_OUTPUT
          fi
        id: check

      - name: Create security triage issue
        if: steps.check.outputs.security_flag == 'true'
        uses: actions/github-script@v7
        with:
          script: |
            await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: `[Security Triage] Copilot flagged PR #${context.payload.pull_request.number}`,
              body: `Copilot posted a REQUEST_CHANGES review on PR #${context.payload.pull_request.number} containing security-related keywords.\n\nPR: ${context.payload.pull_request.html_url}\nCopilot review: ${context.payload.review.html_url}\n\nA human should review this before or shortly after merge.`,
              labels: ['security', 'triage']
            })

This gets you the audit trail and the human escalation path without making Copilot a blocking gate. The PR author is not punished for a potential false positive; the security team is notified regardless.

Integrating with CODEOWNERS

CODEOWNERS and Copilot review are independent systems, but they interact in a way that's worth understanding. When a PR modifies files owned by @your-org/backend-team, those team members are auto-requested as reviewers - that is the CODEOWNERS mechanism. Copilot review runs in addition to that, not as a replacement for it.

The useful integration point is paths that have no CODEOWNERS entry. Files without an owner get no automatic human reviewer. Those are exactly the paths where Copilot review provides the most value - it is the only reviewer that will automatically comment on them.

Audit your CODEOWNERS file for gaps and make them explicit:

# Fully owned - humans review, Copilot supplements
src/api/**             @your-org/backend-team
src/web/**             @your-org/frontend-team

# Infrastructure - requires explicit DevOps review
.github/workflows/**   @your-org/devops
terraform/**           @your-org/devops

# No human owners - Copilot review is the primary automated check
# These should be promoted to owned paths when a team takes ownership
scripts/**
tools/**
docs/**

The comment is explicit that Copilot is providing the primary automated check for unowned paths. It is not invisible; it's a documented policy that the team can read and change. When someone takes ownership of scripts/**, the CODEOWNERS entry gets added and the comment gets updated - and the change goes through a PR.

Working with the Output at Scale

At small team sizes, Copilot review comments are visible in the normal PR review flow. At larger scales - dozens of active PRs across multiple repositories - a few patterns help keep the signal-to-noise ratio manageable.

Resolving vs. Dismissing

GitHub distinguishes between resolving a review comment (you addressed it) and dismissing a review (you are overriding it). For Copilot comments specifically:

Resolve a comment when you fixed the issue Copilot flagged, even if you would have caught it anyway. This keeps the thread clean and signals to teammates that the concern was real.
Dismiss or leave unresolved a comment when it's a false positive or a pattern that's intentional in your codebase. Leave a short reply explaining why - not for Copilot's benefit, but for the next human who reads the PR history and wonders why an apparently valid concern was not addressed.

The discipline of writing one sentence explaining a dismissal ("this is intentional - the retry logic is owned by the caller, not this function") pays for itself quickly in onboarding new developers who read PR histories to understand codebase conventions.

Tracking False Positive Patterns

When Copilot repeatedly flags a pattern that is intentional in your codebase, that is signal to update your custom instructions. A lightweight workflow: create a copilot-review-false-positives label in your repository. When a developer dismisses a Copilot comment as a false positive, they apply the label to the PR. At a monthly retro cadence, review the labeled PRs, identify recurring patterns, and add instructions to .github/copilot-review-instructions.md to suppress them.

This turns false positives from noise into signal about gaps in your instruction file - and the improvement compounds over time.

The Merge Pattern

The practical merge workflow for a team running Copilot review looks like this:

Developer opens a PR. Copilot review runs automatically within a few minutes.
Developer reads Copilot comments before requesting human review. If something is obviously wrong, they fix it and push - no need to involve a human reviewer for issues Copilot caught in seconds.
Human reviewer is requested. They see a PR where the easy catches are already addressed; their review time goes toward design, context, and judgment.
PR merges with or without residual Copilot comments, per the author's judgment. The security triage workflow runs if any comments triggered the keyword check.

The optimization in step 2 is the real productivity gain: Copilot is not replacing human review, it is preprocessing the PR so that human review is higher-signal. A reviewer who arrives at a PR where null checks and hardcoded values are already handled can spend their limited attention on the decisions that actually require a human.

What Copilot Review Doesn't Replace

To use this tool well, it helps to be clear-eyed about where it stops.

Copilot review does not catch architectural problems. A pattern that is locally valid but globally wrong - a service that correctly handles its own logic but violates a system-wide invariant - requires context that a diff reviewer cannot have. Architecture decisions belong in ADRs, reviewed before code is written, not discovered by automated review of the implementation.

Copilot review does not catch correctness for domain logic. Whether a billing calculation is correct, whether a state machine covers the right transitions, whether a data transformation produces the right output for edge cases in your specific domain - these require a reviewer who understands the domain. Copilot does not.

Copilot review does not replace pair programming or design review for complex changes. A PR that involves a significant new feature, a refactor, or a change to a shared contract benefits from synchronous discussion before the code is written, not asynchronous comments after.

The framing that works: Copilot review is good at catching the category of issues that a careful developer would catch by reading their own diff slowly. Most developers, under time pressure, do not read their own diffs that carefully before requesting review. Copilot does it for free, every time. That is a genuine and bounded value proposition - and it leaves the harder problems exactly where they should be.

OpenID Connect in GitHub Actions: Replacing Long-Lived Secrets with Short-Lived Tokens

2026-06-12T00:00:00Z

Most GitHub Actions workflows that deploy to a cloud provider contain something like this:

env:
  AZURE_CREDENTIALS: ${{ secrets.AZURE_CREDENTIALS }}

or this:

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

These credentials are real, long-lived, and stored. They exist in GitHub's secret store, they are injected into the environment of every workflow run, and they work until someone rotates or revokes them - which, in practice, often means they work indefinitely. If a dependency in your build process is compromised, or if a workflow is vulnerable to command injection via an untrusted PR title or issue body, an attacker can exfiltrate these credentials and use them outside GitHub entirely.

OIDC eliminates this attack surface entirely. With OIDC, your workflow authenticates to the cloud provider by presenting a short-lived token that GitHub mints for that specific workflow run. The cloud provider is configured to trust tokens from GitHub. No static credential is ever stored - not in GitHub secrets, not in environment variables, not anywhere. The token that grants cloud access expires in minutes and cannot be reused outside the context of that workflow run.

This post covers how it works, how to configure it for Azure and AWS, and how to audit your existing workflows to find the credentials you can replace today.

How OIDC Works in This Context

The mechanism is an OAuth 2.0 token exchange. GitHub acts as an OpenID Connect identity provider, issuing signed JWTs that describe the current workflow execution. The cloud provider - Azure, AWS, or GCP - is configured as a relying party that trusts those JWTs and will exchange them for its own short-lived access credentials.

The flow for a single workflow run:

The workflow runner requests a JWT from GitHub's OIDC endpoint (https://token.actions.githubusercontent.com). This requires the id-token: write permission in the workflow's permissions block.
GitHub signs the JWT with its private key and includes claims that identify the workflow: the repository name, the ref (branch or tag), the triggering event, and - critically - the environment name if the job is associated with a GitHub Environment.
The workflow presents this JWT to the cloud provider's STS (Security Token Service) endpoint.
The cloud provider verifies the JWT signature against GitHub's published public keys, checks that the claims match the configured trust conditions, and issues a short-lived access token.
The workflow uses the short-lived access token for the duration of the job. It expires - typically within an hour - and cannot be refreshed without a new workflow run.

The critical property: at no point does a human-created, stored credential enter the picture. The JWT is ephemeral. The access token is ephemeral. An attacker who exfiltrates either has minutes, not months, to use it - and even then, only within the scope of what that specific workflow was authorized to do.

The JWT issued by GitHub contains a sub (subject) claim that identifies the execution context precisely. For a job running in the production environment, the sub looks like: repo:owner/repo-name:environment:production. For a branch push, it looks like: repo:owner/repo-name:ref:refs/heads/main. The cloud provider's trust policy checks this claim, which means you can scope trust to a specific environment, branch, or ref - not just to the repository as a whole.

Setting Up OIDC with Azure

Azure's implementation of OIDC federation is called Workload Identity Federation. You configure it on an Azure App Registration, which gives you a service principal you can grant RBAC roles.

Step 1: Create the App Registration

In the Azure portal, navigate to Azure Active Directory -> App registrations -> New registration. Give it a descriptive name that includes the repo and environment, e.g. github-actions-myorg-myrepo-production. You don't need a redirect URI.

After creation, note the Application (client) ID and Directory (tenant) ID - you'll need both in the workflow.

Step 2: Add a Federated Identity Credential

On the App Registration, go to Certificates & secrets -> Federated credentials -> Add credential. Select GitHub Actions deploying Azure resources as the scenario.

Fill in:

Organization: your GitHub org or username
Repository: the repo name (without the org prefix)
Entity type: select Environment and enter production, or Branch and enter main, depending on what you want to trust

The entity type maps directly to what GitHub puts in the JWT's sub claim. Using Environment is strongly preferred over Branch - it means the trust is scoped to jobs that explicitly run in a named GitHub Environment, which you can protect with required reviewer approval and deployment protection rules.

You can add multiple federated credentials to the same App Registration - one for production, one for staging, each with appropriate trust scope.

Step 3: Assign RBAC Roles

The App Registration is now an identity that GitHub Actions can assume. Grant it the minimum RBAC roles it needs on the specific Azure resources the workflow accesses. For a typical deployment workflow:

Contributor on the target resource group (for deploying resources)
AcrPush on the container registry (if pushing images)

Do not grant Owner or Contributor at the subscription level unless the workflow genuinely needs it.

Navigate to the resource group -> Access control (IAM) -> Add role assignment, select the role, and search for your App Registration by name.

Step 4: Update the Workflow

The workflow needs three changes: a permissions block granting id-token: write, the three Azure identifiers as non-secret environment variables (they are not sensitive - they identify the identity, not a credential), and azure/login@v2 using the federated credential flow:

name: Deploy to Azure

on:
  push:
    branches: [main]

permissions:
  contents: read
  id-token: write   # Required for OIDC token request

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production   # Must match the federated credential entity
    steps:
      - uses: actions/checkout@v4

      - name: Log in to Azure
        uses: azure/login@v2
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}        # Repository variable, not secret
          tenant-id: ${{ vars.AZURE_TENANT_ID }}        # Repository variable, not secret
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}  # Repository variable, not secret

      - name: Deploy
        run: az webapp deploy --resource-group my-rg --name my-app --src-path ./dist

Three things worth noting. First, vars.AZURE_CLIENT_ID uses repository variables (not secrets) - these IDs are not credentials and don't need secret-level protection. Storing non-sensitive config in variables rather than secrets is the correct pattern. Second, environment: production on the job is what generates the environment:production subject claim in the JWT - it must match the entity you configured in the federated credential. If this is omitted or mismatched, Azure rejects the token. Third, azure/login@v2 handles the entire OIDC exchange internally - you don't write any token exchange code.

Setting Up OIDC with AWS

AWS's implementation uses IAM Roles and an OIDC Identity Provider. The setup is slightly more involved than Azure because you configure the trust relationship in an IAM role's trust policy.

Step 1: Create the OIDC Identity Provider

In the AWS IAM console, navigate to Identity providers -> Add provider. Select OpenID Connect and enter:

Provider URL: https://token.actions.githubusercontent.com
Audience: sts.amazonaws.com

Click Get thumbprint to fetch GitHub's certificate thumbprint, then create the provider.

This step is done once per AWS account, not once per repo. All GitHub Actions workflows in all repos can use the same OIDC provider.

Step 2: Create an IAM Role with a Trust Policy

Create an IAM role. The trust policy is what controls which GitHub workflows can assume this role. A trust policy scoped to a specific repo and environment:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:myorg/myrepo:environment:production"
        }
      }
    }
  ]
}

Replace 123456789012 with your AWS account ID. The sub condition is the critical constraint - it limits assumption of this role to jobs running in the production environment of myorg/myrepo. A workflow running in staging, or a workflow in a different repo, will get a JWT with a different sub claim and will fail this condition.

If you need to trust a branch instead of an environment, change the condition to:

"token.actions.githubusercontent.com:sub": "repo:myorg/myrepo:ref:refs/heads/main"

Attach permission policies to the role granting only what the workflow needs - s3:PutObject for a deployment that uploads to S3, ecr:GetAuthorizationToken and ecr:BatchCheckLayerAvailability for pushing container images. Never attach AdministratorAccess.

Note the role's ARN - you'll use it in the workflow.

Step 3: Update the Workflow

name: Deploy to AWS

on:
  push:
    branches: [main]

permissions:
  contents: read
  id-token: write   # Required for OIDC token request

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/github-actions-myrepo-production
          aws-region: us-east-1

      - name: Deploy
        run: aws s3 sync ./dist s3://my-bucket --delete

aws-actions/configure-aws-credentials@v4 handles the OIDC exchange and populates the standard AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables with short-lived values. Subsequent steps use the standard AWS SDK and CLI without any special handling - they pick up the credentials from the environment automatically.

The role ARN in role-to-assume is not sensitive - it's a resource identifier, not a credential. Store it as a repository variable (vars.AWS_ROLE_ARN) rather than a hardcoded string, but it doesn't need secret-level protection.

Scoping Trust Correctly

The trust conditions you configure determine the blast radius if GitHub's OIDC infrastructure were ever compromised or if an attacker found a way to trigger your workflow from a fork. Getting this right is not optional.

Use Environments, Not Branches

Prefer environment-scoped subjects (environment:production) over branch-scoped subjects (ref:refs/heads/main) for any role or identity that has write access to production resources.

The reason: you can protect a GitHub Environment with required reviewers and deployment protection rules. A job running in a protected environment cannot proceed until a designated human approves it. Branch-scoped trust has no equivalent human-in-the-loop mechanism - if the workflow is triggered on main, it runs.

Avoid Wildcard Subjects

This trust policy is too broad:

"token.actions.githubusercontent.com:sub": "repo:myorg/myrepo:*"

A wildcard subject means any job in any context in that repo can assume the role - PRs, scheduled runs, workflow_dispatch from any user with write access. If your deployment role should only be assumed by jobs in the production environment, restrict it to exactly that. The specificity costs nothing.

One Role Per Deployment Target

A role that deploys to production and staging from the same trust policy is a staging-to-production privilege escalation waiting to happen. Create separate roles with separate trust conditions for each environment. The IAM overhead is minimal; the blast radius reduction is significant.

Auditing Existing Workflows

If your repository already has workflows using static credentials, here is a practical approach to finding and replacing them.

Finding Static Cloud Credentials

The patterns to search for vary by provider:

# AWS static credentials
grep -r "AWS_ACCESS_KEY_ID\|AWS_SECRET_ACCESS_KEY" .github/workflows/

# Azure service principal JSON credential
grep -r "AZURE_CREDENTIALS\|clientSecret\|azure-credentials" .github/workflows/

# GCP service account key JSON
grep -r "GCP_SA_KEY\|GOOGLE_CREDENTIALS\|service_account_key" .github/workflows/

# Generic patterns
grep -r "secrets\." .github/workflows/ | grep -i "key\|secret\|credential\|token\|password"

For each match, determine:

Which cloud provider and what scope the credential has
Whether the workflow runs on PRs (higher risk - PR workflows can be triggered by external contributors)
When the credential was last rotated (check the cloud provider's IAM console for last-used date)

Migration Priority

Prioritize the migrations in this order:

Workflows triggered by pull_request from forks - these are the highest risk because external contributors can trigger them. A forked PR workflow that exfiltrates a static credential is a realistic attack scenario.
Credentials with broad scope - subscription-level Contributor in Azure, AdministratorAccess in AWS, Owner at the project level in GCP. These have the largest blast radius.
Credentials that have never been rotated - static credentials that have been alive since a project's initial setup are the most likely to have been quietly exfiltrated without anyone knowing.

After migrating each workflow, revoke the static credential immediately. Leaving it active "just in case" defeats the purpose and leaves a dormant attack surface.

The Rotation Trap

A common counterargument to the OIDC migration effort: "we rotate our credentials regularly." Rotation reduces risk but does not eliminate it. A rotated credential is still a credential that exists, that can be exfiltrated, and that provides valid access until the next rotation cycle. The question OIDC asks is not "how fresh is the credential" but "why does a credential need to exist at all?" For workflows that authenticate to cloud providers, the answer is almost always: it doesn't.

What OIDC Doesn't Cover

OIDC is the right solution for cloud provider authentication from GitHub Actions. It is not a general-purpose secret replacement.

Third-party service credentials - API keys for Datadog, Snyk, Slack, npm, and similar services do not support OIDC federation. These still belong in GitHub secrets, rotated regularly, with minimum scope. OIDC reduces your secret surface area; it doesn't eliminate it.

Intra-workflow secrets - values generated during a workflow run (e.g., a database password for a test environment spun up during CI) should be generated ephemerally and passed between jobs using outputs, not stored in secrets.

Self-hosted runner credentials - if your self-hosted runners need to authenticate to resources at startup (not at workflow execution time), OIDC does not apply. Those credentials are infrastructure-level and belong in a secrets manager (Azure Key Vault, AWS Secrets Manager), not GitHub.

The practical target: after a full OIDC migration, your GitHub secrets should contain only third-party API credentials, and your cloud provider access should require no secrets at all. Most repositories can reach that state within a sprint.

OIDC Migration Checklist

Run this against each repository before and during the migration:

[ ] Search for static cloud credentials: grep -r "AWS_ACCESS_KEY_ID\|AZURE_CREDENTIALS\|GCP_SA_KEY\|secrets\..*[Kk]ey\|secrets\..*[Cc]redential" .github/workflows/
[ ] For each credential found: identify the cloud provider, the scope (subscription vs. resource group vs. specific service), and whether the workflow triggers on pull_request from forks
[ ] Prioritize: fork-triggerable workflows first, then broad-scope credentials (subscription Contributor, AdministratorAccess), then credentials that have never been rotated
[ ] Azure: create an App Registration with a Workload Identity federated credential; scope the entity to a GitHub Environment, not a branch
[ ] AWS: register the GitHub OIDC Identity Provider once per account (https://token.actions.githubusercontent.com, audience sts.amazonaws.com), then create an IAM role with a trust policy using StringEquals on the sub claim
[ ] For each role or App Registration: attach only the minimum permissions the workflow actually needs - no AdministratorAccess, no subscription-level Contributor unless absolutely required
[ ] Update each workflow: add permissions: id-token: write, add environment: on the job (matching the federated credential entity), replace the secret-based login step with the OIDC-enabled login action (azure/login@v2 or aws-actions/configure-aws-credentials@v4)
[ ] Move non-sensitive identifiers (client IDs, tenant IDs, role ARNs, subscription IDs) from secrets to repository variables (vars.*)
[ ] Run the migrated workflow end-to-end and confirm successful cloud authentication before revoking the static credential
[ ] Revoke the static credential immediately after successful validation - do not leave it active as a fallback
[ ] After completing all migrations: audit GitHub secrets and confirm no cloud provider credentials remain; only third-party API keys should be left

OIDC is not a complicated migration. The implementation steps are mechanical and the trust model is sound. The credential that cannot be exfiltrated cannot be misused - and that is a meaningfully different security posture than the one most workflows start with.

Questions about OIDC setup, IAM trust policy design, or auditing workflows across an org? Reach out.

steve.kaschimer@slalom.com

Azure AI Foundry: A Developer's First Look at Agentic AI Workflows

2026-06-19T00:00:00Z

When people describe Azure AI Foundry, they usually start with platform language. If you're a developer trying to build your first agent, that framing is not very useful.

The practical way to think about Foundry is this:

Hub is your governance and shared resource boundary.
Project is your app workspace.
Deployment is the model endpoint your agent will call.
Connection is how your project reaches external systems (search, storage, keys, APIs).

In this post, we'll stand up a minimal project from scratch, deploy a model, create a basic agent with one tool, run it, and then map the model to OpenAI Assistants and LangChain so you can transfer what you already know.

What You're Building in 30 Minutes

By the end, you'll have:

An Azure AI Foundry project.
A deployed chat model (for example, gpt-4o-mini where available).
A simple agent configured with one callable tool.
A test run where the agent decides to invoke that tool and returns an answer.

This is intentionally the smallest useful workflow.

Prerequisites

You'll need:

An Azure subscription with permission to create AI Foundry resources.
Access to Azure AI Foundry in your tenant.
A model quota/approval in your target region.

If model deployment fails, it's usually one of two things: wrong region or missing quota for that model family.

Step 1: Create a Hub and Project

In the Foundry UI:

Create a Hub (or use an existing one if your team already has governance in place).
Inside the hub, create a Project for this app.
Keep naming explicit (for example hub-devsecops-lab and proj-foundry-first-agent), because these names surface in audit and billing views later.

Think of this as the same separation you'd use for cloud infrastructure:

Hub = shared platform scope.
Project = application scope.

Step 2: Deploy a Model

Inside the project:

Open Models.
Choose a chat-capable model available in your region.
Create a deployment with a clear name, e.g. gpt-4o-mini-chat.

Your deployment name is what the agent targets at runtime. Don't hardcode model marketing names in app logic; use deployment names so you can swap model versions later without changing business code.

Step 3: Add a Connection

Open Connections in the project and add at least one connection you'll use with tools (for example, Azure AI Search, Blob Storage, or Key Vault).

For this first pass, even a placeholder connection is useful because it makes the project structure real: agents don't run in a vacuum, they run with context and systems access.

Step 4: Create a Basic Agent with One Tool

Create an agent in the project and bind it to your model deployment.

Add one simple function tool so the agent can do something beyond plain prompting. A common first tool is a deterministic lookup (for example: get_build_status(service_name)), because it's easy to verify when the model chose to call the tool.

Pseudo-shape of a tool contract:

{
  "name": "get_build_status",
  "description": "Returns CI build status for a service.",
  "parameters": {
    "type": "object",
    "properties": {
      "service_name": { "type": "string" }
    },
    "required": ["service_name"]
  }
}

Then implement the handler in your app:

async function getBuildStatus(serviceName: string): Promise<string> {
  const fakeStatus: Record<string, string> = {
    api: "passing",
    web: "failing",
  };
  return fakeStatus[serviceName] ?? "unknown";
}

Even with a mock tool, you can validate the full loop:

User asks a question.
Agent chooses tool call.
App executes tool.
Tool result goes back to agent.
Agent produces final answer.

Step 5: Run It End-to-End

Use the Foundry test interface (or your SDK client) and send:

"What's the current build status for the web service?"

Expected behavior:

Agent calls get_build_status with service_name = "web".
Tool returns failing.
Agent responds in natural language using that tool output.

If the agent never calls the tool, tighten your instructions and tool description. Most first-run failures are prompt clarity failures, not platform failures.

The Four Concepts That Matter (Without the Marketing Layer)

Hub

Governance container for teams, policies, and shared controls.

Project

Implementation workspace for one app or bounded initiative.

Deployment

A concrete, versioned model endpoint you can target and swap safely.

Connection

Managed wiring to external dependencies and data systems.

If you understand those four nouns, Foundry becomes predictable.

Foundry vs. OpenAI Assistants vs. LangChain

Here's the practical mapping:

If you know...	Rough equivalent in Foundry	Key difference
OpenAI Assistants	Agent + model deployment + tools	Foundry adds stronger Azure-native project/resource boundaries and enterprise integration points.
LangChain	Orchestrated agent flow with tool calls	LangChain is framework-first and code-driven; Foundry is platform-first with managed project/deployment/runtime surfaces.

Another way to choose:

Use Foundry when you want managed enterprise controls, Azure integration, and standardized deployment surfaces.
Use OpenAI Assistants when you want the fastest path in the OpenAI ecosystem with minimal platform overhead.
Use LangChain when you need maximum orchestration flexibility and are comfortable owning more runtime behavior in code.

They're not mutually exclusive. Many teams prototype orchestration patterns in LangChain, then operationalize on managed platform primitives.

Common First-Week Mistakes

Confusing model name with deployment name
Code should target deployment names.
Skipping connections until later
Agents without real data/tools look good in demos and fail in production.
Giving tools vague descriptions
Tool selection quality is highly sensitive to clear parameter definitions.
Treating the first prompt as final architecture
Start simple, inspect traces, tighten tool contracts, then expand.

Where to Go Next

Once this baseline works, the next useful step is multi-agent coordination: one agent for retrieval, one for reasoning, one for action execution, with explicit handoff rules and traceability.

But don't jump there on day one. A single agent with one tool and clean deployment boundaries teaches almost everything you need to avoid the early architecture traps.

If you've already used Assistants or LangChain, Foundry isn't a new mental model - it's a different operating model for the same core pattern: model + tools + context + orchestration.

GitHub Projects Automation: Custom Fields, Workflows, and the GraphQL API

2026-06-26T00:00:00Z

Most teams use GitHub Projects the way they use a whiteboard: they move cards around manually, they check in on Fridays to see what slipped, and they treat the board as a reporting artifact rather than a live system. That's a reasonable starting point. It's not where you should stay.

GitHub Projects v2 ships with a GraphQL API, a set of built-in automation workflows, and custom field types that compose into a lightweight engineering system. You can auto-assign new issues to the current sprint, reflect PR state back to issue status without human intervention, and schedule a weekly digest from a GitHub Actions workflow. None of this requires a third-party tool. None of it requires leaving GitHub.

The thesis: if your team is already on GitHub, you have most of what Jira offers. You're just not using it.

Custom Field Types

The three field types worth understanding are Iteration, Single Select, and Number. Each has a specific purpose and a specific shape in the API.

Iteration

An iteration field models a sprint. You define a cadence - say, two-week periods starting on Mondays - and GitHub generates named iterations automatically. Each iteration has an id, a title (e.g., "Iteration 3"), a startDate, and a duration in days.

To add an iteration field to your project in the UI: open the project, click + to add a field, choose Iteration, and configure the cadence. You'll configure the start date and duration once; subsequent iterations are created automatically.

The iteration field is the key to sprint assignment automation. When you have its field ID and the current iteration's option ID, you can set it programmatically on any project item - including items added in the last hour.

Single Select

A single-select field is a bounded set of named options with associated colors. This is where you put Status (Todo, In Progress, In Review, Done) and Priority (P0, P1, P2). You define the options in the field configuration; the API refers to each option by its id.

Single-select is what built-in workflows operate on. When you configure a workflow to "set status to In Progress when a PR is opened," the workflow is writing to a single-select field.

Number

A number field stores a decimal value. Use it for story points, effort estimates, or cycle time. There's no special behavior from the API perspective - you set it the same way you set any other field - but it unlocks grouping by numeric range and aggregate views in the project's insight charts.

Adding Fields via the UI

You don't need the API to create fields - use the project UI for one-time configuration, and the API for automation:

Open your project.
Click the + icon at the right of the column headers to add a field.
Choose the field type and configure it (options for single-select, cadence for iteration).
Note the field names exactly - you'll need them when querying the API.

Built-In Workflows

GitHub Projects ships four built-in workflow triggers you should turn on for any active project:

Auto-add to project - when an issue or PR is opened in a linked repository, it's automatically added to the project. Requires no code. Enable it under Workflows in the project settings.
Auto-archive closed items - when an issue or PR is closed and a configurable time passes (default: 7 days), the item is archived. Keeps the board from accumulating closed noise.
Set status when PR is merged - when a linked PR is merged, the associated project item's status field is set to a value you configure (typically Done). Requires a single-select field named Status.
Set status when PR is opened - sets a status value when a PR linked to an issue is opened. Use this to move an item to In Progress automatically when someone begins the work.

To configure: open your project, click ... → Workflows, and enable the relevant triggers. Each workflow lets you choose which field to set and which option to use.

These four cover the majority of status transitions in a normal sprint cycle. The only one you need the API for is sprint assignment, because the current iteration's option ID changes every sprint.

The GraphQL API

GitHub's Projects v2 API lives at https://api.github.com/graphql and requires a personal access token or a fine-grained token with project scope. For GitHub Actions, use a token stored in secrets.

Getting Your Project ID

Everything in the API starts with the project node ID. Fetch it by querying the organization or user:

query {
  organization(login: "your-org") {
    projectV2(number: 1) {
      id
      title
    }
  }
}

Or for a user-owned project:

query {
  user(login: "your-username") {
    projectV2(number: 1) {
      id
      title
    }
  }
}

The id returned is a base64-encoded node ID like PVT_kwDOA.... Store this as a repository variable - it doesn't change.

Fetching Field IDs and Option IDs

Field operations require the field's node ID and, for single-select and iteration fields, the option's node ID. Fetch them with:

query {
  node(id: "PVT_kwDOA...") {
    ... on ProjectV2 {
      fields(first: 20) {
        nodes {
          ... on ProjectV2Field {
            id
            name
          }
          ... on ProjectV2SingleSelectField {
            id
            name
            options {
              id
              name
            }
          }
          ... on ProjectV2IterationField {
            id
            name
            configuration {
              iterations {
                id
                title
                startDate
                duration
              }
            }
          }
        }
      }
    }
  }
}

Run this once and record the IDs for each field and option you'll automate. The field IDs are stable. Iteration option IDs change each sprint - that's the piece you'll query dynamically.

Updating a Field Value

The mutation to update any project item field is updateProjectV2ItemFieldValue. The shape varies slightly by field type.

Single-select field:

mutation {
  updateProjectV2ItemFieldValue(
    input: {
      projectId: "PVT_kwDOA..."
      itemId: "PVTI_lADOA..."
      fieldId: "PVTSSF_lADOA..."
      value: {
        singleSelectOptionId: "abc123"
      }
    }
  ) {
    projectV2Item {
      id
    }
  }
}

Iteration field:

mutation {
  updateProjectV2ItemFieldValue(
    input: {
      projectId: "PVT_kwDOA..."
      itemId: "PVTI_lADOA..."
      fieldId: "PVTIF_lADOA..."
      value: {
        iterationId: "iteration_id_here"
      }
    }
  ) {
    projectV2Item {
      id
    }
  }
}

Number field:

mutation {
  updateProjectV2ItemFieldValue(
    input: {
      projectId: "PVT_kwDOA..."
      itemId: "PVTI_lADOA..."
      fieldId: "PVTF_lADOA..."
      value: {
        number: 3
      }
    }
  ) {
    projectV2Item {
      id
    }
  }
}

The itemId is the project item's node ID (not the issue number). You get it by querying the project's items or by reading it from the projects_v2_item event payload in a GitHub Actions trigger.

Practical GitHub Actions Workflows

Here are three workflows that together cover sprint assignment, status automation, and weekly reporting.

1. Auto-Assign to Current Iteration

This workflow fires whenever a new issue is added to the project. It queries the current iteration from the iteration field, then sets that iteration on the newly added item.

name: Assign to Current Sprint

on:
  projects_v2_item:
    types: [created]

jobs:
  assign-sprint:
    runs-on: ubuntu-latest
    steps:
      - name: Assign item to current iteration
        env:
          GH_TOKEN: ${{ secrets.PROJECT_TOKEN }}
          PROJECT_ID: ${{ vars.PROJECT_ID }}
          ITERATION_FIELD_ID: ${{ vars.ITERATION_FIELD_ID }}
        run: |
          ITEM_ID="${{ github.event.projects_v2_item.node_id }}"

          # Get the current (first active) iteration ID
          ITERATION_ID=$(gh api graphql -f query='
            query($projectId: ID!) {
              node(id: $projectId) {
                ... on ProjectV2 {
                  fields(first: 20) {
                    nodes {
                      ... on ProjectV2IterationField {
                        id
                        configuration {
                          iterations {
                            id
                            startDate
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          ' -f projectId="$PROJECT_ID" \
            --jq '[.data.node.fields.nodes[] | select(.configuration != null) | .configuration.iterations[0].id] | first')

          # Set the iteration field on the item
          gh api graphql -f query='
            mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $iterationId: String!) {
              updateProjectV2ItemFieldValue(input: {
                projectId: $projectId
                itemId: $itemId
                fieldId: $fieldId
                value: { iterationId: $iterationId }
              }) {
                projectV2Item { id }
              }
            }
          ' -f projectId="$PROJECT_ID" \
            -f itemId="$ITEM_ID" \
            -f fieldId="$ITERATION_FIELD_ID" \
            -f iterationId="$ITERATION_ID"

The projects_v2_item event requires the PROJECT_TOKEN to have project scope. A fine-grained personal access token with read and write access to the specific project works here.

2. Set "In Progress" When a Linked PR Is Opened

Built-in workflows handle this for the common case, but if you need custom logic - for example, only triggering on PRs from non-draft state, or updating a secondary field alongside status - a workflow gives you full control:

name: Set In Progress on PR Open

on:
  pull_request:
    types: [opened, ready_for_review]

jobs:
  update-project-status:
    runs-on: ubuntu-latest
    steps:
      - name: Find linked issue and update project status
        env:
          GH_TOKEN: ${{ secrets.PROJECT_TOKEN }}
          PROJECT_ID: ${{ vars.PROJECT_ID }}
          STATUS_FIELD_ID: ${{ vars.STATUS_FIELD_ID }}
          IN_PROGRESS_OPTION_ID: ${{ vars.IN_PROGRESS_OPTION_ID }}
        run: |
          PR_NUMBER="${{ github.event.pull_request.number }}"
          REPO="${{ github.repository }}"

          # Find the project item linked to this PR
          ITEM_ID=$(gh api graphql -f query='
            query($repo: String!, $owner: String!, $pr: Int!) {
              repository(owner: $owner, name: $repo) {
                pullRequest(number: $pr) {
                  projectItems(first: 5) {
                    nodes {
                      id
                      project {
                        id
                      }
                    }
                  }
                }
              }
            }
          ' -f owner="${REPO%%/*}" \
            -f repo="${REPO##*/}" \
            -F pr="$PR_NUMBER" \
            --jq ".data.repository.pullRequest.projectItems.nodes[] | select(.project.id == \"$PROJECT_ID\") | .id")

          if [ -z "$ITEM_ID" ]; then
            echo "No project item found for PR #$PR_NUMBER in project $PROJECT_ID"
            exit 0
          fi

          # Set status to In Progress
          gh api graphql -f query='
            mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
              updateProjectV2ItemFieldValue(input: {
                projectId: $projectId
                itemId: $itemId
                fieldId: $fieldId
                value: { singleSelectOptionId: $optionId }
              }) {
                projectV2Item { id }
              }
            }
          ' -f projectId="$PROJECT_ID" \
            -f itemId="$ITEM_ID" \
            -f fieldId="$STATUS_FIELD_ID" \
            -f optionId="$IN_PROGRESS_OPTION_ID"

Store STATUS_FIELD_ID and IN_PROGRESS_OPTION_ID as repository variables - you get them from the field query above.

3. Weekly Digest via Scheduled Workflow

This workflow runs every Monday morning, queries the project for items by status, and posts a summary to a Slack webhook or writes it to a GitHub issue. The example below writes the digest as a new issue in the repository.

name: Weekly Project Digest

on:
  schedule:
    - cron: '0 8 * * 1'   # Every Monday at 08:00 UTC
  workflow_dispatch:

jobs:
  digest:
    runs-on: ubuntu-latest
    permissions:
      issues: write
    steps:
      - name: Generate weekly digest
        env:
          GH_TOKEN: ${{ secrets.PROJECT_TOKEN }}
          PROJECT_ID: ${{ vars.PROJECT_ID }}
        run: |
          TODAY=$(date +%Y-%m-%d)

          # Query all project items with their status and assignees
          DIGEST=$(gh api graphql -f query='
            query($projectId: ID!) {
              node(id: $projectId) {
                ... on ProjectV2 {
                  items(first: 100) {
                    nodes {
                      id
                      content {
                        ... on Issue {
                          title
                          number
                          url
                          assignees(first: 3) {
                            nodes { login }
                          }
                        }
                        ... on PullRequest {
                          title
                          number
                          url
                          assignees(first: 3) {
                            nodes { login }
                          }
                        }
                      }
                      fieldValues(first: 10) {
                        nodes {
                          ... on ProjectV2ItemFieldSingleSelectValue {
                            name
                            field {
                              ... on ProjectV2SingleSelectField {
                                name
                              }
                            }
                          }
                          ... on ProjectV2ItemFieldIterationValue {
                            title
                            field {
                              ... on ProjectV2IterationField {
                                name
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          ' -f projectId="$PROJECT_ID")

          # Build the digest body using jq
          BODY=$(echo "$DIGEST" | jq -r '
            "## Weekly Project Digest - " + (now | strftime("%Y-%m-%d")) + "\n\n" +
            (
              .data.node.items.nodes
              | map(
                  . as $item |
                  {
                    title: (.content.title // "(no title)"),
                    number: (.content.number // ""),
                    url: (.content.url // ""),
                    status: (
                      .fieldValues.nodes[]
                      | select(.field.name == "Status")
                      | .name
                    ) // "No Status",
                    assignees: (
                      [.content.assignees.nodes[].login]
                      | if length > 0 then join(", ") else "unassigned" end
                    )
                  }
                )
              | group_by(.status)
              | map(
                  "### " + (.[0].status) + "\n\n" +
                  (
                    map("- [#\(.number) \(.title)](\(.url)) - \(.assignees)")
                    | join("\n")
                  )
                )
              | join("\n\n")
            )
          ')

          # Post the digest as a new issue
          gh issue create \
            --repo "${{ github.repository }}" \
            --title "Weekly Digest - $TODAY" \
            --label "digest" \
            --body "$BODY"

The jq query groups items by their Status field value and formats each group as a Markdown section. If you'd rather post to Slack, replace the gh issue create step with a curl call to your Slack webhook URL.

Repository Variables to Set Up

Before running any of these workflows, configure the following as repository variables (Settings → Secrets and variables → Variables):

Variable	How to get it
`PROJECT_ID`	Run the project query above; copy the `id` field
`ITERATION_FIELD_ID`	Run the fields query; copy the `id` of your iteration field
`STATUS_FIELD_ID`	Run the fields query; copy the `id` of your single-select Status field
`IN_PROGRESS_OPTION_ID`	From the Status field's `options` array; copy the `id` for `In Progress`

Store the PROJECT_TOKEN as a secret (it's a credential). Store everything else as a variable - none of the IDs are sensitive.

You Don't Need Jira

Every engineering team has an implicit project management tax: the time spent maintaining a tool that lives outside the actual work. Jira is powerful for the organizations that need its full feature set. For a team that lives in GitHub - where issues, PRs, code review, and CI/CD all coexist - the coordination overhead of a separate tool is real and rarely justified.

GitHub Projects v2, with custom fields and the GraphQL API, covers the things that matter for most teams: sprint tracking, status visibility, and automation that keeps the board accurate without human ceremony. You configure it once, you wire up two or three workflows, and then it runs.

The investment is an afternoon. The return is a project board that doesn't lie to you.

Questions, edge cases, or GraphQL shape oddities? Reach out.

steve.kaschimer@slalom.com

Semantic Kernel and Azure AI Foundry: Building Your First AI Agent in .NET

2026-07-03T00:00:00Z

Python has LangChain. .NET has Semantic Kernel. They solve the same problem - how do you wire a language model to tools, memory, and business logic in a way that's testable and deployable - but for a different runtime and a different operational context.

This post builds a working agent from scratch: a Kernel instance backed by a Foundry-hosted model, a plugin with a real tool function the model can call, and a multi-step interaction where the agent decides when to invoke the tool and uses the result in its answer. At the end, a GitHub Actions workflow deploys the whole thing to Azure Container Apps with no stored secrets.

If you've read the Foundry first-look post, you know the platform primitives. This post puts .NET code behind those primitives.

The Model: What Semantic Kernel Actually Does

Semantic Kernel (SK) is an orchestration SDK. It doesn't host models. It doesn't manage infrastructure. What it does is define a composable execution model for agents:

Kernel - the central object. Holds services (AI connectors, logging) and a plugin registry.
Plugin - a class with annotated methods the model can call as tools.
Chat completion service - the connector to the underlying model (Foundry, Azure OpenAI, OpenAI, etc.).
FunctionChoiceBehavior - tells SK whether to let the model decide which tools to call, require a specific one, or call none.

Azure AI Foundry hosts the model endpoint the kernel connects to. SK handles the rest: tool discovery, function call dispatch, result injection, conversation state.

Prerequisites

You need:

An Azure subscription with access to Azure AI Foundry.
A Foundry hub and project created. If you haven't done this yet, follow Steps 1-2 from the Foundry first-look post.
A model deployment inside the project - for example, gpt-4o-mini-chat.
Your deployment's endpoint URL and API key (or a managed identity - more on that below).
.NET 8 SDK.

Project Setup

Create a new console app and add the required packages:

dotnet new console -n DevOpsAgent
cd DevOpsAgent
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.AzureOpenAI
dotnet add package Azure.Identity

Microsoft.SemanticKernel is the core SDK. Microsoft.SemanticKernel.Connectors.AzureOpenAI provides the Azure OpenAI connector SK uses to talk to your Foundry-hosted model. Azure.Identity is for DefaultAzureCredential - the right way to authenticate in Azure-hosted deployments.

Wiring Up the Kernel

The kernel is built with a builder pattern. You add services - starting with the chat completion connector - and then call Build():

using Azure.Identity;
using Microsoft.SemanticKernel;

// Development: use API key directly
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
    deploymentName: "gpt-4o-mini-chat",
    endpoint: "https://<your-project>.openai.azure.com/",
    apiKey: Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY")!
);

var kernel = builder.Build();

For production - where you want managed identity instead of a stored API key - swap the credential parameter:

builder.AddAzureOpenAIChatCompletion(
    deploymentName: "gpt-4o-mini-chat",
    endpoint: "https://<your-project>.openai.azure.com/",
    credential: new DefaultAzureCredential()
);

The endpoint format is the Azure OpenAI endpoint for your Foundry project. In the Foundry portal, find it under Deployments → your deployment → Target URI, and strip the path - you want just https://<resource-name>.openai.azure.com/. The deployment name is the name you assigned when you created the model deployment, not the model's marketing name.

Plugins and Tool Functions

A plugin is a C# class. Methods you want the model to be able to call get a [KernelFunction] attribute. The [Description] attribute on the method and its parameters is what the model sees - treat those descriptions the same way you'd treat API documentation, because they directly affect whether the model calls the right tool with the right arguments.

using System.ComponentModel;
using Microsoft.SemanticKernel;

public class DevOpsPlugin
{
    [KernelFunction("get_pipeline_status")]
    [Description("Returns the latest CI pipeline status for a given repository.")]
    public string GetPipelineStatus(
        [Description("The repository name, for example 'api-service' or 'web-frontend'")]
        string repositoryName)
    {
        // In production: call your CI system's API here.
        // For this example, a deterministic stub makes the tool call easy to verify.
        var statuses = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
        {
            ["api-service"]    = "passing",
            ["web-frontend"]   = "failing",
            ["data-pipeline"]  = "pending",
        };

        return statuses.TryGetValue(repositoryName, out var status)
            ? $"Pipeline for '{repositoryName}' is currently {status}."
            : $"No pipeline found for repository '{repositoryName}'.";
    }

    [KernelFunction("list_repositories")]
    [Description("Returns a list of all tracked repositories.")]
    public IEnumerable<string> ListRepositories() =>
        ["api-service", "web-frontend", "data-pipeline"];
}

// Before Build():
builder.Plugins.AddFromType<DevOpsPlugin>("DevOps");
var kernel = builder.Build();

// Or after Build():
kernel.Plugins.AddFromType<DevOpsPlugin>("DevOps");

SK inspects the class at registration time, reads the [KernelFunction] and [Description] attributes, and builds a tool schema the model can reason over. The model never sees your C# method names directly - it sees the function name you passed to [KernelFunction] and the descriptions you wrote.

Running a Multi-Step Plan

With the kernel configured and the plugin registered, the agent loop looks like this:

using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.AzureOpenAI;

var chatService = kernel.GetRequiredService<IChatCompletionService>();
var history = new ChatHistory();

history.AddSystemMessage(
    "You are a DevOps assistant. Use the available tools to answer questions " +
    "about CI pipeline status. Always check current status before reporting.");

var executionSettings = new AzureOpenAIPromptExecutionSettings
{
    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
};

// Turn 1: ask about a specific repo
history.AddUserMessage(
    "What's the current pipeline status for api-service and web-frontend?");

var response = await chatService.GetChatMessageContentAsync(
    history,
    executionSettings,
    kernel);

Console.WriteLine(response.Content);
history.AddAssistantMessage(response.Content!);

// Turn 2: follow-up that requires the model to call list_repositories first
history.AddUserMessage("Which of our repos are currently failing?");

response = await chatService.GetChatMessageContentAsync(
    history,
    executionSettings,
    kernel);

Console.WriteLine(response.Content);

FunctionChoiceBehavior.Auto() is the key setting. It tells SK to let the model decide when to call tools. In practice: the model receives the conversation history and the available tool schemas; when it decides a tool call is warranted, SK intercepts the tool call response, dispatches it to the registered plugin method, appends the result to the conversation, and sends the updated history back to the model automatically. Your application code doesn't need to handle the function call / function result round-trip - SK manages that loop.

For the second turn, the model will typically call list_repositories first (because it doesn't know the full list), then call get_pipeline_status for each repo, then synthesize the answer. That's a multi-step plan executed without any orchestration code on your side.

A Note on Memory

Semantic Kernel supports semantic memory via ITextEmbeddingGenerationService and a vector store backend. The pattern:

builder.AddAzureOpenAITextEmbeddingGeneration(
    deploymentName: "text-embedding-3-small",
    endpoint: "https://<your-project>.openai.azure.com/",
    credential: new DefaultAzureCredential()
);

You'd then use ISemanticTextMemory (or the newer kernel memory abstractions in SK 1.x) to store and retrieve documents, runbooks, or conversation history by semantic similarity, grounding agent answers against a knowledge base.

Memory is worth a post of its own. The short version: for agents that need to retrieve internal documentation or past context before answering, add embedding generation and a vector store (Azure AI Search works well here via Microsoft.SemanticKernel.Connectors.AzureAISearch). For tool-calling agents like the one in this post, memory is optional.

Deploying to Azure Container Apps

The agent above is a console app. To run it as a persistent service - listening on a queue, exposing an HTTP endpoint, or processing events - containerize it and deploy to Azure Container Apps. Here's a workflow that does the full build-push-deploy cycle with OIDC authentication and no stored cloud credentials.

First, the Dockerfile (place this at the repo root, adjust paths to match your project layout):

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src
COPY src/DevOpsAgent/DevOpsAgent.csproj DevOpsAgent/
RUN dotnet restore DevOpsAgent/DevOpsAgent.csproj
COPY src/DevOpsAgent/ DevOpsAgent/
RUN dotnet publish DevOpsAgent/DevOpsAgent.csproj -c Release -o /app/publish

FROM mcr.microsoft.com/dotnet/runtime:8.0
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "DevOpsAgent.dll"]

Then the GitHub Actions workflow:

name: Deploy DevOps Agent

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read
  packages: write

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}/devops-agent
  AZURE_RESOURCE_GROUP: rg-devops-agent
  CONTAINER_APP_NAME: devops-agent
  CONTAINER_APP_ENV: cae-devops-prod

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up .NET 8
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '8.0.x'

      - name: Run tests
        run: dotnet test --no-build --verbosity normal

      - name: Log in to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push container image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: |
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest

      - name: Azure login (OIDC - no stored credentials)
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Deploy to Azure Container Apps
        uses: azure/container-apps-deploy-action@v1
        with:
          resourceGroup: ${{ env.AZURE_RESOURCE_GROUP }}
          containerAppName: ${{ env.CONTAINER_APP_NAME }}
          containerAppEnvironment: ${{ env.CONTAINER_APP_ENV }}
          imageToDeploy: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}

A few things worth noting:

OIDC instead of service principal secrets. The azure/login@v2 step exchanges a short-lived GitHub OIDC token for a short-lived Azure access token - no AZURE_CREDENTIALS JSON, no credential rotation. You configure a federated identity credential on the Azure managed identity or app registration once; the workflow runs without stored secrets from then on. The OIDC post covers this setup in detail if you haven't configured it yet.

GITHUB_TOKEN for GHCR. The packages: write permission and secrets.GITHUB_TOKEN handle container registry authentication. No PAT needed.

Agent environment variables at deploy time. The Container App needs AZURE_OPENAI_API_KEY (for development) or a managed identity binding (for production). For managed identity, assign the Cognitive Services OpenAI User role to the Container App's system-assigned identity on the Azure OpenAI resource, and the DefaultAzureCredential() in your kernel setup will work without any key in the environment.

Closing

Python developers reaching for LangChain get orchestration, tool calling, memory, and a deployment story. .NET developers get the same thing with Semantic Kernel and Azure AI Foundry - the kernel manages orchestration and tool dispatch, Foundry manages model hosting and deployment boundaries, and Container Apps runs the workload in a managed environment without cluster overhead.

The pattern here - kernel + plugin + FunctionChoiceBehavior.Auto + Foundry endpoint - is the smallest complete unit of a production-ready .NET agent. It's not a prototype. The same structure scales to multi-plugin agents, streaming responses, Redis or Azure AI Search-backed memory, and distributed agent networks with explicit handoff rules. You grow by adding plugins and services to the kernel, not by rearchitecting.

For teams already deployed on Azure, this is the path of least resistance. For teams evaluating frameworks, it's the path that keeps infrastructure and orchestration inside the same ecosystem.

Questions or corrections? Reach out.

steve.kaschimer@slalom.com

GitHub Actions Advanced Caching: Strategies That Actually Cut Build Times

2026-07-10T00:00:00Z

Your CI takes four minutes to install dependencies. With a well-designed cache, it takes eighteen seconds. The gap between those two numbers is almost never about network speed or disk I/O - it's about whether your cache key is written correctly.

actions/cache is deceptively simple to use and deceptively easy to use wrong. A key that's too broad invalidates on every run. A key that's too narrow never hits. Most teams land somewhere in the middle: a single key that hits inconsistently, a 30-40% hit rate they've stopped questioning, and CI minutes burning at the rate of a full install on every other push.

This post is the fix. Cache key anatomy, restore-keys fallback chains, branch scoping, matrix-aware keys, per-ecosystem examples, and the security model that keeps it all safe.

Why Most Caches Miss

The anti-pattern looks like this:

- uses: actions/cache@v4
  with:
    path: node_modules
    key: npm-cache

This key never changes. The first run writes the cache. Every subsequent run hits it - until someone changes a dependency, at which point the cached node_modules is stale and wrong. The solution most teams reach for is adding the lock file hash:

key: npm-${{ hashFiles('**/package-lock.json') }}

Better - but npm-a3f9c21d is the same key for an Ubuntu runner and a macOS runner, which means a cache written on one may be restored on the other. Compiled native modules, platform-specific binaries, and symlink layouts differ between operating systems. Cross-platform cache sharing silently corrupts builds.

The other failure mode is keys that are too broad - omitting the lock file hash entirely - or too narrow - including something like github.sha in the key. Committing the current SHA to the cache key guarantees a miss on every run except the one that wrote the cache. It turns actions/cache into an expensive no-op.

The actual hit rate problem is almost always one of these three things: missing the runner OS, missing the lock file hash, or including something that changes on every run. Fix those and hit rates climb above 80% without any other changes.

Cache Key Anatomy

A well-formed cache key has three components: runner OS, a stable identifier for the dependency set (usually a tool version), and a hash of the lock file. The canonical pattern for npm:

key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}

This key:

Changes when the OS changes. runner.os returns Linux, Windows, or macOS. An Ubuntu job and a macOS job have separate caches.
Changes when dependencies change. hashFiles('**/package-lock.json') hashes every package-lock.json in the repo. When npm install updates the lock file, the hash changes, the old cache key no longer matches, and the runner installs fresh.
Does not change between runs on the same OS with the same dependencies. A push to a feature branch with no dependency changes hits the same cache as the previous push to that branch.

hashFiles() accepts glob patterns. hashFiles('**/package-lock.json') covers monorepos with multiple lock files. For Gradle, hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') captures both build scripts and the wrapper version. The function computes a SHA-256 over the sorted, concatenated contents of all matching files and returns the first 16 characters of the hex digest.

If you're caching a tool that also depends on its own version - actions/setup-node, actions/setup-python, and similar - include the version in the key:

key: ${{ runner.os }}-node-${{ matrix.node-version }}-${{ hashFiles('**/package-lock.json') }}

Two jobs running different Node versions install packages against different ABIs. Their caches should be separate.

Restore-Keys Fallback Chains

When no cache entry matches the primary key, actions/cache searches for a usable partial match using the restore-keys list, in order, treating each entry as a prefix. The first partial match it finds is restored - stale, but better than nothing for most dependency managers.

Here is a three-level chain for npm:

- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-
      ${{ runner.os }}-

Read this as a priority list:

Exact key match - the current lock file hash. Full hit, nothing to install.
${{ runner.os }}-npm- prefix - a cache from a previous lock file state. Restores, then npm install adds or removes the delta packages. Much faster than a cold install.
${{ runner.os }}- prefix - any cache for this OS. Last resort, but any warmed npm cache beats downloading everything.

The reason stale caches are valuable: npm install against a partially-warm ~/.npm global cache still skips the network fetch for every package that hasn't changed. Only the diff needs downloading. For a project with 800 packages where two change, the difference between a cold install and a stale cache hit is the difference between downloading all 800 and downloading 2.

After the job completes, actions/cache writes a new cache entry for the exact key if one didn't already exist. The stale cache does not overwrite the one it was restored from - it creates a new entry. The exact-key entry is the authoritative one; the partial-match entries are fallbacks.

Branch Scoping

GitHub scopes cache access by repository and branch. A job running on a feature branch can read caches written from main, but a job on main cannot read caches written from a feature branch. Fork PRs are even more restricted: they cannot read caches from the parent repo at all. This is a deliberate security boundary.

The practical implication for cache key design: your default branch cache is the base that all feature branches fall back to. If you include github.ref in the primary key, each branch maintains an isolated cache that starts cold. If you omit github.ref, all branches share a single cache that overwrites each other on every push.

The right balance is to include github.ref in the primary key and exclude it from the restore-keys chain:

- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ github.ref_name }}-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-${{ github.ref_name }}-
      ${{ runner.os }}-npm-refs/heads/main-
      ${{ runner.os }}-npm-

This gives you:

Branch-scoped primary key - feature branches write their own caches and don't pollute the shared pool with every PR's lock file state.
Fallback to main's cache - a new branch's first run restores from main rather than installing cold.
Last-resort fallback - any npm cache for this OS, in case main doesn't have a recent entry either.

The refs/heads/main- prefix in the fallback is explicit. github.ref_name returns main on the default branch, so the prefix matches that branch's cache entries.

For short-lived feature branches, branch-scoped caches are written once and usually read once. GitHub's LRU eviction takes care of them within a few days of the branch going idle. You don't need to clean up manually.

Matrix-Aware Cache Keys

Matrix builds that share a single cache key cross-contaminate. A job testing against Node 18 and a job testing against Node 20 should not share a cache - native addons, some packages, and occasionally .bin symlinks differ between Node versions.

The fix is to include the matrix variable in the key:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest]
        node-version: ['18.x', '20.x']
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}

      - uses: actions/cache@v4
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ matrix.node-version }}-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-${{ matrix.node-version }}-
            ${{ runner.os }}-node-

      - run: npm ci
      - run: npm test

runner.os and matrix.os serve the same purpose here - the OS identifier. Using runner.os is slightly more reliable since it reflects the actual OS the runner is on, whereas matrix.os includes the latest suffix which can shift when GitHub updates the runner image. Either works; be consistent.

For a 2×2 matrix (2 OS × 2 Node versions), this produces four independent cache namespaces. Each one maintains its own history and its own fallback chain. Hits stay within the correct namespace; misses fall back to any cache for that OS/version pair, not to a different version's stale cache.

Per-Ecosystem Examples

npm

Cache ~/.npm (the global npm content-addressable cache), not node_modules. npm ci populates node_modules from ~/.npm - if the global cache is warm, npm ci takes seconds even when node_modules is cold. Caching node_modules directly is fragile: it ties the cache to the local directory structure, breaks cross-version fallbacks, and can cause subtle issues when native modules are involved.

- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-

- run: npm ci

Gradle

Gradle's cache lives in ~/.gradle/caches and ~/.gradle/wrapper. Hash both build scripts and the wrapper properties file - either can change between builds:

- uses: actions/cache@v4
  with:
    path: |
      ~/.gradle/caches
      ~/.gradle/wrapper
    key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
    restore-keys: |
      ${{ runner.os }}-gradle-

- run: ./gradlew build

The **/*.gradle* pattern matches build.gradle, build.gradle.kts, settings.gradle, and settings.gradle.kts. If your build scripts are stable but your dependency versions in a libs.versions.toml file change frequently, include that file in the hash as well.

pip

Cache ~/.cache/pip. Include the Python version in the key - wheels are version and platform-specific:

- uses: actions/setup-python@v5
  with:
    python-version: '3.12'

- uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements*.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

- run: pip install -r requirements.txt

hashFiles('**/requirements*.txt') matches requirements.txt, requirements-dev.txt, requirements-test.txt, and any other variant. If you use pyproject.toml for dependencies, substitute hashFiles('**/pyproject.toml').

If you use actions/setup-python@v5 with the cache parameter, it handles this automatically. The built-in caching in setup actions is equivalent to the manual pattern above - use whichever you prefer, but don't do both.

Docker Layer Caching

Docker layer caching in GitHub Actions uses docker/build-push-action with the cache-from and cache-to parameters and the type=gha cache backend, which writes directly to the GitHub Actions cache API:

- uses: docker/setup-buildx-action@v3

- uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
    cache-from: type=gha
    cache-to: type=gha,mode=max

mode=max caches all layers, including intermediate build stages. mode=min (the default) caches only the final stage. For multi-stage Dockerfiles where the build stage is expensive - compiling a large Go or Rust binary, for example - mode=max is worth the additional cache storage.

The type=gha backend respects the same cache scoping rules as actions/cache. A push to a feature branch can restore layers cached from main, but writes from the feature branch don't pollute the main cache. This means your CI's Docker builds get free warm-layer fallback on every branch without any additional configuration.

Cache Poisoning and GitHub's Isolation Model

Cache poisoning is the scenario where a malicious cache entry is written by an untrusted source and then read by a trusted workflow, causing it to execute or deploy compromised artifacts.

GitHub's isolation model addresses this structurally:

Caches are scoped to a repository and branch. A cache written by a workflow run in repo-A is never readable by repo-B.
Fork PRs cannot read the parent repository's cache. A pull_request event from a fork runs in the fork's security context. The ACTIONS_CACHE_URL token issued to that runner is scoped to the fork - it has no read access to the parent repo's cache entries. This blocks the attack where a malicious PR poisons the parent's cache and waits for a privileged workflow to restore it.
Cache entries are immutable once written. A key that already has an entry cannot be overwritten by another run. The first writer wins. This prevents a race where an attacker writes a poisoned entry by racing the legitimate workflow.

The practical implication: if your workflows run on pull_request events from forks (common for open source projects), the isolation model already protects you. Fork PR workflows cannot read or write your repo's caches. You do not need to add logic to skip caching on fork PRs.

The residual risk is internal: a workflow triggered from a branch in your own repo by a contributor with push access could write a malicious cache. This is a trust-level issue, not a caching issue - if a contributor has push access, they can already modify workflow files, which is a far larger attack surface than cache entries.

One concrete hardening step: use actions/cache/restore and actions/cache/save as separate steps rather than the combined actions/cache. This lets you restore before the main work and save after it, and it gives you explicit control over when and whether saving happens. In particular, you can skip the save step if the primary key already had a hit:

- uses: actions/cache/restore@v4
  id: cache-restore
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-

- run: npm ci

- uses: actions/cache/save@v4
  if: steps.cache-restore.outputs.cache-hit != 'true'
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}

The if condition prevents redundant saves when the primary key already matched. It's a minor optimization - actions/cache already avoids overwriting an existing key - but the split step pattern is valuable for a different reason: the save step runs even if the main job fails. The combined actions/cache action's post-step also runs on failure, but having explicit control makes the behavior readable rather than implicit.

Cache Size Limits and Eviction

GitHub enforces a 10 GB total cache limit per repository. When the total size exceeds 10 GB, the oldest entries are evicted until the repo is back under the limit. There is no per-entry size limit - a single large cache entry is valid, it just consumes the shared budget.

The practical limit this creates: don't cache build output. Build artifacts, compiled binaries, and test reports are not dependency caches - they change on every commit, they consume space fast, and restoring a stale build artifact is almost never useful. Cache only the inputs that don't change on every run: downloaded packages, resolved dependency trees, compiler toolchains.

A few rules that keep cache storage under control:

Cache the global package manager cache, not the local install directory. For npm, ~/.npm not node_modules. For pip, ~/.cache/pip not venv/. The global cache stores compressed tarballs; the local install directory stores unpacked files which are larger and more volatile.

Include enough specificity in the key that stale entries expire quickly. A key with a lock file hash creates a new entry every time dependencies change. If you change dependencies frequently, old entries accumulate. They'll be evicted by LRU, but if you're near the 10 GB limit, you can add an explicit cache version prefix to force a clean sweep:

key: v2-${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}

Increment v2 to v3 to immediately stop using any previously written v2 entries.

Use actions/cache/save with if: always() for jobs that run long test suites. By default, the save step only runs if the main job succeeded. For long-running jobs where you want to preserve the partially-warm state even on failure, use the split step pattern with if: always() on the save step.

Closing

The cache key is three things: runner OS, tool version or ecosystem identifier, and lock file hash. The restore-keys chain is how you trade exactness for hit rate - a stale cache that installs a 50-package delta is faster than a cold install every time. Branch scoping keeps feature caches from polluting main, and the fallback to main means new branches don't start cold. Matrix builds need one cache namespace per combination - omit the matrix variable and you're sharing state across jobs that shouldn't share it.

This is five minutes of work per workflow. The return is measured in CI minutes per week, compounded across every developer on every push. Most teams treat cache key design as an afterthought, which is why most teams have 30% hit rates when they could have 90%.

Write the key correctly. Add the fallback chain. Scope it to the branch. That's the whole thing.

Questions or corrections? Reach out.

steve.kaschimer@slalom.com

Container Image Security in CI: Scanning with Trivy and GitHub Advanced Security

2026-07-17T00:00:00Z

The scanner runs. Two hundred alerts appear. Nobody fixes anything.

This is not a Trivy problem or a GitHub problem. It's a configuration problem. An unconfigured vulnerability scanner produces a list sorted by count, not by urgency, with no distinction between CVEs that have a fix available today and CVEs where the upstream hasn't shipped a patch in two years. Every alert looks equally important, which means none of them are. The team stops looking at the output. The scanner keeps running. The checkbox stays green.

The workflow design is the security control, not the scanner. Trivy run without a severity threshold, without ignore-unfixed, and without SARIF integration into a triage workflow is just noise generation. This post is about running Trivy in a way that produces decisions: block builds on what you can fix today, surface what you can't for explicit acceptance, and eliminate the middle ground where alert fatigue lives.

The Signal-to-Noise Problem

A fresh node:20-alpine base image typically has between 20 and 80 reported CVEs. Most are MEDIUM or LOW severity. Most are in OS-level packages that your application doesn't directly invoke. Most have no fix available because the Alpine maintainers haven't shipped an update yet. Running Trivy without configuration against this image and failing the build on any vulnerability means your CI fails the day you adopt it and never passes again until you've suppressed every one of those alerts - at which point you've also suppressed the ones that matter.

The goal is a workflow where:

CRITICAL vulnerabilities with a fix available block the build immediately
HIGH vulnerabilities are reported to the Security tab for triage, but don't block every push
Unfixed vulnerabilities are tracked separately, accepted with explicit justification, and revisited when upstream patches ship
MEDIUM and below are invisible unless someone goes looking for them

This isn't being permissive about security - it's being precise about what can be acted on. A build that fails because your base image has a MEDIUM CVE in a compression library that's been unpatched for eighteen months is not a secure build. It's a build that nobody bothered to un-break. The team learns to add the suppression rule and move on. The CRITICAL vulnerability that shipped last week because the developer suppressed everything to make CI green is what you're actually trying to prevent.

Running Trivy in GitHub Actions

The aquasecurity/trivy-action wraps Trivy's scanning modes behind a clean interface. The two modes you'll use for container security are image (scan a built image from the Docker daemon or a registry) and config (scan a Dockerfile for misconfigurations).

Scanning a built image

- name: Scan container image
  uses: aquasecurity/trivy-action@0.28.0
  with:
    image-ref: 'ghcr.io/${{ github.repository }}:${{ github.sha }}'
    scan-type: 'image'
    format: 'table'
    severity: 'CRITICAL,HIGH'
    exit-code: '1'
    ignore-unfixed: true

The critical inputs:

severity limits which findings are processed. CRITICAL,HIGH means Trivy only reports and acts on those two severity levels - MEDIUM, LOW, and UNKNOWN are ignored entirely. This is your noise filter.

exit-code: '1' makes Trivy return a non-zero exit code when findings match the severity filter. Without this, Trivy scans and reports but never fails the step, which means it never blocks the build. Set this to '1'.

ignore-unfixed: true excludes CVEs where no fixed version exists in the advisory database. This is the single most important configuration decision for base image scanning. If upstream hasn't released a patch, you cannot fix it by changing your code. Failing the build on an unfixable CVE generates a suppression rule, not a security improvement.

image-ref points to the image to scan. This should be the image you just built in the same job, not a generic tag - scanning the exact artifact that will be pushed to the registry is the point.

Scanning a Dockerfile for misconfigurations

- name: Scan Dockerfile
  uses: aquasecurity/trivy-action@0.28.0
  with:
    scan-type: 'config'
    scan-ref: '.'
    format: 'table'
    exit-code: '1'

The config scan checks your Dockerfile against Trivy's misconfiguration rules: running as root, using ADD instead of COPY, missing USER directives, HEALTHCHECK absence, and others. This is orthogonal to vulnerability scanning - it catches bad Dockerfile patterns independent of what CVEs are in the image. Run both.

SARIF Output and the GitHub Security Tab

The table format is useful for humans reading workflow logs. The SARIF format is what connects Trivy to GitHub's Security tab - a persistent, triage-oriented interface that's separate from your CI runs and doesn't reset when the workflow finishes.

Generating SARIF output

- name: Scan container image (SARIF)
  uses: aquasecurity/trivy-action@0.28.0
  with:
    image-ref: 'ghcr.io/${{ github.repository }}:${{ github.sha }}'
    scan-type: 'image'
    format: 'sarif'
    output: 'trivy-results.sarif'
    severity: 'CRITICAL,HIGH'
    ignore-unfixed: true

Note: no exit-code here. The SARIF upload step is for visibility, not for build gating. The build gate is a separate step with exit-code: '1'.

Uploading to the Security tab

- name: Upload SARIF results
  uses: github/codeql-action/upload-sarif@v3
  if: always()
  with:
    sarif_file: 'trivy-results.sarif'
    category: 'container-scanning'

if: always() is important. Without it, if the Trivy scan step fails (because exit-code: '1' triggered), the upload step is skipped - which means you get a failing build but no Security tab entry showing you what failed. always() ensures the SARIF is uploaded regardless of whether the previous step succeeded.

category: 'container-scanning' namespaces the results in the Security tab, separating them from CodeQL static analysis alerts. If you run multiple Trivy scans in the same workflow (image scan + config scan), use distinct categories: container-scanning and container-config.

What the Security tab shows

The Security tab (Security → Code scanning) displays each finding as a persistent alert with its CVE ID, severity, affected package, fixed version (if available), and the workflow run that detected it. Alerts can be dismissed with a reason - "risk accepted", "false positive", "used in tests" - and a note. Dismissed alerts remain visible and auditable. A security reviewer can see what was dismissed, by whom, and why.

This matters more than it seems. A suppression in .trivyignore is a file in your repo that gets committed and approved in PRs - but it doesn't capture the reasoning. A dismissal in the Security tab has an author, a timestamp, and a reason attached. For compliance conversations, that's the difference between "we accepted this risk" and "we can prove we accepted this risk, here's who made the decision, here's when."

GitHub Advanced Security (GHAS) is required for private repositories. The Security tab and code scanning features are available on public repos for free. Private repos need either a GHAS license or GitHub Enterprise Cloud. Public repos get all of this at no cost.

Severity Thresholds That Work

The severity + exit-code combination is how you express your security policy in code. Here's a starting configuration that most teams can adopt immediately and tighten over time:

# Step 1: Upload all CRITICAL and HIGH to Security tab (always runs)
- name: Scan image - SARIF upload
  uses: aquasecurity/trivy-action@0.28.0
  with:
    image-ref: 'ghcr.io/${{ github.repository }}:${{ github.sha }}'
    scan-type: 'image'
    format: 'sarif'
    output: 'trivy-results.sarif'
    severity: 'CRITICAL,HIGH'
    ignore-unfixed: true

- name: Upload SARIF results
  uses: github/codeql-action/upload-sarif@v3
  if: always()
  with:
    sarif_file: 'trivy-results.sarif'
    category: 'container-scanning'

# Step 2: Fail the build on CRITICAL only
- name: Scan image - build gate
  uses: aquasecurity/trivy-action@0.28.0
  with:
    image-ref: 'ghcr.io/${{ github.repository }}:${{ github.sha }}'
    scan-type: 'image'
    format: 'table'
    severity: 'CRITICAL'
    exit-code: '1'
    ignore-unfixed: true

Two separate steps: one that uploads HIGH and CRITICAL to the Security tab for triage, and one that fails the build on CRITICAL only. This gives you visibility into HIGH vulnerabilities without blocking every push over something that might be low-priority for your application's threat model.

Why not block on HIGH?

MEDIUM is obviously too aggressive - the noise volume is too high and the fix rate too low. CRITICAL is clearly the right threshold to start with. HIGH is where teams argue, and the argument usually resolves as follows: block on HIGH after your team has demonstrated it can triage and resolve CRITICAL findings consistently. If your Security tab currently has 40 unresolved CRITICAL findings, adding HIGH to the build gate adds noise without adding security. Clear the CRITICAL backlog first, then raise the bar.

The threshold evolution looks like this over time:

Start: fail on CRITICAL (fixable only), report HIGH to Security tab
After 2-3 sprints: fail on CRITICAL and HIGH (fixable only), report MEDIUM to Security tab
Mature state: fail on CRITICAL and HIGH (fixable only), suppress accepted risks in .trivyignore with expiry dates, audit MEDIUM findings monthly

Don't jump to step 3 on day one. Alert fatigue kills scanning programs faster than unconfigured ones.

`ignore-unfixed` and `.trivyignore`

`ignore-unfixed: true`

When a CVE appears in the advisory database but no patched version of the affected package exists yet, it's unfixable at your level. You can't upgrade past it. The only options are: wait for upstream, switch base images, or accept the risk. Failing the build on an unfixable CVE forces you to suppress it immediately just to unblock development, which means you're creating suppressions that may never get revisited.

ignore-unfixed: true excludes these from the build gate. They still appear in the SARIF output if you want visibility. The key property: when a fix does become available, the CVE re-enters the fixable pool and your next scan will catch it.

`.trivyignore`

For vulnerabilities where you've made an explicit acceptance decision - "this library is only used in tests", "the affected code path is unreachable in our use case", "we're waiting for a base image update and this is documented in ADR-047" - .trivyignore suppresses the finding.

A basic .trivyignore:

# CVE-2024-12345 - libssl in base image, no fix available upstream (as of 2026-07-17)
# Re-evaluate when node:20-alpine ships OpenSSL 3.1.6+
CVE-2024-12345

# CVE-2024-67890 - only affects the zlib compression CLI flag we don't use
CVE-2024-67890

Trivy 0.45+ supports expiry dates, which is the right way to handle accepted-risk suppressions:

# CVE-2024-12345 exp:2026-10-17
CVE-2024-12345

When the expiry date passes, Trivy treats the suppression as absent and the CVE returns to scan results. This forces periodic re-evaluation without requiring manual calendar reminders. Set expiry dates at 90 days for base-image CVEs with active upstream work, and at 180 days for anything else.

Every entry in .trivyignore should have a comment explaining why the suppression exists. A suppression without a comment is technical debt - nobody six months later knows whether it was a considered decision or something someone added to make CI green at 4pm on a Friday.

Base Image Pinning

# Mutable - this resolved to a different image last week than it does today
FROM node:20-alpine

# Pinned - this resolves to exactly one image, forever
FROM node:20-alpine@sha256:a3f9c21d8e4b7f5d9c2a1b0e3d6f8c5e7a4b9d2f0e1c3a5b7d9f2e4c6a8b0d2

Docker image tags are mutable pointers. node:20-alpine today resolves to one image; after the next Alpine security update it resolves to a different one with different packages and potentially different vulnerabilities. Pinning to a digest guarantees that your reproducible build is actually reproducible: the same digest always resolves to the same image, always has the same package set, and always produces the same scan results.

The objection: "we want to pick up base image security updates automatically." That's correct - and it's exactly what Dependabot handles.

Keeping pinned digests updated with Dependabot

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "docker"
    directory: "/"
    schedule:
      interval: "weekly"
    commit-message:
      prefix: "docker"
    labels:
      - "dependencies"
      - "docker"

With this configuration, Dependabot opens a PR whenever a new digest is available for your pinned base image. The PR goes through your normal review process - which means your security scan runs on the new image before the updated digest merges, not after it's already in production. You get automatic updates and reproducible builds simultaneously.

If you have Dockerfiles in subdirectories, add a separate entry per directory:

  - package-ecosystem: "docker"
    directory: "/services/api"
    schedule:
      interval: "weekly"

Renovate is an alternative with more configuration options - it supports group PRs that update multiple digests in a single PR, which is useful when you have five Dockerfiles all using the same base image.

Multi-Stage Build Hardening

A standard Node.js Dockerfile often looks like this:

FROM node:20

WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

EXPOSE 3000
CMD ["node", "dist/server.js"]

This ships the build toolchain, the npm executable, the full Node.js standard library, and every package in node_modules - including your devDependencies - into the production image. The attack surface is everything in that image. If an attacker achieves code execution in the container, they have npm, node, curl, sh, and a full Linux userspace to work with.

A hardened multi-stage version:

# Build stage - has everything needed to compile
FROM node:20-alpine@sha256:a3f9c21d8e4b7f5d9c2a1b0e3d6f8c5e7a4b9d2f0e1c3a5b7d9f2e4c6a8b0d2 AS builder

WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm prune --production

# Final stage - has only what's needed to run
FROM gcr.io/distroless/nodejs20-debian12

WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules

USER nonroot

EXPOSE 3000
CMD ["dist/server.js"]

The final stage is based on gcr.io/distroless/nodejs20-debian12 - Google's distroless Node.js image. Distroless images contain only the runtime and your application. No shell, no package manager, no curl, no libc tools. The attack surface is dramatically smaller: if code execution is achieved, the attacker has the Node.js runtime and your application's files - nothing else.

What multi-stage removes from the final image:

The build toolchain (compilers, build utilities)
npm and package management tools
Shell (sh, bash)
devDependencies (pruned before the COPY step)
Build-time secrets (if any were mounted as --secret in the builder stage)
Any intermediate build artifacts not explicitly copied

npm prune --production in the builder stage before the COPY removes devDependencies from node_modules, so only production dependencies land in the final image.

USER nonroot in the distroless image runs the process as uid 65532, not root. If there's a vulnerability that allows container escape to host, running as non-root significantly limits what the attacker can do on the host.

Trivy scanning the distroless final image instead of the node:20 image typically reduces the CVE count by 60-80%. The scan still runs - distroless isn't vulnerability-free - but the surface area is smaller and the results are more actionable.

What to Do With Unfixable CVEs in Your Base Image

You've run Trivy with ignore-unfixed: false (temporarily, to assess the full picture) and found a CVE in your base image where upstream hasn't patched yet. Your options, in order of preference:

1. Wait for upstream. If the CVE is in the base OS packages and upstream is actively working on it, the right answer is often to wait. Pin the current digest, monitor the base image's release notes, and have Dependabot open a PR when the updated digest arrives. Your SARIF upload keeps the finding visible in the Security tab while you wait.

2. Switch base images. If a specific base image has a pattern of slow patching, consider alternatives. Chainguard's Node.js images (cgr.dev/chainguard/node) are rebuilt daily from source with minimal packages and often have zero CVEs in the base OS. The tradeoff is ecosystem stability - Chainguard images are more aggressive about removing old package versions, which can require more active maintenance of your Dockerfile.

Alpine-based images (node:20-alpine) generally have lower CVE counts than Debian-based ones (node:20) because Alpine uses musl libc and a smaller package set. If you're using a Debian base without a specific reason, switching to Alpine often resolves several unfixable CVEs immediately.

3. Suppress with justification in .trivyignore. When the CVE is genuinely not applicable to your use case - the vulnerable code path isn't reachable from your application, the affected binary isn't present in your final stage, or the CVSS vector doesn't apply to your deployment - suppress it with a comment explaining the reasoning and an expiry date. This is the correct pattern for accepted risk, not a workaround.

4. Document the decision as an ADR. For CVEs that will be suppressed for more than a sprint cycle, an Architecture Decision Record (as covered in an earlier post) is the right place to document the threat model reasoning. An ADR that says "we accept CVE-2024-XXXX in the base image because X, and we will re-evaluate when Y" is auditable and revisable. A .trivyignore comment is not, really.

Full Workflow Example

This is a complete job: build the image, scan it with Trivy (SARIF upload + build gate), and push to the registry only if the scan passes. Registry authentication uses OIDC - no stored credentials.

name: Build and Scan

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  id-token: write        # For OIDC registry authentication
  security-events: write # For SARIF upload to Security tab
  packages: write        # For GHCR push

jobs:
  build-scan-push:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build image (do not push yet)
        uses: docker/build-push-action@v6
        with:
          context: .
          push: false
          load: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

      # Scan 1: Upload CRITICAL and HIGH to Security tab
      - name: Scan image - SARIF upload
        uses: aquasecurity/trivy-action@0.28.0
        with:
          image-ref: 'ghcr.io/${{ github.repository }}:${{ github.sha }}'
          scan-type: 'image'
          format: 'sarif'
          output: 'trivy-results.sarif'
          severity: 'CRITICAL,HIGH'
          ignore-unfixed: true

      - name: Upload Trivy SARIF to Security tab
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: 'trivy-results.sarif'
          category: 'container-scanning'

      # Scan 2: Block build on CRITICAL only
      - name: Scan image - build gate
        uses: aquasecurity/trivy-action@0.28.0
        with:
          image-ref: 'ghcr.io/${{ github.repository }}:${{ github.sha }}'
          scan-type: 'image'
          format: 'table'
          severity: 'CRITICAL'
          exit-code: '1'
          ignore-unfixed: true

      # Scan 3: Dockerfile misconfiguration check
      - name: Scan Dockerfile for misconfigurations
        uses: aquasecurity/trivy-action@0.28.0
        with:
          scan-type: 'config'
          scan-ref: '.'
          format: 'table'
          exit-code: '1'

      # Push only if all scans pass
      - name: Push image to registry
        uses: docker/build-push-action@v6
        with:
          context: .
          push: ${{ github.ref == 'refs/heads/main' }}
          tags: |
            ghcr.io/${{ github.repository }}:${{ github.sha }}
            ghcr.io/${{ github.repository }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

A few things worth noting:

Build before push, scan before push. The build-push-action with push: false and load: true builds the image and loads it into the Docker daemon without pushing to the registry. The scans run against this local image. The final step pushes only after all scans pass. This ensures the image in the registry was scanned before it arrived - not after.

security-events: write permission is required for upload-sarif. Without it, the upload step fails with a permissions error. Include it in the job-level permissions block.

${{ secrets.GITHUB_TOKEN }} for GHCR authentication is not a stored secret - GITHUB_TOKEN is the auto-provisioned workflow token. Combined with packages: write permission, it authenticates to the GitHub Container Registry without any manual credential setup.

Push condition ${{ github.ref == 'refs/heads/main' }} pushes to the registry only on merges to main. On PRs, the workflow builds and scans without pushing. This gives you scan feedback on PRs without polluting the registry with PR builds.

Container Scanning Checklist

[ ] Pin base image digests in all Dockerfiles - FROM node:20-alpine@sha256:..., not FROM node:20-alpine
[ ] Configure Dependabot for Docker (package-ecosystem: "docker") to auto-update pinned digests weekly
[ ] Use multi-stage builds: builder stage with full toolchain, final stage with distroless or alpine runtime only
[ ] Run USER nonroot (distroless) or create a non-root user in the final stage
[ ] Set ignore-unfixed: true on the build gate scan - only block on what can be fixed
[ ] Start with severity: 'CRITICAL' for the build gate; add HIGH after the team has cleared the CRITICAL backlog
[ ] Upload severity: 'CRITICAL,HIGH' to the Security tab with if: always() on the upload step
[ ] Run a separate Dockerfile config scan (scan-type: 'config') as its own build gate step
[ ] Add .trivyignore entries for accepted-risk CVEs with comment and expiry date (exp:YYYY-MM-DD)
[ ] For long-lived suppressions, document the decision in an ADR - not just a .trivyignore comment
[ ] Build before you push: scan the local image, push to registry only if scans pass
[ ] For private repos: confirm GHAS is enabled for the Security tab to receive SARIF uploads
[ ] Require security-events: write permission in the workflow for upload-sarif to work

Closing

The scanner is the easy part. Trivy is well-maintained, the aquasecurity/trivy-action integration is straightforward, and the SARIF upload to GitHub's Security tab takes five lines of YAML. None of that is where teams struggle.

Where teams struggle is the policy: what to block on, what to report but not block on, and what to explicitly accept. An unconfigured scanner produces undifferentiated noise. A scanner configured to block on CRITICAL-only with ignore-unfixed: true produces decisions - specifically, the decision to fix something or accept it with a documented reason.

The workflow design is the security control. The scanner surfaces findings; the severity threshold, the ignore-unfixed flag, and the .trivyignore file express your team's threat model in code. Treat them that way: review changes to .trivyignore as carefully as changes to your authentication logic, and treat the first CRITICAL finding in the Security tab as a P1, not a thing to suppress before the end of the sprint.

Pinned digests, multi-stage builds, and distroless final images reduce the finding count before the scanner runs. A smaller attack surface produces a more actionable scan report. Both things are true: reduce the surface and scan what remains.

Questions about Trivy configuration, GHAS setup, or container hardening? Reach out.

steve.kaschimer@slalom.com

Multi-Agent Patterns with Azure AI Foundry: Orchestration, Handoff, and Shared State

2026-07-24T00:00:00Z

The bottleneck in agentic systems isn't the model. It's the orchestration.

A single agent eventually runs into three walls: the context window fills up before a complex task is done, specialization conflicts with generalization (a security reviewer that also writes code does both worse), and latency stacks up when one agent is responsible for every step of a pipeline. None of those are model problems. They're architecture problems.

Azure AI Foundry provides the primitives to solve them - managed agent runtime, thread-based message passing, tool registration - without requiring you to build orchestration infrastructure from scratch. But Foundry doesn't tell you which pattern to reach for. That's the design decision, and it determines whether your multi-agent system scales or collapses under real workloads.

This post covers three patterns: sequential pipeline, parallel fan-out, and hierarchical orchestrator/sub-agent. Each one addresses a different constraint. They're not mutually exclusive. The closing section covers shared state, error handling, and observability across all three.

If you haven't set up a Foundry project yet, start with the first-look post before continuing here.

Setup

Install the required packages:

pip install azure-ai-projects azure-ai-agents azure-identity opentelemetry-sdk opentelemetry-api

The azure-ai-projects package is your entry point to Foundry project resources. azure-ai-agents provides the agent and thread runtime. All examples below use Python 3.11+.

Initialize the project client once and reuse it:

import os
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

client = AIProjectClient(
    endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)

AZURE_AI_PROJECT_ENDPOINT is the endpoint URI for your Foundry project - find it in the portal under your project's Overview tab.

Pattern 1: Sequential Pipeline

Each agent's output is the next agent's input. The pipeline runs to completion in order. No step starts until the previous step finishes.

Use case: a content pipeline - research agent gathers sources, draft agent writes from them, review agent flags problems, publish agent formats the final output. Each stage depends strictly on the previous one's output. You can't draft before you have sources; you can't review a draft that doesn't exist.

When to use it: strict data dependencies between stages. The output of stage N is meaningfully different from the input to stage N - it's not just forwarded, it's transformed.

Where it breaks: error propagation and latency. A failure in stage 2 kills stages 3 and 4. And because stages run serially, total latency is the sum of all stage latencies. Don't use this pattern for independent subtasks - that's what fan-out is for.

Here's a minimal sequential pipeline with three agents - researcher, drafter, and reviewer:

import asyncio
from azure.ai.agents.models import MessageRole

async def run_sequential_pipeline(topic: str) -> str:
    agents_client = client.agents

    # Create three specialized agents
    researcher = await agents_client.create_agent(
        model="gpt-4o",
        name="researcher",
        instructions=(
            "You are a technical researcher. Given a topic, return a structured "
            "set of key findings and source references. Be factual and concise."
        ),
    )
    drafter = await agents_client.create_agent(
        model="gpt-4o",
        name="drafter",
        instructions=(
            "You are a technical writer. Given research findings, produce a clear, "
            "well-structured draft suitable for a developer audience."
        ),
    )
    reviewer = await agents_client.create_agent(
        model="gpt-4o",
        name="reviewer",
        instructions=(
            "You are a technical editor. Given a draft, identify factual errors, "
            "unclear explanations, and structural problems. Return structured feedback."
        ),
    )

    try:
        # Stage 1: Research
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id,
            role=MessageRole.USER,
            content=f"Research this topic and return structured findings: {topic}",
        )
        run = await agents_client.create_and_process_run(
            thread_id=thread.id, agent_id=researcher.id
        )
        research_output = await _get_last_message(agents_client, thread.id)

        # Stage 2: Draft - pass research output as input to a new thread
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id,
            role=MessageRole.USER,
            content=f"Write a draft based on these research findings:\n\n{research_output}",
        )
        run = await agents_client.create_and_process_run(
            thread_id=thread.id, agent_id=drafter.id
        )
        draft_output = await _get_last_message(agents_client, thread.id)

        # Stage 3: Review - pass draft as input
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id,
            role=MessageRole.USER,
            content=f"Review this draft and return structured feedback:\n\n{draft_output}",
        )
        run = await agents_client.create_and_process_run(
            thread_id=thread.id, agent_id=reviewer.id
        )
        review_output = await _get_last_message(agents_client, thread.id)

        return review_output

    finally:
        # Clean up agents after use
        for agent in [researcher, drafter, reviewer]:
            await agents_client.delete_agent(agent.id)


async def _get_last_message(agents_client, thread_id: str) -> str:
    messages = await agents_client.list_messages(thread_id=thread_id)
    assistant_messages = [
        m for m in messages.data if m.role == MessageRole.ASSISTANT
    ]
    if not assistant_messages:
        raise RuntimeError(f"No assistant message in thread {thread_id}")
    # Messages are returned newest-first
    content = assistant_messages[0].content
    return content[0].text.value if content else ""

Each stage gets its own thread. The state handoff is explicit: the orchestrator extracts the final message from stage N and passes it as the user message to stage N+1. This is intentional - implicit state sharing between threads creates hard-to-trace bugs. Make the data flow visible in the code.

Pattern 2: Parallel Fan-Out

The orchestrator dispatches to N specialist agents simultaneously and collects results. Stages don't depend on each other - they run in parallel and converge at aggregation.

Use case: analyzing a codebase across multiple dimensions simultaneously - security vulnerabilities, performance anti-patterns, code style violations. Each dimension is independent. Running them sequentially wastes time.

When to use it: independent subtasks that don't need each other's output before they start. Total latency is bounded by the slowest agent, not the sum of all agents.

Where it breaks: partial failure. If one agent fails, do you fail the whole operation or return partial results? That's a policy decision. The example below returns partial results and includes failure information in the aggregated output.

import asyncio
from dataclasses import dataclass
from typing import Optional

@dataclass
class AgentResult:
    dimension: str
    output: Optional[str]
    error: Optional[str] = None

    @property
    def succeeded(self) -> bool:
        return self.error is None


async def analyze_dimension(
    agents_client,
    dimension: str,
    instructions: str,
    content: str,
) -> AgentResult:
    agent = await agents_client.create_agent(
        model="gpt-4o",
        name=f"analyst-{dimension}",
        instructions=instructions,
    )
    try:
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id,
            role=MessageRole.USER,
            content=content,
        )
        await agents_client.create_and_process_run(
            thread_id=thread.id, agent_id=agent.id
        )
        output = await _get_last_message(agents_client, thread.id)
        return AgentResult(dimension=dimension, output=output)
    except Exception as exc:
        return AgentResult(dimension=dimension, output=None, error=str(exc))
    finally:
        await agents_client.delete_agent(agent.id)


async def run_parallel_fanout(codebase_summary: str) -> dict[str, AgentResult]:
    agents_client = client.agents

    dimensions = {
        "security": (
            "You are a security reviewer. Analyze the provided code for vulnerabilities, "
            "hardcoded secrets, injection risks, and insecure dependencies. Return findings "
            "as a JSON array of {severity, finding, location} objects."
        ),
        "performance": (
            "You are a performance engineer. Identify N+1 queries, blocking I/O, "
            "unnecessary allocations, and missing cache opportunities. Return findings "
            "as a JSON array of {impact, finding, recommendation} objects."
        ),
        "style": (
            "You are a code reviewer focused on maintainability. Flag naming inconsistencies, "
            "missing error handling, and structural problems. Return findings "
            "as a JSON array of {severity, finding, suggestion} objects."
        ),
    }

    tasks = [
        analyze_dimension(
            agents_client=agents_client,
            dimension=dim,
            instructions=instructions,
            content=f"Analyze this codebase summary:\n\n{codebase_summary}",
        )
        for dim, instructions in dimensions.items()
    ]

    # All three agents run simultaneously
    results = await asyncio.gather(*tasks, return_exceptions=False)

    aggregated = {r.dimension: r for r in results}

    failed = [r.dimension for r in results if not r.succeeded]
    if failed:
        # Log partial failure but return what succeeded
        print(f"Warning: {len(failed)} analysis dimension(s) failed: {failed}")

    return aggregated

asyncio.gather() dispatches all coroutines concurrently. If one agent call raises an unhandled exception, gather propagates it by default. The try/except inside analyze_dimension converts agent-level failures into structured AgentResult objects so the aggregation layer can reason about them without crashing.

The caller decides what to do with partial results. That's the right place for that policy - not buried inside the agent call.

Pattern 3: Hierarchical Orchestrator/Sub-Agent

A planner agent decomposes an incoming task and routes subtasks to specialists via tool calls. The orchestrator never executes domain logic directly - it delegates. Specialists never decide what to work on - they only execute what's delegated.

Use case: a customer support system. An orchestrator receives incoming requests and routes to a billing agent, a technical support agent, or an escalation agent based on the content. The orchestrator understands intent; the specialists own execution.

When to use it: dynamic routing where the task structure isn't known in advance. The orchestrator figures out which specialist to call based on the input - something you can't hard-code in a sequential pipeline.

Where it breaks: prompt quality for the orchestrator. If the orchestrator's routing logic is vague, it will hallucinate routing decisions. Tool definitions must be precise - the orchestrator picks tools the same way models pick any function: based on the description.

The orchestrator's tools are wrappers around sub-agent calls. Define them as standard function tools:

import json
from azure.ai.agents.models import FunctionTool, ToolSet

async def call_billing_agent(query: str) -> str:
    """Route a billing-related query to the billing specialist agent."""
    agents_client = client.agents
    agent = await agents_client.create_agent(
        model="gpt-4o",
        name="billing-specialist",
        instructions=(
            "You are a billing specialist. Answer questions about invoices, charges, "
            "subscription changes, and refunds accurately and concisely. If you cannot "
            "resolve the issue, return a JSON object: {\"escalate\": true, \"reason\": \"...\"}."
        ),
    )
    try:
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id, role=MessageRole.USER, content=query
        )
        await agents_client.create_and_process_run(
            thread_id=thread.id, agent_id=agent.id
        )
        return await _get_last_message(agents_client, thread.id)
    finally:
        await agents_client.delete_agent(agent.id)


async def call_technical_agent(query: str) -> str:
    """Route a technical support query to the technical specialist agent."""
    agents_client = client.agents
    agent = await agents_client.create_agent(
        model="gpt-4o",
        name="technical-specialist",
        instructions=(
            "You are a technical support specialist. Diagnose and resolve technical issues "
            "with API integrations, SDK behavior, and platform configuration. Return a "
            "structured resolution or escalation recommendation."
        ),
    )
    try:
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id, role=MessageRole.USER, content=query
        )
        await agents_client.create_and_process_run(
            thread_id=thread.id, agent_id=agent.id
        )
        return await _get_last_message(agents_client, thread.id)
    finally:
        await agents_client.delete_agent(agent.id)


# Tool definitions the orchestrator will see
billing_tool = FunctionTool(
    name="route_to_billing",
    description=(
        "Route a customer query to the billing specialist. Use this for questions about "
        "invoices, charges, subscription plans, payment failures, and refund requests."
    ),
    parameters={
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "The customer's billing question, verbatim.",
            }
        },
        "required": ["query"],
    },
)

technical_tool = FunctionTool(
    name="route_to_technical",
    description=(
        "Route a customer query to the technical support specialist. Use this for questions "
        "about API errors, SDK configuration, integration issues, and platform behavior."
    ),
    parameters={
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "The customer's technical question, verbatim.",
            }
        },
        "required": ["query"],
    },
)

Wire the orchestrator with the tool definitions and a function dispatch map:

async def run_orchestrator(customer_query: str) -> str:
    agents_client = client.agents

    toolset = ToolSet()
    toolset.add(billing_tool)
    toolset.add(technical_tool)

    orchestrator = await agents_client.create_agent(
        model="gpt-4o",
        name="support-orchestrator",
        instructions=(
            "You are a customer support orchestrator. Analyze the customer query and route it "
            "to the appropriate specialist using the available tools. You do not answer "
            "domain questions directly - you always delegate. If the query spans multiple "
            "domains, make multiple tool calls. Synthesize the specialist responses into a "
            "single coherent reply for the customer."
        ),
        tools=toolset.definitions,
    )

    # Map tool names to async handler functions
    tool_handlers = {
        "route_to_billing": call_billing_agent,
        "route_to_technical": call_technical_agent,
    }

    try:
        thread = await agents_client.create_thread()
        await agents_client.create_message(
            thread_id=thread.id, role=MessageRole.USER, content=customer_query
        )

        # Process the run, handling tool calls as they arrive
        run = await agents_client.create_run(
            thread_id=thread.id, agent_id=orchestrator.id
        )

        while run.status in ("queued", "in_progress", "requires_action"):
            run = await agents_client.get_run(thread_id=thread.id, run_id=run.id)

            if run.status == "requires_action":
                tool_outputs = []
                for tool_call in run.required_action.submit_tool_outputs.tool_calls:
                    fn_name = tool_call.function.name
                    fn_args = json.loads(tool_call.function.arguments)
                    handler = tool_handlers.get(fn_name)
                    if handler:
                        result = await handler(**fn_args)
                    else:
                        result = json.dumps({"error": f"Unknown tool: {fn_name}"})
                    tool_outputs.append(
                        {"tool_call_id": tool_call.id, "output": result}
                    )
                run = await agents_client.submit_tool_outputs_to_run(
                    thread_id=thread.id,
                    run_id=run.id,
                    tool_outputs=tool_outputs,
                )
            elif run.status not in ("queued", "in_progress"):
                break

            await asyncio.sleep(0.5)

        return await _get_last_message(agents_client, thread.id)

    finally:
        await agents_client.delete_agent(orchestrator.id)

The orchestrator reasons about the query and decides which tool to call - or both, in sequence or parallel, if the query spans domains. Your application code handles the dispatch; the orchestrator handles the routing logic. Neither leaks into the other's domain.

Shared State and Memory

Agents don't share memory by default. Every thread is isolated. That isolation is a feature - it prevents state contamination across runs. But real pipelines need shared context. There are three ways to provide it, and they trade off differently.

Passing state in messages is the simplest approach and the one used in all the examples above. The orchestrator extracts output from agent N and passes it as input to agent N+1. The data flow is explicit and traceable. The limitation: large context increases token usage at every step, and there's no retrieval - if agent N+3 needs something from agent N, the orchestrator has to carry it the whole way.

Azure AI Search as a shared vector store works well when agents need to retrieve knowledge from a shared corpus - documentation, previous run outputs, domain knowledge. Each agent gets an Azure AI Search connection and performs retrieval before generating its response. The agents don't share working memory; they share a retrieval layer.

from azure.ai.projects.models import AzureAISearchTool

search_tool = AzureAISearchTool(
    index_connection_id=os.environ["AZURE_AI_SEARCH_CONNECTION_ID"],
    index_name="shared-knowledge-base",
)

agent = await agents_client.create_agent(
    model="gpt-4o",
    name="retrieval-agent",
    instructions="Use the available search tool to retrieve relevant context before answering.",
    tools=[search_tool.definitions],
    tool_resources=search_tool.resources,
)

Azure Cosmos DB or Blob Storage for structured shared state is the right choice when agents need to write state that other agents will read - progress records, intermediate computation results, task queues. The orchestrator writes a structured record after each agent completes; the next agent reads it at startup. This decouples agents from each other in time, which matters for long-running pipelines where agents may run hours apart.

The tradeoff is straightforward: pass state in messages when it's small and the pipeline is short; externalize it when it's large, needs retrieval, or spans a pipeline that runs over minutes or hours.

Error Handling When a Sub-Agent Fails

Sub-agents fail. Networks time out, rate limits hit, model responses come back malformed. The orchestrator needs to handle these failures gracefully rather than propagating exceptions upward blindly.

The key design decision: errors from sub-agents should be structured, not plain text. The orchestrator is a model - it reasons over tool outputs. A structured error gives the model enough information to make a policy decision (retry, fallback, degrade gracefully). A plain text exception does not.

import asyncio
from typing import Any

async def call_agent_with_retry(
    agents_client,
    agent_id: str,
    query: str,
    max_retries: int = 3,
    base_delay: float = 1.0,
) -> dict[str, Any]:
    """
    Call an agent with exponential backoff retry.
    Returns a structured result the orchestrator can reason about.
    """
    last_error = None

    for attempt in range(max_retries):
        try:
            thread = await agents_client.create_thread()
            await agents_client.create_message(
                thread_id=thread.id, role=MessageRole.USER, content=query
            )
            run = await agents_client.create_and_process_run(
                thread_id=thread.id, agent_id=agent_id
            )
            output = await _get_last_message(agents_client, thread.id)
            return {
                "status": "success",
                "output": output,
                "attempt": attempt + 1,
            }

        except Exception as exc:
            last_error = exc
            if attempt < max_retries - 1:
                delay = base_delay * (2 ** attempt)
                await asyncio.sleep(delay)

    # All retries exhausted - return a structured failure
    return {
        "status": "failed",
        "error_type": type(last_error).__name__,
        "error_message": str(last_error),
        "attempts": max_retries,
    }

The orchestrator receives a dict in both the success and failure cases. If status == "failed", the orchestrator can choose a fallback path - a simpler agent, a cached response, a graceful degradation message to the user - without that decision being forced on it by an unhandled exception.

Return structured JSON errors, not plain text. A model reading "The billing service is unavailable due to a transient error after 3 attempts." has less to work with than a model reading {"status": "failed", "error_type": "TimeoutError", "attempts": 3}. The latter is something it can act on.

Observability: Tracing a Request Through the Chain

A request enters the orchestrator and fans out across multiple agents, threads, and model calls. Without tracing, debugging a failure means reading logs in isolation without understanding causality. With tracing, you can reconstruct the entire call graph from a single trace_id.

Azure AI Foundry integrates with Azure Monitor / Application Insights. Enable it on the project client:

from azure.ai.projects.models import ApplicationInsightsConfiguration

# Enable built-in tracing to Application Insights
client.telemetry.enable(
    destination=client.telemetry.get_connection_string()
)

For custom spans at each handoff point, use OpenTelemetry directly:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

# Configure the tracer once at startup
provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter())
)
trace.set_tracer_provider(provider)

tracer = trace.get_tracer("multi-agent-pipeline")


async def run_pipeline_with_tracing(topic: str) -> str:
    with tracer.start_as_current_span("pipeline.run") as root_span:
        root_span.set_attribute("pipeline.topic", topic)

        with tracer.start_as_current_span("agent.research") as span:
            span.set_attribute("agent.model", "gpt-4o")
            span.set_attribute("agent.name", "researcher")
            research_output = await run_research_stage(topic)
            span.set_attribute("agent.output_tokens", len(research_output.split()))

        with tracer.start_as_current_span("agent.draft") as span:
            span.set_attribute("agent.model", "gpt-4o")
            span.set_attribute("agent.name", "drafter")
            span.set_attribute("agent.input_source", "researcher.output")
            draft_output = await run_draft_stage(research_output)

        with tracer.start_as_current_span("agent.review") as span:
            span.set_attribute("agent.model", "gpt-4o")
            span.set_attribute("agent.name", "reviewer")
            review_output = await run_review_stage(draft_output)

        return review_output

What to record at each handoff: input token count, output content hash or length (not the full content - keep spans lightweight), the model and deployment name, the thread ID and run ID, and latency. The thread ID and run ID let you correlate a span in your trace backend with the full conversation in the Foundry portal.

Correlate the Foundry run_id with your trace spans by attaching it as an attribute. When a run fails, you can pivot from your trace view directly to the Foundry thread that contains the full model interaction.

Choosing the Right Pattern

The decision is mechanical once you understand the constraints:

Sequential pipeline when: stage N's output is the direct input to stage N+1, and partial results from stage N have no value if stage N+1 never runs. Use it when data transformation between stages is the point.

Parallel fan-out when: subtasks are independent - they don't read each other's outputs before they start. Use it when the bottleneck is latency and the subtasks can run side-by-side.

Hierarchical orchestrator/sub-agent when: the routing decision is dynamic and can't be hard-coded. The orchestrator reasons about what to call and when. Use it when different requests need different agent combinations.

Patterns compose. A hierarchical orchestrator can route to a sequential pipeline for one task class and a parallel fan-out for another. Start with the simplest pattern that fits the constraint. Introduce complexity when the constraint changes.

Closing

The orchestration layer is where agentic applications win or lose. A single agent with GPT-4o and a good prompt is a prototype. A multi-agent system that correctly decomposes tasks, routes dynamically, shares state explicitly, handles partial failures gracefully, and produces traceable execution is production infrastructure.

Azure AI Foundry gives you the building blocks: managed agent runtime, thread isolation, tool registration, built-in observability. The architecture is still yours to design. The three patterns here - sequential pipeline, parallel fan-out, hierarchical orchestrator - cover most of the routing problems you'll encounter. What they don't do is choose themselves.

Pick the pattern that matches the constraint. Don't add orchestration complexity until you've hit the wall that requires it.

Questions or corrections? Reach out.

steve.kaschimer@slalom.com

IaC Security Scanning in CI: Catching Terraform and Bicep Misconfigurations Before They Deploy

2026-07-31T00:00:00Z

An S3 bucket with public read access. A storage account without encryption at rest. A virtual machine with SSH open to 0.0.0.0/0. These are not exotic attack vectors - they are the default configuration in half the tutorials on the internet, and they ship to production every day because nobody ran a scanner before terraform apply.

IaC misconfigurations are infrastructure bugs. The same discipline that catches application bugs in CI - write a check, fail the build, fix before merge - applies here. The tooling exists. The GitHub Actions integration is straightforward. The reason most teams don't do this isn't complexity; it's that nobody set it up on the first sprint, and now the pipeline has been running without it for eighteen months.

This post sets it up. Checkov and tfsec for Terraform, PSRule for Bicep, SARIF output to the GitHub Security tab, thresholds that block on exploitable misconfigurations without halting every PR for cosmetic issues, and inline suppressions that are visible in PR diffs so they actually get reviewed.

The Cost Curve

The argument for catching misconfigurations in CI is simple arithmetic. A Checkov finding caught in a pull request takes five minutes to fix. The same misconfiguration caught by a cloud security posture management tool after deployment takes hours to investigate, validate, and remediate - with a window of exposure that starts at merge and ends when someone acts on the alert. Caught post-breach, the cost is measured differently.

The CI integration is a one-time investment. Every PR that runs through it gets a scan for free.

Tool Landscape

Three tools cover most of the ground:

Checkov is the most broadly adopted. It's a Python-based static analysis framework from Bridgecrew (now Prisma Cloud) that covers Terraform, Bicep, ARM, CloudFormation, Kubernetes manifests, Dockerfiles, and more. It has over 1,000 built-in rules, produces SARIF natively, and has both a CLI and an official GitHub Action. It's the right default for mixed-provider or multi-format repositories.

tfsec is a Go-based scanner focused exclusively on Terraform. It's fast - faster than Checkov on large Terraform directories - and its rule set has different coverage than Checkov's. The two scanners are complements: running both in parallel catches things either one might miss. tfsec was acquired by Aqua Security and is actively maintained as aquasecurity/tfsec.

PSRule for Azure is a PowerShell-based framework from Microsoft, purpose-built for Bicep and ARM templates. It covers naming conventions, SKU restrictions, security baselines, and Azure Well-Architected Framework alignment. If your infrastructure is Azure-first and you're using Bicep, PSRule is the right tool for that layer - its rules are closer to Azure-native than the generic cloud rules in Checkov.

The short version: use Checkov as the baseline for everything, add tfsec for Terraform for faster feedback and additional coverage, and add PSRule for Bicep if you're on Azure.

Checkov for Terraform

Checkov's GitHub Action is bridgecrewio/checkov-action. It scans a directory, applies your rule configuration, and - critically - produces SARIF output that uploads directly to the GitHub Security tab.

A minimal step:

- name: Checkov scan
  uses: bridgecrewio/checkov-action@v12
  with:
    directory: terraform/
    output_format: sarif
    output_file_path: checkov-results.sarif
    soft_fail: true

soft_fail: true means the step exits 0 even when findings exist - the build failure comes later, after the SARIF is uploaded. This is the right pattern: you want findings in the Security tab regardless of whether they block the build.

Key flags to know when using the CLI directly (pip install checkov):

# Hard fail on HIGH and CRITICAL, soft fail on everything else
checkov -d terraform/ \
  --output sarif \
  --output-file-path checkov-results.sarif \
  --soft-fail-on LOW,MEDIUM \
  --compact

# Run only specific checks
checkov -d terraform/ --check CKV_AWS_18,CKV_AWS_21

# Skip specific checks
checkov -d terraform/ --skip-check CKV_AWS_18

The --soft-fail-on LOW,MEDIUM flag is the threshold design: LOW and MEDIUM findings don't block the job; HIGH and CRITICAL do. This keeps the scanner from becoming noise-only by day two.

To upload results to the GitHub Security tab, you need the security-events: write permission and the github/codeql-action/upload-sarif action:

permissions:
  security-events: write
  contents: read

jobs:
  checkov:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Checkov scan
        uses: bridgecrewio/checkov-action@v12
        with:
          directory: terraform/
          output_format: sarif
          output_file_path: checkov-results.sarif
          soft_fail: true

      - name: Upload SARIF to GitHub Security tab
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: checkov-results.sarif
          category: checkov

if: always() on the upload step is not optional. When Checkov finds HIGH or CRITICAL issues and exits non-zero, subsequent steps don't run by default. if: always() ensures the SARIF upload happens even when the scan step fails - which is exactly when you most need the results in the Security tab.

tfsec for Terraform

tfsec is faster than Checkov on pure Terraform - it compiles to a single Go binary and doesn't have the Python startup overhead. For a large Terraform monorepo, the difference is meaningful. Its rule set also differs from Checkov's; running both in parallel catches more than either alone.

The official action is aquasecurity/tfsec-action:

- name: tfsec scan
  uses: aquasecurity/tfsec-action@v1.0.0
  with:
    working_directory: terraform/
    format: sarif
    sarif_file: tfsec-results.sarif
    soft_fail: true

- name: Upload tfsec SARIF
  uses: github/codeql-action/upload-sarif@v3
  if: always()
  with:
    sarif_file: tfsec-results.sarif
    category: tfsec

CLI equivalent:

tfsec terraform/ \
  --format sarif \
  --out tfsec-results.sarif \
  --minimum-severity HIGH

--minimum-severity HIGH means tfsec exits non-zero only when it finds HIGH or CRITICAL issues. LOW and MEDIUM are still written to the SARIF file and appear in the Security tab - they just don't break the build. This mirrors the Checkov threshold design and makes both scanners consistent in their blocking behavior.

To run Checkov and tfsec in parallel (the right default for Terraform repositories), use separate jobs:

jobs:
  checkov:
    runs-on: ubuntu-latest
    permissions:
      security-events: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: bridgecrewio/checkov-action@v12
        with:
          directory: terraform/
          output_format: sarif
          output_file_path: checkov-results.sarif
          soft_fail: true
      - uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: checkov-results.sarif
          category: checkov

  tfsec:
    runs-on: ubuntu-latest
    permissions:
      security-events: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: aquasecurity/tfsec-action@v1.0.0
        with:
          working_directory: terraform/
          format: sarif
          sarif_file: tfsec-results.sarif
          soft_fail: true
      - uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: tfsec-results.sarif
          category: tfsec

Two parallel jobs, two SARIF uploads, two categories in the Security tab. The category parameter keeps the findings separated - Checkov findings and tfsec findings don't merge into a single undifferentiated list.

PSRule for Bicep

PSRule for Azure is the right scanner for Bicep templates. It's maintained by Microsoft, its rules map directly to Azure Policy and Well-Architected Framework requirements, and it understands Bicep's type system better than general-purpose scanners do.

The GitHub Action is microsoft/ps-rule:

- name: PSRule for Azure
  uses: microsoft/ps-rule@v2
  with:
    modules: PSRule.Rules.Azure
    inputPath: bicep/
    outputFormat: Sarif
    outputPath: psrule-results.sarif
    outcome: Fail

- name: Upload PSRule SARIF
  uses: github/codeql-action/upload-sarif@v3
  if: always()
  with:
    sarif_file: psrule-results.sarif
    category: psrule

outcome: Fail means the step exits non-zero only for failed rules, not for warnings or informational findings. Adjust to outcome: Problem if you want warnings to block as well.

What PSRule.Rules.Azure covers that Checkov doesn't: Azure naming convention enforcement, allowed SKUs for compute and storage resources, diagnostic settings requirements, Azure Policy compliance, and resource-level security baselines that are Azure-specific (Key Vault soft delete, Azure Monitor integration, Defender for Cloud plans). For Azure-native teams, this layer is worth adding even when Checkov is already running.

PSRule also supports a ps-rule.yaml baseline configuration file at the repo root for module-wide settings:

# ps-rule.yaml
requires:
  PSRule.Rules.Azure: '>=1.38.0'

configuration:
  AZURE_BICEP_CHECK_IMPORTS: true
  AZURE_BICEP_FILE_EXPANSION: true

AZURE_BICEP_FILE_EXPANSION: true tells PSRule to expand Bicep templates before analysis rather than scanning the raw HCL-like syntax - this produces more accurate results for templates that use modules.

SARIF and the GitHub Security tab

SARIF (Static Analysis Results Interchange Format) is the common language between security scanners and GitHub's Security tab. Every scanner in this post produces it. The upload action is always github/codeql-action/upload-sarif.

What the Security tab gives you:

A persistent list of open findings, tied to the file and line that introduced them
Grouping by rule, severity, and category - so you can see "all PUBLIC_ACL findings" without grepping through JSON
Automatic dismissal when the finding is no longer present (the scanner doesn't find it in the next run)
A history of when findings were introduced, dismissed, or reopened
Ability to manually dismiss findings with a reason (false positive, won't fix, risk accepted)

The SARIF upload deduplicates: if Checkov and tfsec both report the same misconfiguration, they appear as separate findings from separate tools, not as a single merged finding. This is useful - it confirms the misconfiguration is real when two independent scanners agree.

One important caveat: the Security tab with SARIF code scanning requires GitHub Advanced Security for private repositories. Public repositories get it for free. For private repos on GitHub Enterprise Cloud or GitHub Enterprise Server, GHAS must be enabled on the repository. If you're on a plan without GHAS and running private repos, the SARIF upload step will fail silently or produce a 403 - run it conditionally or accept that findings will only appear in workflow logs.

Break-on-Severity Thresholds

The default behavior of every scanner in this post is to fail on any finding. That's the wrong default for day-to-day CI. A Terraform module that's been in production for two years will have dozens of LOW and MEDIUM findings. Failing the build on all of them means every PR that touches infrastructure blocks until someone works through the entire backlog - which means developers start suppressing everything to get PRs to merge, which defeats the purpose.

The right threshold design:

HIGH and CRITICAL block the build. These are the misconfigurations with clear, exploitable attack paths: open security groups, unencrypted data stores, publicly accessible resources, missing authentication. They should never reach production without a deliberate decision.
MEDIUM goes to the Security tab, doesn't block. Real issues worth tracking and fixing in the next sprint, but not enough to halt a PR that's touching an unrelated module.
LOW is visible on request. Write it to SARIF so it appears in the Security tab if someone goes looking, but don't let it block anything.

Checkov threshold configuration:

# In checkov-action
soft_fail_on: LOW,MEDIUM
# or via .checkov.yaml
soft-fail-on:
  - LOW
  - MEDIUM

tfsec threshold configuration:

tfsec terraform/ --minimum-severity HIGH

PSRule doesn't have a native severity threshold in the same sense - its outcome parameter controls which result types cause a non-zero exit. For severity-based control, use a PSRule suppression group (covered in the next section) or a .ps-rule/Baseline.Rule.yaml to exclude specific rule severities.

The key principle: the threshold is a policy decision, not a tool decision. Write it down in your team's contributing guide, not just in the workflow YAML. When someone asks "why is this HIGH finding blocking my PR?", the answer should be in a document, not lost in a commit comment on the workflow file.

Handling False Positives

False positives are inevitable. A scanner sees a public S3 bucket and flags it. The bucket is your static website's CDN origin - public by design. The right response is a suppression with a reason, not a suppression without one.

The key principle: every suppression must include a reason, and the reason must be visible in the PR diff. A suppression comment without a reason is indistinguishable from "I didn't want to fix this." A suppression that's buried in a config file and not visible in the code is easy to miss in review.

Checkov Inline Suppression

Add a comment to the Terraform resource:

resource "aws_s3_bucket" "website" {
  bucket = "my-public-website-assets"
  #checkov:skip=CKV_AWS_20:Public read is intentional - this is the website CDN origin
  #checkov:skip=CKV2_AWS_65:ACL configuration managed separately via aws_s3_bucket_acl
}

The format is #checkov:skip=<CHECK_ID>:<reason>. The reason is not optional in the sense that Checkov will accept the suppression without it - it's optional in the sense that your code review should reject it without one.

tfsec Inline Suppression

resource "aws_security_group_rule" "allow_ssh" {
  type        = "ingress"
  from_port   = 22
  to_port     = 22
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"] #tfsec:ignore:AVD-AWS-0107:Bastion host - SSH locked down at network ACL level
}

The format is #tfsec:ignore:<AVD-ID>:<reason>. tfsec uses Aqua Vulnerability Database identifiers (AVD-*) in newer versions; older rules use the aws-* prefix. Both forms work.

PSRule Suppression

PSRule suppressions live in .ps-rule/suppress.Rule.yaml:

# .ps-rule/suppress.Rule.yaml
---
apiVersion: github.com/microsoft/PSRule/v1
kind: SuppressionGroup
metadata:
  name: SuppressedStorageRules
spec:
  rule:
    - name: Azure.Storage.BlobPublicAccess
  if:
    name: '.'
    equals: storage/website-assets.bicep
  reason: >
    Public blob access is intentional. This storage account hosts static website assets
    served via Azure CDN. Access is read-only and content is not sensitive.

This suppression is file-scoped - it only suppresses the rule for storage/website-assets.bicep, not for all storage accounts in the repository. File-scoped suppressions are preferable to rule-wide suppressions because they fail safely: if a new storage account is added to a different file, it still gets scanned.

Full Workflow Example

Here is a complete pull_request workflow that runs all three scanners, uploads SARIF for everything, blocks on HIGH/CRITICAL, and only runs IaC jobs when IaC files actually changed:

name: IaC Security Scan

on:
  pull_request:
    paths:
      - 'terraform/**'
      - 'bicep/**'
      - '**.tf'
      - '**.bicep'

permissions:
  contents: read
  security-events: write

jobs:
  checkov:
    name: Checkov (Terraform)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Checkov
        uses: bridgecrewio/checkov-action@v12
        with:
          directory: terraform/
          output_format: sarif
          output_file_path: checkov-results.sarif
          soft_fail_on: LOW,MEDIUM

      - name: Upload Checkov SARIF
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: checkov-results.sarif
          category: checkov

  tfsec:
    name: tfsec (Terraform)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tfsec
        uses: aquasecurity/tfsec-action@v1.0.0
        with:
          working_directory: terraform/
          format: sarif
          sarif_file: tfsec-results.sarif
          minimum_severity: HIGH
          soft_fail: true

      - name: Upload tfsec SARIF
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: tfsec-results.sarif
          category: tfsec

  psrule:
    name: PSRule (Bicep)
    runs-on: ubuntu-latest
    if: >
      contains(github.event.pull_request.changed_files, '.bicep') ||
      github.event_name == 'push'
    steps:
      - uses: actions/checkout@v4

      - name: Run PSRule for Azure
        uses: microsoft/ps-rule@v2
        with:
          modules: PSRule.Rules.Azure
          inputPath: bicep/
          outputFormat: Sarif
          outputPath: psrule-results.sarif
          outcome: Fail

      - name: Upload PSRule SARIF
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: psrule-results.sarif
          category: psrule

A few notes on this workflow:

The paths filter on the trigger means the workflow only runs when IaC files change. A PR that touches only application code doesn't pay the scan cost. This is important for monorepos where infrastructure and application code coexist - you don't want every frontend PR waiting for a Terraform scan.

The permissions block is at the workflow level and applies to all jobs. security-events: write is required for SARIF upload. contents: read is the minimum for checkout. If your workflow also needs to comment on PRs or create check annotations, add pull-requests: write and checks: write as needed - but only those.

The checkov and tfsec jobs run in parallel. Both upload to the Security tab with different category values. The build fails if either job fails (both are required jobs with no continue-on-error).

The psrule job has a conditional that tries to detect whether .bicep files changed. Note that github.event.pull_request.changed_files is not a reliable API for this - a more robust approach uses dorny/paths-filter or a dedicated detection step. For a repository where Bicep files are in a known directory, the paths filter on the trigger is sufficient and the per-job conditional is redundant; I've included it to show the pattern.

Closing

IaC is code. The same process that prevents application bugs from reaching production - write a test, run it in CI, block the merge if it fails - applies to infrastructure misconfigurations. An S3 bucket that's accidentally public is a bug. A storage account without encryption is a bug. They show up in the codebase before they show up in the cloud; that's when they're cheapest to fix.

The CI integration shown here is a one-time setup. After that, every PR that touches Terraform or Bicep gets scanned automatically. Findings appear in the Security tab with file and line context. Suppressions are inline, with reasons, visible in PR diffs. HIGH and CRITICAL issues block merges. Everything else is tracked for remediation without stopping work.

The alternative is catching the same issues post-deployment, in a CSPM alert, after the resource has been sitting exposed for however long it took someone to notice. That cost comparison isn't close.

Questions or corrections? Reach out.

steve.kaschimer@slalom.com