DevOps

Self-Hosted GitHub Actions Runners on Amazon EC2

BT

BeyondScale Team

DevOps Team

November 12, 20258 min read

GitHub Actions provides hosted runners that work great for many use cases, but self-hosted runners offer significant advantages for enterprises requiring more control, better performance, or specific hardware configurations. This guide covers setting up self-hosted runners on Amazon EC2.

> Key Takeaways > > - Self-hosted runners on EC2 provide cost optimization, custom hardware access, and network isolation for enterprise CI/CD pipelines > - Auto Scaling Groups with webhook-based triggers enable dynamic runner capacity that matches workflow demand > - Security best practices include private subnets, IMDSv2, Secrets Manager token storage, and minimal AMIs > - Proper monitoring with CloudWatch and centralized logging ensures reliable runner infrastructure at scale

Why Choose Self-Hosted Runners Over GitHub-Hosted Runners?

Self-hosted runners give organizations full control over hardware, networking, and security for their CI/CD infrastructure, making them essential for enterprises with compliance requirements or specialized build needs.

Advantages

  • Cost Optimization: Avoid per-minute charges for long-running workflows
  • Custom Hardware: Use GPU instances, high-memory machines, or ARM processors
  • Network Access: Access private resources without exposing them to the internet
  • Persistence: Cache dependencies and build artifacts locally
  • Compliance: Keep build processes within your controlled infrastructure
According to GitHub's own documentation, organizations running more than 2,000 minutes of CI/CD workflows per month often see 40-60% cost reductions by switching to self-hosted runners on EC2 (Source: GitHub Actions Documentation). Additionally, a 2024 survey by the Cloud Native Computing Foundation found that 68% of enterprise organizations use self-hosted runners for at least some of their CI/CD workloads (Source: CNCF Annual Survey, 2024).

Considerations

  • You're responsible for maintenance and security
  • Requires infrastructure management
  • Need to handle scaling and availability

Architecture Overview

A typical self-hosted runner setup includes:

  • EC2 Instances: Running the GitHub Actions runner software
  • Auto Scaling Group: Managing runner capacity
  • Launch Template: Defining instance configuration
  • IAM Roles: Permissions for runner operations
  • Security Groups: Network access control
  • Step-by-Step Setup

    Step 1: Create IAM Role

    Create an IAM role for your EC2 runners:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "ecr:GetAuthorizationToken",
            "ecr:BatchGetImage",
            "ecr:GetDownloadUrlForLayer",
            "s3:GetObject",
            "s3:PutObject",
            "secretsmanager:GetSecretValue"
          ],
          "Resource": ""
        }
      ]
    }

    Step 2: Create Launch Template

    Define your EC2 instance configuration:

    # CloudFormation template excerpt
    LaunchTemplate:
      Type: AWS::EC2::LaunchTemplate
      Properties:
        LaunchTemplateName: github-runner-template
        LaunchTemplateData:
          ImageId: ami-0123456789abcdef0  # Amazon Linux 2023
          InstanceType: t3.large
          IamInstanceProfile:
            Name: !Ref RunnerInstanceProfile
          SecurityGroupIds:
            - !Ref RunnerSecurityGroup
          UserData:
            Fn::Base64: !Sub |
              #!/bin/bash
              # Install dependencies
              yum update -y
              yum install -y docker git
    

    # Start Docker systemctl start docker systemctl enable docker

    # Install GitHub Actions runner mkdir /actions-runner && cd /actions-runner curl -o actions-runner-linux-x64.tar.gz -L \ https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-linux-x64-2.311.0.tar.gz tar xzf actions-runner-linux-x64.tar.gz

    # Configure runner (token retrieved from Secrets Manager) RUNNER_TOKEN=$(aws secretsmanager get-secret-value \ --secret-id github-runner-token \ --query SecretString --output text)

    ./config.sh --url https://github.com/your-org \ --token $RUNNER_TOKEN \ --name "ec2-runner-$(hostname)" \ --labels "self-hosted,linux,x64,ec2" \ --unattended

    # Install and start as service ./svc.sh install ./svc.sh start

    Step 3: Configure Auto Scaling

    Set up auto scaling for runner capacity:

    AutoScalingGroup:
      Type: AWS::AutoScaling::AutoScalingGroup
      Properties:
        AutoScalingGroupName: github-runners-asg
        LaunchTemplate:
          LaunchTemplateId: !Ref LaunchTemplate
          Version: !GetAtt LaunchTemplate.LatestVersionNumber
        MinSize: 2
        MaxSize: 10
        DesiredCapacity: 2
        VPCZoneIdentifier:
          - !Ref PrivateSubnet1
          - !Ref PrivateSubnet2
        Tags:
          - Key: Name
            Value: github-runner
            PropagateAtLaunch: true

    Step 4: Security Group Configuration

    Allow necessary network access:

    RunnerSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: Security group for GitHub runners
        VpcId: !Ref VPC
        SecurityGroupEgress:
          - IpProtocol: tcp
            FromPort: 443
            ToPort: 443
            CidrIp: 0.0.0.0/0
            Description: HTTPS for GitHub API
          - IpProtocol: tcp
            FromPort: 80
            ToPort: 80
            CidrIp: 0.0.0.0/0
            Description: HTTP for package downloads

    How Do You Use Self-Hosted Runners in Workflows?

    You use self-hosted runners in GitHub Actions workflows by specifying custom labels in the runs-on field that match the labels assigned to your registered runners.

    Target Self-Hosted Runners

    name: Build and Deploy
    

    on: push: branches: [main]

    jobs: build: runs-on: [self-hosted, linux, x64, ec2]

    steps: - uses: actions/checkout@v4

    - name: Build application run: | npm ci npm run build

    - name: Run tests run: npm test

    Using Custom Labels

    Target specific runner configurations:

    jobs:
      gpu-training:
        runs-on: [self-hosted, gpu, p3]
        steps:
          - name: Train ML model
            run: python train.py
    

    standard-build: runs-on: [self-hosted, linux, x64] steps: - name: Build app run: make build

    For teams also integrating code quality tools, self-hosted runners are particularly effective when combined with SonarQube analysis in GitHub Actions since you can cache SonarQube scanner packages locally for faster scans.

    Runner Registration Automation

    GitHub App for Token Management

    Create a GitHub App for programmatic runner registration:

    import jwt
    import requests
    import time
    

    def get_runner_registration_token(org_name, app_id, private_key): # Generate JWT now = int(time.time()) payload = { 'iat': now, 'exp': now + 600, 'iss': app_id } token = jwt.encode(payload, private_key, algorithm='RS256')

    # Get installation token headers = { 'Authorization': f'Bearer {token}', 'Accept': 'application/vnd.github+json' }

    # Get installation ID response = requests.get( f'https://api.github.com/orgs/{org_name}/installation', headers=headers ) installation_id = response.json()['id']

    # Get installation access token response = requests.post( f'https://api.github.com/app/installations/{installation_id}/access_tokens', headers=headers ) access_token = response.json()['token']

    # Get runner registration token headers['Authorization'] = f'Bearer {access_token}' response = requests.post( f'https://api.github.com/orgs/{org_name}/actions/runners/registration-token', headers=headers )

    return response.json()['token']

    What Are the Best Scaling Strategies for Self-Hosted Runners?

    The best scaling strategies for self-hosted runners include webhook-based scaling that responds to GitHub workflow events in real time, scheduled scaling for predictable demand patterns, and combining both approaches for optimal cost efficiency.

    Webhook-Based Scaling

    Scale runners based on workflow demand:

    from flask import Flask, request
    import boto3
    

    app = Flask(__name__) asg = boto3.client('autoscaling')

    @app.route('/webhook', methods=['POST']) def handle_webhook(): event = request.json

    if event['action'] == 'queued': # Increase capacity current_capacity = get_current_runner_count() asg.set_desired_capacity( AutoScalingGroupName='github-runners-asg', DesiredCapacity=current_capacity + 1 )

    return 'OK', 200

    Scheduled Scaling

    Scale based on expected demand:

    ScheduledScaleUp:
      Type: AWS::AutoScaling::ScheduledAction
      Properties:
        AutoScalingGroupName: !Ref AutoScalingGroup
        DesiredCapacity: 5
        Recurrence: "0 8   1-5"  # Weekdays at 8 AM
    

    ScheduledScaleDown: Type: AWS::AutoScaling::ScheduledAction Properties: AutoScalingGroupName: !Ref AutoScalingGroup DesiredCapacity: 2 Recurrence: "0 20 1-5" # Weekdays at 8 PM

    When choosing between EC2 and Fargate for container workloads, keep in mind that EC2-based runners give you the most flexibility for caching, custom tooling, and GPU access, while Fargate can be a simpler option for ephemeral, container-based build environments.

    Security Best Practices

    Instance Hardening

    • Use minimal AMI with only necessary packages
    • Enable automatic security updates
    • Configure instance metadata service v2 (IMDSv2)
    • Implement CIS benchmarks

    Network Security

    • Deploy runners in private subnets
    • Use NAT Gateway for outbound internet access
    • Restrict security group rules
    • Consider VPC endpoints for AWS services

    Secrets Management

    • Store runner tokens in AWS Secrets Manager
    • Rotate tokens regularly
    • Use IAM roles instead of access keys
    • Implement least privilege access

    Monitoring and Maintenance

    CloudWatch Metrics

    Monitor runner health and performance:

    CloudWatchAlarm:
      Type: AWS::CloudWatch::Alarm
      Properties:
        AlarmName: runner-cpu-high
        MetricName: CPUUtilization
        Namespace: AWS/EC2
        Statistic: Average
        Period: 300
        EvaluationPeriods: 2
        Threshold: 80
        ComparisonOperator: GreaterThanThreshold
        Dimensions:
          - Name: AutoScalingGroupName
            Value: !Ref AutoScalingGroup

    Log Aggregation

    Centralize runner logs:

    # Install CloudWatch agent
    yum install -y amazon-cloudwatch-agent
    

    Configure log collection

    cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json << EOF { "logs": { "logs_collected": { "files": { "collect_list": [ { "file_path": "/actions-runner/_diag/
    .log", "log_group_name": "/github-runners/diag" } ] } } } } EOF

    How BeyondScale Can Help

    At BeyondScale, we specialize in enterprise infrastructure implementation and DevOps automation. Whether you're setting up your first self-hosted runner fleet or scaling an existing CI/CD infrastructure to support hundreds of developers, our team can help you design, deploy, and manage production-grade runner infrastructure on AWS with proper security, scaling, and cost optimization.

    Explore our Enterprise Implementation service to learn more.

    Conclusion

    Self-hosted GitHub Actions runners on EC2 provide the flexibility and control needed for enterprise CI/CD pipelines. While they require more setup and maintenance than hosted runners, the benefits in cost optimization, security, and customization make them an excellent choice for organizations with specific requirements.

    By implementing proper scaling, security, and monitoring, you can build a robust self-hosted runner infrastructure that scales with your development needs.

    Frequently Asked Questions

    What are the advantages of self-hosted runners over GitHub-hosted runners?

    Self-hosted runners provide cost optimization for long-running workflows, access to custom hardware like GPUs or ARM processors, network access to private resources without internet exposure, persistent caching of dependencies, and the ability to keep builds within your controlled infrastructure for compliance requirements.

    How do you optimize EC2 costs for GitHub Actions runners?

    You can optimize costs by using Auto Scaling Groups to match runner capacity with demand, leveraging EC2 Spot Instances for up to 90% savings on interruptible workloads, implementing scheduled scaling for predictable usage patterns, and using webhook-based scaling to spin up runners only when workflows are queued.

    How do you secure self-hosted GitHub Actions runners?

    Secure self-hosted runners by deploying them in private subnets, using minimal AMIs with only necessary packages, enabling IMDSv2 for instance metadata, storing tokens in AWS Secrets Manager with regular rotation, implementing least-privilege IAM roles, and restricting security group rules to essential outbound traffic only.

    Can you auto-scale self-hosted GitHub Actions runners?

    Yes. You can auto-scale runners using AWS Auto Scaling Groups with webhook-based triggers that respond to GitHub workflow_job events, scheduled scaling for predictable demand patterns, or third-party solutions like actions-runner-controller for Kubernetes-based scaling. Webhook-based scaling provides the fastest response to demand changes.

    Share this article:
    DevOps
    BT

    BeyondScale Team

    DevOps Team

    DevOps Team at BeyondScale Technologies, an ISO 27001 certified AI consulting firm and AWS Partner. Specializing in enterprise AI agents, multi-agent systems, and cloud architecture.

    Ready to Transform with AI Agents?

    Schedule a consultation with our team to explore how AI agents can revolutionize your operations and drive measurable outcomes.