cki_tools.gitlab_repo_config

Automated GitLab repository and group configuration management

Overview

The gitlab_repo_config tool provides automated GitLab repository and group configuration management through declarative YAML configuration files.

The tool operates by reading YAML configuration files that describe the desired state of GitLab resources, comparing this against the current state via GitLab API calls, and optionally applying necessary changes to bring repositories and groups into the desired state.

Usage

CLI Interface

usage: gitlab_repo_config.py [-h] [--fix] [--project-url PROJECT_URL]
                             [--group-url GROUP_URL] [--config CONFIG]
                             [--config-path CONFIG_PATH]

options:
  -h, --help            show this help message and exit
  --fix                 Fix detected problems
  --project-url PROJECT_URL
                        Only run the checker for one project
  --group-url GROUP_URL
                        Only run the checker for one group
  --config CONFIG       Path to the config file
  --config-path CONFIG_PATH
                        Path to the config file

Operation Modes

Check Mode (default): Validates current GitLab configuration against desired state and reports discrepancies without making changes.
Fix Mode (--fix): Automatically applies necessary changes to bring GitLab resources into compliance with the desired configuration.

Configuration Files

General configuration file structure

The configuration file defines settings for GitLab groups and projects using a hierarchical YAML structure based on a top-level mapping. Mapping keys follow the format human-readable-group-prefix/suffix, with only the suffix being meaningful:

Group Settings (suffix group): Configure GitLab group properties, member access, and LDAP/SAML integration:

# Group configuration for https://gitlab.com/example-org/
org/group:
  url: https://gitlab.com
  group_path: example-org
  group:
    description: 'Example organization'
    project_creation_level: maintainer
    visibility: public

Project Settings (suffix equals the project path): Individual repository configurations including features, access controls, and CI/CD settings:

# Project configuration for https://gitlab.com/example-org/example-project/
org/example-project:
  url: https://gitlab.com
  group_path: example-org
  project:
    issues_access_level: enabled
    merge_method: merge
    visibility: public

Project Defaults (suffix default): Apply configuration templates to all projects directly within a group automatically:

# Applied to all direct projects in the group https://gitlab.com/example-org/
org/default:
  group_path: example-org
  project:
    only_allow_merge_if_pipeline_succeeds: true
    remove_source_branch_after_merge: true

Ignore Flag: Skip processing for specific projects:
```
org/example-project:
  ignore: true
```

Inheritance and Composition

On top of the YAML-provided features like anchors, a basic inheritance mechanism is provided:

Default Template (.default): Default configuration recursively applied to top-level mapping values:

.default:
  url: https://gitlab.com
  project:
    default_branch: main
    merge_method: merge
    only_allow_merge_if_pipeline_succeeds: true

Mixins: Reusable configuration fragments for common patterns, starting with a dot:

.org_project:
  group_path: example-org

.private_mixin:
  project:
    builds_access_level: private
    visibility: private

.security_mixin:
  protectedbranches:
    '*':
      code_owner_approval_required: true
      merge_access_levels: [access_level: 40]

.webhook_mixin:
  webhooks:
    "https://webhook.example.com/gitlab":
      token: "{{ SHARED_WEBHOOK_SECRET }}"
      push_events: true
      merge_requests_events: true
      pipeline_events: true

Composition (.extends): Inherit from multiple configuration templates/mixins on top of the .default template:

org/example-project:
  .extends: [.org_project, .private_mixin, .security_mixin, .webhook_mixin]
  # Inherits base settings, group membership, private visibility, security controls and webhook config
  project:
    description: "Some internal project"

GitLab API Coverage

The tool integrates with the applicable GitLab APIs to manage the repository and group settings. For detailed parameter information, refer to the GitLab API documentation.

Repository Management

Projects API: Repository visibility, feature toggles, merge settings, and general project configuration:

org/example-project:
  project:
    description: "Example project demonstrating project API settings"
    issues_access_level: enabled
    merge_requests_access_level: enabled
    visibility: public

Archive a project: Archive or unarchive a project. GitLab does not accept archived on project update (PUT). --fix uses the archive and unarchive endpoints. Set archived at the top level, not under project::
```
org/example-project:
  archived: true
```
This requires owner access to the project.

Protected Branches API: Branch protection rules, push/merge access levels, and force push controls:

org/example-project:
  protectedbranches:
    main:
      merge_access_levels: [access_level: 30]  # developer level
      push_access_levels: [access_level: 0]    # not allowed
      allow_force_push: false
      code_owner_approval_required: true
    'feature/*':
      merge_access_levels: [access_level: 30]
      push_access_levels: [user_ids: [1234, 5678]]
      allow_force_push: true

Protected Tags API: Tag creation permissions and access controls:

org/example-project:
  protectedtags:
    '*':
      create_access_levels: [access_level: 40]  # Maintainer level
    'v*':
      create_access_levels: [access_level: 50]  # Owner level

Remote Mirrors API: Repository mirroring from upstream sources:

org/example-project:
  mirror:
    url: "https://github.com/example/upstream-repo.git"
    username: "mirror-user"
    password: "mirror-token"

Project Forks API: Fork relationship configuration:

org/example-project:
  forked_from_project: "upstream-org/upstream-project"

Group Management

Groups API: Group settings and LDAP/SAML group linking:

org/group:
  group:
    description: "Example organization"
    project_creation_level: maintainer
    require_two_factor_authentication: true
    shared_runners_setting: enabled
    subgroup_creation_level: owner
    visibility: public
  ldap_group_links:
    developers:
      group_access: 30
      provider: ldapmain
    admins:
      group_access: 50
      provider: ldapmain
  saml_group_links:
    gitlab-developers:
      access_level: 30
    gitlab-admins:
      access_level: 50

Access Control

Share a project with a group: Invited groups listed as shared_with_groups. GitLab does not accept this list on project or group update (PUT). --fix uses the share and unshare endpoints. Set shared_with_groups at the top level, not under project::
```
org/example-project:
  shared_with_groups:
    - group_id: 123
      group_name: Trusted Group
      group_full_path: trusted-group
      group_access_level: 30  # Developer
      expires_at: null
```
This requires owner access to the project.

Merge Request Approvals API: Merge request approvals and associated rules:

org/example-project:
  approvals:
    approvals_before_merge: 2
    reset_approvals_on_push: false
    merge_requests_author_approval: false
  approvalrules:
    "All Members":
      rule_type: any_approver
      approvals_required: 1
    "Security Team":
      rule_type: regular
      approvals_required: 1
      user_ids: [123, 456]

Push Rules API: Repository push restrictions like file size limits, and commit message validation:

org/example-project:
  pushrules:
    prevent_secrets: true
    commit_message_regex: '^(feat|fix|docs|style|refactor|test|chore):'
    branch_name_regex: '^(main|develop|feature/.+|hotfix/.+)$'
    file_name_regex: '\.envrc$'
    max_file_size: 100  # MB

CI/CD Integration

Project-level CI/CD Variables API: Project and group-level CI/CD variable management:

org/example-project:
  variables:
    DATABASE_URL: "postgres://localhost/myapp"
    API_TOKEN:
      value: "secret-token"
      environment_scope: production
      masked: true
      protected: true

Pipeline Schedules API: Automated pipeline triggers with cron-based scheduling:

org/example-project:
  pipeline_schedules:
    "Daily Build":
      description: "Run daily builds"
      ref: refs/heads/main
      cron: "0 2 * * *"
      active: true
      variables:
        BUILD_TYPE: daily
        ENVIRONMENT: staging
        API_TOKEN:
          value: "secret-token"
          variable_type: file

Protected Environments API: Environment-specific deployment access controls:

org/example-project:
  protectedenvironments:
    production:
      deploy_access_levels: [user_ids: [123, 456]]
    staging:
      deploy_access_levels: [access_level: 30]  # Developer level

CI/CD Job Token Scope API: Control which external projects and groups may use their CI_JOB_TOKEN to access a project. When job_token_scope is present, the inbound allowlist restriction is implicitly enforced. Both sub-keys are optional; a missing or empty sub-key means the list should be empty:
```
org/example-project:
  job_token_scope:
    allowlist:
      - other-group/project-a
      - other-group/project-b
    groups_allowlist:
      - trusted-group
```

Project Hooks API and Group Hooks API: Webhook configuration for automated notifications on GitLab events:

# Project-level webhooks
org/example-project:
  webhooks:
    "https://webhook.example.com/gitlab":
      url: "https://webhook.example.com/gitlab"
      token: "{{ WEBHOOK_SECRET }}"
      push_events: true
      merge_requests_events: true
      pipeline_events: true
    "https://external-system.webhook.com/gitlab":
      .ignore: true  # Ignore existing webhook - don't manage it

# Group-level webhooks (includes subgroup_events)
org/group:
  webhooks:
    "https://webhook.example.com/group":
      url: "https://webhook.example.com/group"
      token: "{{ GROUP_WEBHOOK_SECRET }}"
      push_events: true
      merge_requests_events: true
      subgroup_events: true  # Only available for group webhooks
      pipeline_events: true

Feature Flags API: Declarative management of feature flag existence. Only the flag name and description are managed; operational state (active toggle, strategies, rollout parameters, and environment scopes) is left to out-of-band tooling or the GitLab UI.

⚠️ Unlike labels, feature flags are fully reconciled: flags not listed in config are deleted when --fix is active. Use feature_flags: {} with caution — it will remove all existing feature flags from the project.
```
org/example-project:
  feature_flags:
    my_feature:
      description: "Enables the new checkout flow"
    gradual_rollout:
      description: "Gradual rollout of feature X"
    user_beta: {}  # no description
```

Labels API: Declarative management of project and group labels. Labels listed in config are created or updated; unlisted labels are left untouched (conservative mode). Supports scoped labels (names containing :: for mutual exclusion within a scope):

# Group-level labels (available to all projects in the group)
org/group:
  labels:
    "managed-by::workqueue":
      color: "#428bca"
      description: "MR managed by the workqueue-service"

# Project-level labels
org/example-project:
  labels:
    "priority::high":
      color: "#ff0000"
      description: "High priority items"
    "status::review":
      color: "#00ff00"
    simple-label: {}  # default color #428bca, no description

color defaults to #428bca (blue) when omitted; required by GitLab for label creation
description is optional
Omitting the labels key entirely skips the check; labels: {} means “no labels to manage” (existing labels are not deleted)

Resource Cleanup & Retention

Branch and Tag Cleanup: Policies for notifications about unexpected branches and tags - they are never removed even with --fix:
```
org/example-project:
  branches: {}  # only keep the default branch
  tags:
    'v.*':  # regex fullmatch()
```

Environment Management: Environment retention policies:

org/example-project:
  environments:
    keep:
      - production/.*  # regex fullmatch()
      - staging/.*
    expire_days: 30

Container Registry Cleanup: Container image cleanup policies:

org/example-project:
  container_registry:
    default:
      'latest': {}
      'mr-\d+':   # regex fullmatch()
        keep_n: 10
        keep_younger: 1w