GPTLint Rule Informal Spec

About

The GPTLint Rule Spec (abbreviated GRS in this doc) is an attempt to define a standard for how to describe higher-level lint rules that can be enforced across codebases using LLM-based tools compatible with GPTLint.

Example Rule

Here is an example markdown rule with yaml frontmatter:

---
fixable: false
tags: [best practices]
languages: [javascript, typescript]
---
 
# Example Rule
 
Plain-text description of the rule's intent.
 
### Bad
 
optional example code blocks showing the rule being used incorrectly
 
### Good
 
optional example code blocks showing the rule being used correctly

This rule would canonically be stored in .gptlint/example-rule.md and have the name example-rule.

Rule Format

GRS rules are parsed into the following TypeScript format:

export type Rule = {
  // core rule definition
  name: string
  title: string
  description?: string
  positiveExamples?: RuleExample[]
  negativeExamples?: RuleExample[]
  scope: LintRuleScope // defaults to 'file'
  level: LintRuleLevel // defaults to 'error'
 
  // optional metadata
  fixable?: boolean
  languages?: string[]
  tags?: string[]
  eslint?: string[]
  include?: string[]
  exclude?: string[]
  resources?: string[]
  model?: string
  gritql?: string
  gritqlNumLinesContext?: number
 
  // optional custom functionality for rules scoped to the file-level
  preProcessFile?: PreProcessFileFn
  processFile?: ProcessFileFn
  postProcessFile?: PostProcessFileFn
 
  // optional custom functionality for rules scoped to the project-level
  preProcessProject?: PreProcessProjectFn
  processProject?: ProcessProjectFn
  postProcessProject?: PostProcessProjectFn
}
 
export type LintRuleScope = 'file' | 'project' | 'repo'
export type LintRuleLevel = 'error' | 'warn' | 'off'

See the full types for more details.

Rule File Format

A GRS rule is defined in a GitHub Flavored Markdown (GFM) document. Each GRS rule must have its own markdown file, and GRS markdown files may only contain a single rule.

• GRS rule files must contain a single markdown h1 header containing the rule’s title property.
• Within this h1 section, GRS rule files may optionally contain a metadata table for customizing the rule’s behavior.
• The content from the h1 section up until any optional example header sections will comprise the rule’s description property which is intended to explain the rule’s intent in natural language.
• The rule’s name will default to the rule’s filename (without the .md extension).
  • This can be overridden via the frontmatter metadata’s name value.
  • If no valid name is found, it will fall back to a slugified version of the rule’s title value (main h1).
• GRS rules may optionally contain a single markdown h3 header named “Bad” or “Incorrect” or “Fail”.
  • The content of this section should contain 1 or more code blocks to use as negativeExamples.
• GRS rules may optionally contain a single markdown h3 header named “Good” or “Correct” or “Pass”.
  • The content of this section should contain 1 or more code blocks to use as positiveExamples.
• It is encouraged to specify a language for all code block examples, but it is not required.
• Rules with file scope may optionally contain a single grit or gritql code block. If provided, it must contain a valid gritql pattern which will be used to filter source files to only include matching portions before being processed by the linting engine.
  • Non-file-scoped rules must not contain a gritql pattern.
  • The gritql pattern will be assumed to be JS/TS for now.

Rule Frontmatter Metadata

A GRS file may optionally contain YAML frontmatter for specifying metadata. Note that this is the same frontmatter format used by GitHub, so the metadata renders as an inline table on GitHub (example).

All metadata fields are optional.

Here is a breakdown of the supported metadata fields and their expected types.

All metadata keys are case-sensitive, and all metadata values must match their expected types if present.

Example Rules

Documentation for all built-in rules is available here and their source is available here.

Parsing-wise, fixtures/valid-rules contains valid rules which test different parts of the spec, and fixtures/invalid-rules contains invalid rules which violate the spec and should fail parsing.