How to Parse SEC EDGAR Filings with Python

Why Parse EDGAR Directly

The SEC's EDGAR database is the canonical source for corporate filings: 10-K, 10-Q, 8-K, S-1, and hundreds of other document types. If you're building a systematic trading signal, conducting risk analysis, or just want raw data without vendor markup, you need to know how to fetch and parse these filings yourself.

Most quant shops and financial data vendors ingest EDGAR programmatically. The API is free, well-documented, and public. You don't need an institutional account. This post walks through a production-grade pipeline using standard Python libraries.

Setting Up Your Environment

Start with the essentials. You'll need requests for HTTP calls, BeautifulSoup4 for HTML parsing, and Polars for tabular manipulation. If you prefer pandas, that works too, but Polars is faster for large filing batches.

pip install requests beautifulsoup4 polars lxml

We'll also use the time module to respect SEC rate limits (10 requests per second per the User-Agent header rule).

Finding the Correct CIK and Document

Every public company in EDGAR has a Central Index Key (CIK). The SEC publishes a company tickers dataset. You can fetch it or hardcode common ones for testing.

Example: AAPL has CIK 0000320193. Microsoft is 0000789019. You'll often see them zero-padded to 10 digits.

Once you have the CIK, construct the filing index URL:

https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK={cik}&type=10-K&dateb=&owner=exclude&count=100

This returns an HTML page listing all 10-K filings for that company. You can filter by type (10-K, 10-Q, 8-K, etc.) and date range.

Fetching the Filing Index with Requests

Write a function to grab the index page and extract filing metadata:

import requests
from bs4 import BeautifulSoup
import time

headers = {
    'User-Agent': 'MyBot 1.0 (contact@mycompany.com)'
}

def fetch_filings(cik, form_type='10-K'):
    url = f'https://www.sec.gov/cgi-bin/browse-edgar'
    params = {
        'action': 'getcompany',
        'CIK': cik,
        'type': form_type,
        'owner': 'exclude',
        'count': 100
    }
    time.sleep(0.1)  # Respect rate limits
    resp = requests.get(url, params=params, headers=headers)
    resp.raise_for_status()
    return resp.text

html = fetch_filings('320193', '10-K')
soup = BeautifulSoup(html, 'lxml')

The response is HTML. You'll need to parse the table that lists accession numbers, dates, and filing sizes. The SEC's structure is stable, but not semantic (no JSON endpoint for this particular index).

Extracting Accession Numbers

The filing index table contains rows with links to each document. Each row has a unique accession number (e.g., "0000320193-23-000077"). Extract them:

import re

def extract_accessions(html):
    soup = BeautifulSoup(html, 'lxml')
    rows = soup.find_all('tr')
    accessions = []

    for row in rows:
        cells = row.find_all('td')
        if len(cells) >= 2:
            link = cells[1].find('a')
            if link and 'href' in link.attrs:
                href = link['href']
                match = re.search(r'(\d{10}-\d{2}-\d{6})', href)
                if match:
                    accessions.append(match.group(1))

    return accessions

accessions = extract_accessions(html)
print(accessions[:3])
# ['0000320193-23-000077', '0000320193-22-000105', ...]

Accessing the Filing Document

Once you have an accession number, construct the path to the filing's index page:

cik = '320193'
accession = '0000320193-23-000077'
acc_clean = accession.replace('-', '')  # Remove dashes
filing_url = f'https://www.sec.gov/cgi-bin/viewer?action=view&cik={cik}&accession_number={accession}&xbrl_type=v'

This gives you a viewer, but for raw data you want the raw filing document. The SEC hosts the main document and exhibits under a predictable path:

base_url = f'https://www.sec.gov/Archives/edgar/{cik}/{acc_clean}/{accession}-index.htm'

Fetch this index and extract the main document name (usually ends in .htm or .xml for XBRL filings):

def get_filing_text(cik, accession):
    acc_clean = accession.replace('-', '')
    index_url = f'https://www.sec.gov/Archives/edgar/{cik}/{acc_clean}/{accession}-index.htm'

    time.sleep(0.1)
    resp = requests.get(index_url, headers=headers)
    resp.raise_for_status()

    soup = BeautifulSoup(resp.text, 'lxml')

    # The filing table lists documents. Find the 10-K document.
    rows = soup.find_all('tr')
    for row in rows:
        cells = row.find_all('td')
        if len(cells) >= 4:
            doc_name = cells[3].get_text(strip=True) if len(cells) > 3 else ''
            if '10-K' in doc_name or '.htm' in doc_name.lower():
                link = cells[4].find('a') if len(cells) > 4 else None
                if link:
                    doc_path = link.get('href', '')
                    filing_url = f'https://www.sec.gov/Archives/edgar/{cik}/{acc_clean}/{doc_path}'

                    time.sleep(0.1)
                    doc_resp = requests.get(filing_url, headers=headers)
                    doc_resp.raise_for_status()
                    return doc_resp.text

    return None

filing_text = get_filing_text('320193', '0000320193-23-000077')
print(filing_text[:500])

Parsing Document Structure

10-K filings are HTML or XBRL+HTML hybrids. For Item-level extraction, look for Item headers. The SEC uses consistent Item numbering per Regulation S-K:

• Item 1: Business
• Item 1A: Risk Factors (Item 105 in Form S-1)
• Item 7: Financial Statements and Supplementary Data
• Item 8: Changes in and Disagreements with Accountants

Parse by regex or semantic HTML:

import re

def extract_item(filing_text, item_num):
    pattern = rf'(?i)Item\s+{item_num}[:\s](.+?)(?=Item\s+\d+:|$)'
    match = re.search(pattern, filing_text, re.DOTALL)
    if match:
        return match.group(1).strip()[:5000]  # First 5000 chars
    return None

business_section = extract_item(filing_text, 1)
print(business_section[:200])

This is lossy but functional. For production, use SEC's XBRL instance documents (filed as .xml) which have proper semantic markup:

def get_xbrl_filing(cik, accession):
    acc_clean = accession.replace('-', '')
    # XBRL instance usually named like {accession}.xml
    xbrl_url = f'https://www.sec.gov/Archives/edgar/{cik}/{acc_clean}/{accession}.xml'

    time.sleep(0.1)
    resp = requests.get(xbrl_url, headers=headers)
    if resp.status_code == 200:
        return resp.text
    return None

xbrl_text = get_xbrl_filing('320193', '0000320193-23-000077')

Extracting Financial Data with Polars

For structured data (balance sheet, income statement), parse the embedded tables or use the XBRL directly. Here's a toy example with Polars:

import polars as pl
from bs4 import BeautifulSoup

def extract_tables(filing_text):
    soup = BeautifulSoup(filing_text, 'lxml')
    tables = soup.find_all('table')

    data_frames = []
    for table in tables:
        rows = table.find_all('tr')
        if len(rows) < 2:
            continue

        headers = [td.get_text(strip=True) for td in rows[0].find_all(['th', 'td'])]
        data = []

        for row in rows[1:]:
            cells = [td.get_text(strip=True) for td in row.find_all(['td', 'th'])]
            if len(cells) == len(headers):
                data.append(cells)

        if data:
            df = pl.DataFrame(dict(zip(headers, zip(*data))))
            data_frames.append(df)

    return data_frames

tables = extract_tables(filing_text)
if tables:
    print(tables[0].head())

Convert numeric strings to floats for analysis:

df = tables[0].with_columns(
    pl.col('*').str.replace_all(r'[\$,()]', '').cast(pl.Float64, strict=False)
)

Handling Edge Cases and Rate Limits

The SEC rate-limits by IP and User-Agent. Supply a descriptive User-Agent string with your contact info. Respect a 10-request/second ceiling. For bulk pulls, add exponential backoff on 429 (Too Many Requests) or 503 (Service Unavailable):

import time
import random

def fetch_with_backoff(url, max_retries=3):
    for attempt in range(max_retries):
        try:
            time.sleep(0.1 + random.uniform(0, 0.05))
            resp = requests.get(url, headers=headers, timeout=10)
            resp.raise_for_status()
            return resp
        except requests.exceptions.HTTPError as e:
            if resp.status_code in [429, 503]:
                wait_time = 2 ** attempt
                print(f'Rate limited, waiting {wait_time}s')
                time.sleep(wait_time)
            else:
                raise
    return None

Putting It Together

A complete workflow:

cik = '320193'  # AAPL
accessions = extract_accessions(fetch_filings(cik, '10-K'))

for acc in accessions[:1]:
    text = get_filing_text(cik, acc)
    if text:
        business = extract_item(text, 1)
        tables = extract_tables(text)
        print(f'Filing {acc}: {len(business)} chars, {len(tables)} tables')

Many teams use tools like FilingFirehose to avoid the plumbing, but understanding the raw pipeline means you control parsing logic, caching, and error handling. You'll spot filing structure variations early and adapt your schema accordingly.

This approach scales to hundreds of companies and years of history. Monitor your rate-limit headers and log accession numbers for debugging. Happy filing hunting.