From Receipt Photo to Logged Expense: How AI Does It

You take a photo of a receipt. A few seconds later, the expense appears in your tracker with the merchant name, date, total, and category filled in. Magic.

Except it is not magic, it is a pipeline with several stages, each of which can fail. Understanding the stages explains why receipt scanning works flawlessly on a printed CVS receipt and falls apart on a crumpled Thai restaurant check.

The five stages of receipt processing

Stage 1: Image preprocessing. Before any text can be extracted, the image needs to be usable. This involves correcting for skew (the receipt is tilted), adjusting contrast, handling shadows, and identifying the receipt boundaries in the photo. Bad lighting, extreme tilt, or a background that blends with the receipt all degrade quality here.

Stage 2: OCR (Optical Character Recognition). The preprocessed image is converted to text. OCR reads characters pixel by pixel and produces a raw text string. This stage works well on machine-printed text and poorly on handwriting. It also fails on faded receipts, thermal paper that has greyed out, and receipts with water damage.

Stage 3: Entity extraction. The raw OCR text is parsed for specific fields: merchant name (usually at the top), date (various formats), subtotal, tax, tip, total, and individual line items if needed. This requires understanding receipt layouts, which vary widely.

Stage 4: Categorization. The merchant name is matched to a spending category. A grocery store receipt goes to Groceries. A restaurant receipt goes to Dining Out. This stage has the same accuracy issues as any automatic categorization.

Stage 5: Record creation. The extracted fields are assembled into an expense record. If confidence is high, it is created automatically. If confidence is low, the app surfaces it for your review.

Why results vary

The difference between "works every time" and "fails often" comes down to receipt quality and the model's training data.

Receipt quality: Thermal paper receipts (the shiny ones) are the most common failure point. They fade, crinkle, and are sensitive to heat and light. A freshly printed thermal receipt scans well. A three-week-old one in your pocket might not.

Layout diversity: Every merchant formats receipts differently. The model was trained on a sample of receipt types. Receipts outside that training distribution - regional chain formats, international receipts, hand-written notes - perform worse.

Ambiguous totals: Receipts with multiple totals (subtotal, tax, service charge, total after discount, tip) are harder to parse correctly. The model has to identify the right field as the amount to log.

What good receipt scanning looks like

The best receipt scanning apps:
- Process in 2-3 seconds (not 15-20)
- Handle perspective correction automatically (you do not have to hold the camera perfectly)
- Show you the extracted fields before saving, so you can correct mistakes
- Handle common receipt types accurately without training
- Work offline or with minimal data

The review step before saving is the most important. Even the best scanning systems make occasional errors. Apps that write directly to your expense record without showing you the extraction are optimizing for speed at the cost of accuracy.

When to use photo vs typing

Photo scanning is best for:
- Longer receipts with multiple items you want to capture
- Business expense receipts you need to keep for records
- Situations where you have the receipt in hand but have not logged yet

Typing is better for:
- Simple single-item purchases where you know the amount
- When you do not have the receipt (card payment, digital purchase)
- Faster total-only logging when items do not matter

DrakeAI supports both. Photo receipt scanning extracts merchant, date, and total. Text input handles everything else. For most day-to-day spending, typing is faster. For receipt documentation, photo works.

Bottom line

Receipt scanning is a genuinely useful feature when it works. Understanding its limitations - and using the review step before saving - makes it reliable rather than frustrating.

Try DrakeAI free on Android - iOS coming soon.