add image classification, type and account fields, support mixed expense types

This commit is contained in:
Connor Rhodes 2026-04-29 02:11:32 +00:00
parent bdb86c6c43
commit e3939965d5

View file

@ -1,11 +1,11 @@
---
name: log-work-expense
description: Log work expense receipts and mileage into MongoDB. Use when the user sends receipt images or odometer photos along with context about the expense, or says things like "log this expense", "record this receipt", "add this to my work expenses", "expense entry", "mileage for XYZ", or provides receipt/mileage details for work reimbursement tracking.
description: Log work expenses into MongoDB. Use when the user sends receipt images or odometer photos along with context, or says things like "log this expense", "record this receipt", "mileage for XYZ", "meals for XYZ", or provides any expense details for work reimbursement tracking. Triggers on any message that includes expense context (account, type, images of receipts or odometers).
---
# Log Work Expense
Log receipt images and mileage into the `wip.work_expenses` collection in MongoDB.
Log work expenses into the `wip.work_expenses` collection in MongoDB.
## Trigger Phrases
@ -14,41 +14,42 @@ Log receipt images and mileage into the `wip.work_expenses` collection in MongoD
- "add this to my work expenses"
- "expense entry"
- "mileage for XYZ"
- "meals for XYZ"
## Steps
1. **Upload images to S2** — Upload any images the user provided to S2 and collect the URLs.
1. **Classify images** — If the user sends multiple images of different types (e.g. receipts AND odometer photos), use the vision workaround from TOOLS.md to classify each image as either `"receipt"` or `"odometer"`. This is the ONLY reason to analyze images — to determine which type they are so they get grouped correctly. Do not extract amounts, dates, or other details from images.
2. **Insert into MongoDB** — Insert a document into `wip.work_expenses` with this structure:
2. **Determine type and account from the user's message** — The user will tell you the expense type (e.g. "mileage", "meals", "mileage and meals") and the account (e.g. "LTISD"). If either is unclear, ask. Do not infer type or account from images.
3. **Upload images to S2** — Upload all images to S2 and collect the URLs.
4. **Insert into MongoDB** — Group images by type and insert documents:
**Single receipt:**
```json
{
"files": ["https://s2.connorrhodes.com/agent/{sha256}.{ext}"],
"date": "YYYY-MM-DD",
"note": "Context the user provided",
"type": "meal",
"account": "LTISD",
"note": "Context from user's message",
"status": "todo"
}
```
**Mileage (multiple odometer photos):**
```json
{
"files": ["https://s2.connorrhodes.com/agent/{sha2561}.jpg", "https://s2.connorrhodes.com/agent/{sha2562}.jpg"],
"date": "YYYY-MM-DD",
"note": "Context the user provided about the trip",
"status": "todo"
}
```
- **files**: Array of S2 URLs. 1 item for meal receipts, 2 items for mileage (start/end odometer).
- **date**: Today's date, formatted as YYYY-MM-DD.
- **type**: Expense type from the user's message (e.g. `"meal"`, `"mileage"`).
- **account**: Account from the user's message (e.g. `"LTISD"`).
- **note**: The context the user provided (e.g. "meals during LTISD CBR onsite").
- **status**: Always `"todo"`.
- **files**: Array of S2 URLs. One entry for receipts, two entries for mileage (start and end odometer).
- **date**: Always use today's date (the date you receive the message), formatted as YYYY-MM-DD.
- **note**: What the user told you about the expense or trip. If they didn't provide context, ask.
- **status**: Always set to `"todo"` when inserting.
If the user sends both mileage and meals, create two separate entries — one for each type.
3. **Confirm** — Reply with a brief confirmation showing what was logged (date, note, and file links).
5. **Confirm** — Reply with a brief summary of what was logged.
## Notes
- Use `uv run --with pymongo` for MongoDB scripts.
- Use the S2 upload endpoint from TOOLS.md for file uploads.
- Use the vision workaround from TOOLS.md for image classification only.