create initial scraping prototype

This commit is contained in:
2024-12-19 00:02:48 -07:00
commit c4521af5c2
8 changed files with 1620 additions and 0 deletions

11
todo.txt Normal file
View File

@ -0,0 +1,11 @@
[ ] Obtain embeddings for chunks of the bills
[ ] store the chunk and embeddings in a turso DB with associated metadata for linking to the docs and updates
[ ] see if summarizing bills is useful or loses too much info to be useful
[ ] test semantic searches for relevant info
[ ] figure out how to format and render diffs for the various versions of the docs a la git diff style
[ ] profiles for users should contain enough info to allow me to build a prompt that would find relevant info for the person
[ ] allow the user to enter their own prompts
[ ] add periodic checking for updates
[ ] add notifications to users when they are subscribed to a certain search