2489 Commits

Author SHA1 Message Date
Paul Frazee
63b9873bb1
Lexicon rework (#379)
* Implement types and validation for new NIH lexicon DSL

* Update xprc and xrpc-server to use new NIH lexicon DSL

* Simplification: dont allow lexicon definitions to be refs or ref arrays

* Add 'unknown' primitive type to NIH lexicon

* Implement client codegen for NIH Lexicon

* Implement server codegen for NIH Lexicon

* Add the 'datetime' primitive type to the NIH Lexicon

* Rework lexicons to use new NIH Lexicon DSL

* Readd tokens to code generation

* Update all code to use new NIH Lexicon (all tests passing!)

* Add imports from external files to lex-cli codegen

* Do not allow records, queries, or procedures outside of the main def

* Update packages/lex-cli/src/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Linter pass

* Add knownValue output to codegen

* Add comments to codegen

* Use enum and const in codegen

* Rework lexicon 'properties' structure

* Treat falsy query params as undefined

Co-authored-by: devin ivy <devinivy@gmail.com>

* Remove now-redundant param presence validation

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-12-01 12:26:18 -06:00
devin ivy
645a2ca5f7
Enforce max payload size for PLC and PDS (#376)
* Sketching out max payload size in xrpc-server

* Fixes and testing for external payload parsing

* Apply max json payload limit to pds and plc

* Move parsing middleware into xrpc-server w/ options
2022-11-29 13:02:03 -05:00
Paul Frazee
47c69f5469
Update getPostThread to handle missing parents (#373) 2022-11-28 21:35:40 -06:00
Paul Frazee
ebf8e5c9ed
Add NotFoundError to getPostThread (#375) 2022-11-28 21:33:32 -06:00
devin ivy
134595e26e
Standardize limit schema across lexicons (#377) 2022-11-28 21:34:55 -05:00
devin ivy
9d29ef96f6
Well-defined pagination in PDS (#365)
* Implement tuple keyset cursor to break pagination ties, apply to feeds and assertions

* Tuple keyset cursor on rest of views

* Tidy keysets for postgres

* Add default implementation to Keyset to remove some boilerplate

* Simplify Keyset interface

* Remove dead code

* Move to new Keyset interface, incl. TimeCidKeyset

* Make pds assertion view tests deterministic

* Apply pds keyset interface to user search
2022-11-28 11:51:27 -05:00
devin ivy
fd359bada0
Use integers in lexicons (#367)
Use json schema integers rather than numbers where appropriate in lexicons
2022-11-23 16:13:37 -05:00
Daniel Holmgren
083f53078a
Fix a few MST bugs (#364)
* fix MST stage & getPointer operations

* test

* comments

* fix pointer issue on higher level splits
2022-11-22 14:37:38 -06:00
Daniel Holmgren
6fd50bd46c
Wait for servers to close at end of test suite (#362)
wait for servers to close
2022-11-18 15:47:03 -06:00
Daniel Holmgren
0f2562419b
Return post ancestors (#361)
* recursively get post ancestors

* tests
2022-11-17 17:06:44 -06:00
Daniel Holmgren
40458a6249
Fix unclear indexedAt in processor (#360)
fix unclear indexedAt in processor
2022-11-17 15:36:05 -06:00
Daniel Holmgren
dafeeee4af
MessageQueue fix (#358)
* keep message queue from getting behind

* better error hadnling that doesnt crash sanity loop

* 3 -> 1 min

* timeout + destory on mq

* destory mq on db close
2022-11-17 15:34:18 -06:00
Daniel Holmgren
1abe9e7e0d
Remove Level (#357)
remove level & move did-web server to tests
2022-11-17 15:22:33 -06:00
Daniel Holmgren
53d8209c9c
Don't give notifications for mentions & replies to self (#359)
Dont give notifciations for metnions & replies to self
2022-11-17 15:22:14 -06:00
Daniel Holmgren
227b85608e
Deduping indexing (#354)
* record processor

* moving records to new processor

* plugins finished

* hook up to db

* migration

* yay tests working

* Tx migration (#355)

* migrations in txs

* fix tx issue

* testing

* test deduping

* test assertions

* rm types on migrations

* dont do migrations in txs, fixes pg issue
2022-11-17 14:10:37 -06:00
devin ivy
4fa2110553
Use public hostname in invite codes (#353) 2022-11-17 12:03:17 -05:00
Daniel Holmgren
c207947bd5
Stagable blockstore (#351)
* stagable blockstore + remove mutable repo

* pds wip

* nearly there

* bugfix

* assertion ordering

* del old code

* save staged if no callback

* removing level

* pr fixup
2022-11-17 09:51:36 -06:00
devin ivy
13aead171c
Support shallow builds and sourcemaps (#352)
* Enable all modules to be built shallowly, externalizing their deps

* Setup shallow builds to modify package.json main

* Hoist update-main-to-dist to separate call, fix lex-cli and plc shallow builds

* Generate sourcemaps during build

* Tidy

* Hoist new build deps up to root
2022-11-15 19:24:58 -05:00
Paul Frazee
e02671ba82
Give 'HandleNotAvailable' error on reserved handles (#349) 2022-11-14 15:32:04 -06:00
Daniel Holmgren
43788cdc5a
Show own reposts in timeline views (#348)
show own reposts in timelines
2022-11-14 13:42:12 -06:00
Paul Frazee
60071e0770
Handle updateProfile when no record exists (#343)
* WIP handle updateProfile when no record exists

* create new profile in updateProfile

Co-authored-by: dholms <dtholmgren@gmail.com>
2022-11-11 14:10:56 -06:00
devin ivy
5b4e2b9ee3
Fix test snapshots post-merge (#342) 2022-11-11 14:59:29 -05:00
Daniel Holmgren
2d947be9c2
Allow updating scene profiles (#341)
allow updating scene profiles
2022-11-11 13:36:36 -06:00
Paul Frazee
a1bee406e2
Add app.bsky.graph.getAssertions() (#340)
* Add app.bsky.graph.getAssertions()

* Update getAssertions to support query by author or subject

* Revert accidental change to members test
2022-11-11 12:38:25 -06:00
Daniel Holmgren
962041e6c1
Scene trending (#327)
* wip

* views

* trending schema

* starting message queue

* scene processor

* wip

* send mq messages from db

* db events

* undo screwing up codegen lol

* setup queue

* db migrations

* fixing up message processing

* div by 0 check

* tx issue

* queue use cursor

* update not insert

* sql bugfix + tests

* trying to linearize tests

* correclty serializing txs

* attempt update before insert

* log errors

* handle gaps in cursor

* cleanup

* oops reenable test

* correctly incr cursor
2022-11-11 12:29:48 -06:00
Paul Frazee
8e548af023
Add missing execute() to confirmation delete (#339) 2022-11-10 16:51:26 -06:00
devin ivy
8ab87ff1f7
Standard actor declarations in app.bsky (#338)
* Include scene trends in feeds

* Tidy

* Tidy

* Sweep app.bsky schemas for standard actor declarations, remove reference from com.atproto

* Replace declarationCid on follow, assertion, confirmation records

* Sweep pds for standard app.bsky actor declarations

* Fix dev-env

* Tidy after merge
2022-11-10 15:46:16 -05:00
devin ivy
c52b9f0771
Include scene trends in feeds (#335)
* Include scene trends in feeds

* Tidy

* Tidy
2022-11-10 14:26:57 -05:00
Paul Frazee
7431e65f2a
Fix codegen signature for the record delete methods (#336) 2022-11-10 12:26:32 -06:00
Daniel Holmgren
e6471a7419
Trend schemas (#333)
trend schemas
2022-11-09 14:44:46 -06:00
devin ivy
da8f055816
Align XRPC parameters with subset of json schema (#328)
* Model procedure for setting a vote on a post

* Implement app.bsky.feed.setVote() method

* Support json schema subset for parameters in lexicon and lex-cli

* Update lexicons w/ json schema parameters

* Use enum in getVotes parameters

* Support json schema subset for parameters in xrpc-server, ajv validation

* Include parameter enums in generated markdown

* Tidy

* Unify xrpc parameter md docs with inputs and outputs: show ts interfaces

* Tidy
2022-11-09 15:39:47 -05:00
devin ivy
1bd8ef1811
Implement setVote() method (#325)
* Model procedure for setting a vote on a post

* Implement app.bsky.feed.setVote() method

* Ensure setVote() no-ops when already in correct state
2022-11-09 15:24:35 -05:00
Paul Frazee
e1150465ab
Add 'member' to getProfile state output, and add some dev-env mock data (#332)
* Add a scene to the dev-env mock data

* Add 'member' to myState output of getProfile

* Await db writes in dev env mock data

Co-authored-by: devin ivy <devinivy@gmail.com>

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-11-09 14:09:16 -06:00
Daniel Holmgren
8b689de071
Reenable account tests + quick fix (#326)
reenable + quick fix
2022-11-08 16:12:11 -06:00
Daniel Holmgren
58634c47d7
Scene membership (#322)
* schemas

* routes

* bugfix

* tests

* test fixups

* delcaration in member views

* sort notif output
2022-11-08 14:22:16 -06:00
Daniel Holmgren
0d98291c42
Indexing rework (#317)
* remove unnecessary db plugin methods

* plugin types & remove declaration table

* assertion things

* wrap up things

* simplify table names
2022-11-08 12:14:15 -06:00
devin ivy
b117ae4568
Move from likes to votes (#319)
* Model votes in lexicon, remove likes

* Model votes in db, setup notifs, fix schema

* Update pds logic, move likes to votes

* Codegen

* Update tests for votes

* Test vote filter by direction

* Fix dev-env
2022-11-08 11:02:41 -05:00
Paul Frazee
cd1b64e600
Add app.bsky.actor.getSuggestions (#318)
* Fix some lexicon filenames

* Add app.bsky.actor.getSuggestions
2022-11-08 09:40:52 -06:00
Alberto Roura
1a32d5664f
Added full stop to descriptions (#312)
Added full stop to description

Done for consistency across files and for potential future documentation.
2022-11-08 07:58:03 -06:00
Daniel Holmgren
b530a13d09
Scene creation & membership (#304)
* scene creation w unique handles

* wip

* user_dids -> did_handle

* invites -> assertion/confirmation

* do all the ops on scene create

* start tests + bug fixin

* user can make requests on behalf of scene

* accepting invites

* fix schema names

* get scene profiles

* return actor type in getProfile

* scene views test

* notifications test

* patch up a few db things

* rework handle & email noramlization

* codegen createscene errors
2022-11-07 14:33:33 -06:00
Paul Frazee
70c525f806
Rework post text slices to be objects (#311) 2022-11-04 18:26:33 -05:00
Paul Frazee
6053031d0d
A couple improvements to the markdown generator to improve clarity (#310) 2022-11-04 15:28:00 -05:00
devin ivy
d6a189f59a
Healthchecks for PDS and PLC (#309)
* Add plc healthcheck, unify plc server interface with pds

* Add pds healthcheck

* Make plc test util file consistent with other packages

* Fix types in dev-env

* Type fixes

* Tidy

* Rename healthcheck routes, place pds's under /xrpc
2022-11-04 16:26:56 -04:00
Paul Frazee
e61127111a
Split lexicons into more specific namespaces (#306)
* Segment lexicons into more specific namespaces and tighten up some naming conventions

* Update API to use new lexicons

* Update pds and dev-env to use new lexicons
2022-11-04 12:59:00 -05:00
Daniel Holmgren
7ee821312a
Handle package (#305)
* handle package

* forgot a couple things

* few more fixes
2022-11-03 17:55:33 -05:00
Daniel Holmgren
7fdbb4dbb5
Username -> Handle (#303)
* schemas

* db

* atproto xrpc

* bsky xrpc

* got er done
2022-11-03 11:35:21 -05:00
devin ivy
f87cb10d36
Add subject to password reset email (#302) 2022-11-02 20:35:56 -04:00
devin ivy
a42d98b280
Allow duplicate block by the same did w/ sql blockstore (#301)
* Allow duplicate block by the same did w/ sql blockstore

* Typo
2022-11-02 20:27:41 -04:00
Paul Frazee
9289ce15c6
Rename getHomeFeed to getTimeline to improve some semantic clarity (#300) 2022-11-02 16:52:25 -05:00
Paul Frazee
c986066e4c
Update XRPC procedures to use request body by default (#297)
* Update lexicons of xprc procedures to avoid using query params when possible

* Update code generated by lex-cli to reduce param counts

* Update all API usages to new signatures

* Fix readme

* Remove params from postNotifiactionsSeen lexicon schema
2022-11-02 16:37:58 -05:00