Commit Graph

2335 Commits

Author SHA1 Message Date
Daniel Holmgren 21ff052da9 Dont reexport CID type (#277)
dont reexport cid
2022-10-26 23:13:50 -05:00
Daniel Holmgren 9b66f20abb Kms keypair (#270)
* scaffold package

* kms working!

* interface

* normlize for low s

* no der encoding in crypto

* comments

* rm kms tests

* pr feedback
2022-10-26 23:13:36 -05:00
Daniel Holmgren e9428647c5 Fix a couple build issues (#275)
* build issues

* hoist better-sqlite3 to monorepo root
2022-10-26 16:32:10 -05:00
Paul Frazee 07a2bd117d Improve codegen quality (shorter typenames) (#269)
* Improve lex-cli's generated code (specifically type names)

* Update to use the shorter typenames

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2022-10-26 13:45:42 -05:00
devin ivy 4a65804db5 Miscellaneous dep and build fixes (#272)
Misc dep and build fixes
2022-10-26 14:19:14 -04:00
Daniel Holmgren e507d95dd2 Secp256k1 support (#267)
* add secp256k1 to crypto module

* integrate into plc

* use secp256k1 key in test

* use jwt alg consts & fix did resolution
2022-10-25 16:35:27 -05:00
Daniel Holmgren d6fd69d3a3 Rename pds (#266)
* clean up old todos

* some more cleanup

* rename server -> pds
2022-10-25 15:52:46 -05:00
Daniel Holmgren 7e6019fbda PDS cleanup (#265)
* clean up old todos

* some more cleanup

* remove server did for now
2022-10-25 15:36:44 -05:00
Daniel Holmgren 29463093c1 Recovery key (#263)
* clean up old todos

* recovery key in server cfg & allow user provided key

* tests & rename

* testing did creation

* recoveryKey in dev-env

* require recovery key
2022-10-25 15:25:44 -05:00
devin ivy e3c4efd7d9 PDS transactions (#245)
* Run getProfile in transaction, setup/test transaction helpers

* Test tidy

* Transactionalize invite code usage in acct creation, add transaction assertions

* Transactionalize account creation

* crud txns

* test txns for account

* cleanup

* lock table

* fixed up crud operations

* fixed updateProfile route

* test data races

* cleanup

* Transactional invite code use, username/email creation (#253)

* Remove table lock for invite code use, surface username race

* Add and use db constraint for unique account username/email

* Update packages/server/src/db/index.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/server/src/db/records/badgeAccept.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/server/src/db/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr fixups

* remove old todos

* test for updateProfile races

* logging on repo update errors

* give feedback on misuse of forUpdate

Co-authored-by: dholms <dtholmgren@gmail.com>
2022-10-24 15:59:40 -05:00
Daniel Holmgren a8368d148d Linting (#262)
* move eslint ignore to eslintrc

* dont verify codgen stuff

* require node 18
2022-10-24 14:48:43 -05:00
devin ivy 26b554fee2 Setup PDS server with migrations (#254)
* Setup PDS with kysely migrations

* Nix KyselyWithDialect from PLC, found it didn't work as intended in PDS

* Fix down migration in PDS

* Fix update profile on postgres

* Support postgres and migrations in PDS bin

* Lint
2022-10-24 15:27:30 -04:00
zhuowei 3d2ea06bb5 server: fix startup by adding better-sqlite3 to externals, create tables (#257)
* server: fix startup by adding better-sqlite3 to externals, create tables

* Lint

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-10-24 15:17:08 -04:00
devin ivy 000507e491 Setup PLC server with postgres and migrations (#249)
* Setup PLC with postgres

* Setup PLC with kysely migrations

* Add note to PLC migrations file
2022-10-21 09:47:18 -04:00
Daniel Holmgren 960a0fce1e Immutable repo structure (#247)
* immutable repo

* wrap mutable repo around immutable

* fixing up repo & structure

* added back in safeCommit

* fix up server

* return rebase cid on commit callback

* getReocrd on structure + a bit of cleanup

* pr cleanup
2022-10-20 11:11:51 -05:00
dholms 287fc40f48 Fix typo 2022-10-18 16:14:34 -05:00
Paul Frazee feca3622c1 Rename ADX -> ATP (#244)
* Rename ADX to ATP

* Remove old docs (now in atproto website repo)

* Update readme

* Update README.md

Co-authored-by: devin ivy <devinivy@gmail.com>

* A couple of @ -> AT

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-10-18 12:21:59 -05:00
Daniel Holmgren fd18da2cb5 Schema keyed hint (#243)
* update schemas for keyed

* codegen dict

* codgen record ids

* determine key type from schema

* allow non-tid keys

* rm tids from batch write schema

* use schema ids record in db layouts

* standardize on rkey

* doc update

* crud test

* repo tests for custom rkeys
2022-10-18 11:14:47 -05:00
devin ivy 0820013316 Unify pagination interface across all methods (#242)
* Update lexicons for cursor convention, fix minor cid usage in interface

* Liked by cursor pagination, temp fix for other tests

* Home and author feed cursor pagination

* Paginate crud list w/ cursor, minor adjustments to lex and xrpc

* Paginate notifications view w/ cursor

* Paginate reposts view w/ cursor

* Paginate follow views w/ cursor

* Paginate user search view w/ cursor

* Fix notification tests

* Make repoListRecords order desc by default for consistency

* Paginate badge memebers view w/ cursor
2022-10-17 18:19:37 -04:00
Daniel Holmgren 60a537a6e4 MIT license (#241)
mit license errything
2022-10-17 13:20:06 -05:00
Daniel Holmgren 83691da3f6 Repo metadata (#240)
* repo metadata

* tests & write meta to car file
2022-10-17 13:11:09 -05:00
Daniel Holmgren 3679719571 Badging utilities (#237)
* record type for accepted badge

* getBadgeMembers view

* updateProfile procedure

* better return type on updateProfile & gen client

* add badges to getProfile view

* testing & bug fixes

* tests for updating profile & linking badges

* errors

* badgeOffer record type

* getProfile view ensures badges are both offered & accepted

* getBadgeMembers test

* PR feedback

* cid check
2022-10-17 11:18:08 -05:00
devin ivy e314974d1b User search (#238)
* Add initial schemas for getUsersSearch and getUsersTypeahead

* Stub-out pds user search/typeahead, add pg indexes to support it

* Implement and test user typeahead method for postgres

* Implement user typeahead for sqlite

* Tidy

* Get user search view working, except pagination

* Get pagination working on pds user search w/ postgres

* Bail on jest snapshots for the conditional pg/sqlite search results

* Refactor user search queries out of user search method

* Apply user search helpers to typeahead

* Tidy user search implementation

* Test bad search input for sqlite

* Fix loading of pg_trgm extension to be friendly with the test suite

* Fix typo

* Query consistency

* All server tests can reference app

* Handle bad user search cursors
2022-10-14 16:16:32 -04:00
Paul Frazee 90b3792d90 Finalize lexicon reference behaviors, adding lex: URI references (#236)
* Update lexicon doc format to support .defs field and .descriptions within method bodies

* Add correct  resolution to lexicon, including to lex: URIs

* Update lexicons and generated code to use new  resolution
2022-10-13 14:08:09 -05:00
Daniel Holmgren 7a9d773e7e Postbuild typecheck (#235)
* typecheck postbuild

* rm space

Co-authored-by: devin ivy <devinivy@gmail.com>

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-10-12 20:03:43 -05:00
Daniel Holmgren bfaf3a86f9 Handling updates (#231)
* disabled udpates

* remove update tests

* update schemas

* crud operations

* working cids into db layer

* crud working

* views tests working & updated snapshots

* rm tests for likes on reposts

* fix up repo tests

* quick pr feedback

* recordKey -> rkey & nameOrDid -> user

* reference records by uri & cid obj

* fix dev-env
2022-10-12 18:36:40 -05:00
devin ivy 0b6bf87b66 Run PDS using postgres (#233)
* Setup dockerization of postgres db

* Fix typo

* Setup postgres dialect config in pds, fix migrations for pg

* Convert home and author feed to work on pg

* Convert pds follow views to work on pg

* Convert pds liked-by and notifications views to work on pg

* Unify bigint handling between sqlite and pg

* Convert pds account, crud, and profile, reposts, thread views to work on pg

* Ensure a more complete reset of containers/volumes in with-test-db.sh, set libpq env vars

* Add readme for dockerized postgres

* Fix build issues
2022-10-12 16:43:19 -04:00
Paul Frazee 45ddf23c58 Fixes to dev env (#234) 2022-10-12 15:07:31 -05:00
Paul Frazee 90cdb99bdd Update NSID semantics to final (#232)
* Improve lex-cli to remove files that no longer exist

* Update lex-cli to use new ns

* Rename lexicons: todo.adx to com.atproto, todo.social to app.bsky

* Improve lex-cli md generator
2022-10-12 14:32:18 -05:00
Daniel Holmgren 7f2dfb8d50 Switch TypeORM to Kysely (#229)
* kysely interfaces

* migrated record plugins

* db types + notifications

* index record emthods on db

* Convert author feed to kysely

* rest of db index

* Convert home feed to kysely

* Remove references to sqlite3 package

* Convert liked by view to kysely

* create tables

* Convert notif count view to kysely

* move scrypt to own file

* Fix typo

* Convert notifs view to kysely

* Convert post thread to kysely

* Convert profile view to kysely

* accounts tests passing!

* Convert reposted by view to kysely

* Convert followers view to kysely

* Convert follows view to kysely

* Convert notifs last seen to kysely

* Tidy

* Re-enable social methods

* Minor tidy of author and home feed queries

* Tidying and minor fixes for pds view queries

* fix up crud, disabled PUTs for now

* Fix profile view, use nulls in table interfaces

* Fix pds follow/follower views, remove special db date logic

* Fix pds notification count view

* Fix pds feed views, reposted by join

* Remove typeorm from server

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2022-10-11 13:39:11 -05:00
Paul Frazee 72add11740 Fix the external better-sqlite3 path in dev-env build (#230) 2022-10-10 19:05:50 -05:00
devin ivy ef69a725f0 Fix PLC server usage in dev-env (#228)
Fix PLC server usage in dev-env, ensure better-sqlite3 bindings are external
2022-10-10 18:58:24 -05:00
devin ivy af5daacb90 More comprehensive testing for PDS views, using snapshots (#223)
* Create basic seed for server based on view tests

* Utilize basic seed in view tests

* Split-up pds view tests

* Trying out snapshot testing for author feed

* Tidy

* Generalize normalization for snapshot testing

* Snapshot test home feed, fix inclusion of own reposts

* Fix typo

* Add follow snapshot tests, fix ordering and db date handling

* Add like snapshot tests, fix ordering and db date handling

* Add notification snapshot tests

* Add profile snapshot tests

* Add reposts snapshot tests, fix ordering and db date handling

* Add thread snapshot tests

* Fix typo

* Remove unneeded space
2022-10-10 11:38:57 -04:00
Daniel Holmgren 214e4a14bb Plc tests & cleanup (#226)
* ensure we understand the key type a user is rotating to

* race condition test

* handle username/pds with or without https://

* fix a quick types issue

* adding logging

* fix better-sqlite3 build issues

* bring did-resolver up to date w plc

* fix pds tests for plc
2022-10-10 09:50:49 -05:00
Daniel Holmgren 47c1e1f327 Use Kysely for PLC DB (#220)
* switching out typeorm for kysely

* refigure migrations
2022-10-07 17:16:23 -05:00
devin ivy 604f4d2ab2 Seeds for testing server views (#222)
* Create basic seed for server based on view tests

* Utilize basic seed in view tests

* Split-up pds view tests
2022-10-07 18:11:34 -04:00
Daniel Holmgren 92ab06941d Tests for did conversion (#225)
added tests for did conversion
2022-10-07 16:45:45 -05:00
devin ivy 5a9c0e9058 Fix dev-env build, account for server's hbs templates (#218)
* Fix dev-env build, account for server's hbs templates

* Remove out-of-date comments
2022-10-06 16:02:17 -04:00
devin ivy e30c45d38a Enforce single user per email, usernames case-insensitive (#217)
* Schemas and scaffolding for reset password methods

* Initial handler for todo.adx.requestAccountPasswordReset

* Initial handler for todo.adx.resetAccountPassword

* Implement server mailer

* Configure server for mailer and testing w/ mailer

* Test happy path of pass reset, fix reset bug

* Update lex to fix types bug for requestAccountPasswordReset

* Fix handlebars reference to config getters

* Test some negative password reset flows

* Minor cleanup to pass reset

* Tidy handlebars file with prettier, supporting double-quotes for html

* Fix esbuild of server for mailer templates, fix test issue

* Misc tidying for password reset

* Misc tidying for password reset

* Enforce single user per email, test unique email and username

* Remove resolved TODO re: duplicate emails
2022-10-06 14:28:20 -04:00
devin ivy 59f18ac707 Forgot password methods (#216)
* Schemas and scaffolding for reset password methods

* Initial handler for todo.adx.requestAccountPasswordReset

* Initial handler for todo.adx.resetAccountPassword

* Implement server mailer

* Configure server for mailer and testing w/ mailer

* Test happy path of pass reset, fix reset bug

* Update lex to fix types bug for requestAccountPasswordReset

* Fix handlebars reference to config getters

* Test some negative password reset flows

* Minor cleanup to pass reset

* Tidy handlebars file with prettier, supporting double-quotes for html

* Fix esbuild of server for mailer templates, fix test issue

* Misc tidying for password reset

* Misc tidying for password reset
2022-10-06 14:24:26 -04:00
dholms 904fa6448d SDK -> Resolver 2022-10-05 18:07:34 -05:00
Daniel Holmgren 7233ce070f Cleanup (#210)
* messing around w pino

* rework locals & add pino-http

* subsystem logging off env vars

* remove auth tokens from logs

* lots of logging

* re-enable tests

* update enabled check

* fix subsystem logging

* some quick cleanup
2022-10-05 17:55:08 -05:00
Daniel Holmgren b26eb75277 Logging (#209)
* messing around w pino

* rework locals & add pino-http

* subsystem logging off env vars

* remove auth tokens from logs

* lots of logging

* re-enable tests

* update enabled check

* fix subsystem logging

* pr fixups
2022-10-05 17:47:00 -05:00
devin ivy 685c656729 Fixes for handling of posts vs. reposts in feeds (#208)
* Fix ordering of reposts in feed

* Preserve separate posts and reposts in feeds

* Include users' own posts in their feed

* Remove unused dep from test file
2022-10-04 17:52:44 -04:00
Daniel Holmgren b3c60696da Invite codes (#204)
* implement invite codes

* siwtch to admin basic auth & tests

* signular InviteCodeUse for table name

* comment

* pr fixups

* tx TODOs
2022-10-04 15:13:55 -05:00
Paul Frazee 11928993dc Make the cursor field a required output in get*Feed (#206) 2022-10-04 12:28:49 -05:00
devin ivy d64e4f6a75 Support firehose feed algorithm (#205)
* Split getFeed() into getAuthorFeed() and getHomeFeed()

* Remove unused getFeed codegen file

* Unify queries for getHomeFeed and getAuthorFeed

* Factor out common query parts for building feeds

* Test firehose and default feed algorithm

* Fix api tsconfig reference
2022-10-04 12:18:34 -05:00
Paul Frazee c70911ab20 Fix: support multiple post entities per post (#203) 2022-10-04 09:29:07 -05:00
Paul Frazee 2cb0e8d36c Feed bugfixes (#202)
* Fix feed test

* Fixes to repost information in getFeed
2022-09-30 11:57:38 -05:00
Daniel Holmgren 8b0884a315 Notifications (#200)
* notification methods on tables

* process notifs

* last seen

* notifs xrpc methods

* seen notifs route

* register route

* notifications tests

* rm unused db method
2022-09-30 11:09:42 -05:00