2067 Commits

Author SHA1 Message Date
bnewbold
3be9c741ce
helpers for rkey and tid syntax; validate rkey at record creation time (#1738)
* syntax: fix jest config displayName

* syntax: TID validation

* syntax: add recordkey validation

* pds: verify rkey syntax at record creation time

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-12-01 14:28:47 -06:00
bnewbold
c17971a2d8
harden datetime verification (#1702)
* syntax: add datetime validator (and interop tests)

* syntax: improve datetime normalization

* lexicon: stronger datetime validation (from syntax package)

* syntax: make datetime syntax norm test more flexible

* make fmt

* datetime: docs, normalize and always variant

* bsky replace toSimplifiedISOSafe with normalizeDatetimeAlways

* more rigorous datetime parsing on record creation

* handle negative dates

* syntax: disallow datetimes before year 0010

* syntax: datetime normalization functions validate output

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-11-30 18:19:06 -06:00
Matthieu Sieben
691511b9b9
fix(debug): properly type debugCatch wrapper result (#1817) 2023-11-30 17:55:20 -06:00
Matthieu Sieben
2513bd1b26
style(xrpc-server): avoid un-neccessary "if" statement (#1826)
fix(xrpc-server): avoid un-neccessary "if" statement
2023-11-30 17:53:23 -06:00
Matthieu Sieben
2fc6ca54c1
perf(bsky): avoid re-creating auth functions on every request (#1822)
perf(bsky): avoid re-creating auth utilities on every request

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
2023-11-30 17:53:16 -06:00
dan
3c0ef382c1
Don't create unnecessary error objects (#1908)
* Don't create unnecessary error objects

* add changeset

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-11-30 17:53:04 -06:00
Mary
6da8182530
fix(pds): include aspectRatio on read-sticky posts (#1824)
* fix: include aspectRatio on read-sticky

* chore: add missing alt text test case
2023-11-30 17:51:07 -06:00
Daniel Holmgren
401538a933
Handle missing creator on lists and feed generators (#1906)
handle missing creator on lists and feed generators
2023-11-30 15:42:00 -06:00
Foysal Ahamed
4a5b65893d
Expose labels attached with legacy actions when events are queried and fix email event builder (#1905)
*  Expose labels attached with legacy actions when events are queried

* 🐛 Fix property name for modEventEmail

*  Only attach label vals for legacy actions if they exist

* 🐛 Set empty string as default subjectLine for email event

* 🧹 Cleaning up
2023-11-30 20:00:39 +01:00
Foysal Ahamed
1f9040a44d
Evented architecture for moderation system (#1617)
* 🚧 WIP with proposed lexicons for event based mod architecture

* 🚧 Remove unnecessary moderation action lexicon

* 🚧 Working on event based actions

*  Add escalated subject status

* 🐛 Alright, fixed the error in lexicon

* 🚧 Working through reversal

*  Cleanup build errors

*  Add subject status endpoint

*  Add handler

*  get reports from mod actions table

* :rightwards_twisted_arrows: Merge with upstream

* 🚧 Builds but test network doesnt start

*  Tests passing on event based status change

*  Rename index

* ♻️ Rename takeModerationAction->emitModerationEvent

*  Implement label reversal

*  Auto-revert test working

* ♻️  Refactored to event types and tests are passing

*  Add takedown event sequence validation

*  Adds support for blobCid status

* 🧹 Cleanup unnecessary method:

*  Hydrate handles with status and events

*  Re-implement auto reversal

*  Add takendown and mute filters

*  Allow filtering events by type

*  Allow filtering events by creator did

*  Add subjectStatus to record and repoview

*  Add persistent note feature

*  Log send email event

* 🐛 Fix logging send email event

*  Better type

*  Adjust migration to create separate moderation_event table

* 🧹 Cleanup types

*  Adjust tests with mod event emitter

*  Fix more tests around takedowns

*  Get test suite to pass

*  Get test suite to pass for pds

*  Get test suite to pass for pds

*  Update snapshot for feedgen

*  Why are more snapshots updating?

* ♻️ Rename getModerationEvents -> queryModerationEvents

* ♻️ Rename getModerationStatuses -> queryModerationStatuses

* ♻️ Rename persistNote->sticky

* 🐛 Rename subject

* ♻️ Cleanup expiresAt for scheduled actions

*  Add more tests, allow fetching mod history for all content by a user

*  Fix repo and record tests

*  Migrate reports and actions to events

* 🐛 Fix escalated status overwrite

*  Implement direct sql query to create events from actions and reports

* 🚧 Adding keyset pagination for subject statuses

*  Add migration for lastReportedAt

*  Migrate blob cids

*  Fix pagination on mod subject list endpoint

* 🐛 Fix blob actions

*  All tests passing on bsky package

*  Bring back snapshots

*  Skipping timeline test temporarily

*  Skipping some more tests to isolate failing ones

*  Bring back list-feed test

*  Bring back timeline test

*  Fix label action in seeding

*  Enable timeline proxied test

*  Enable search actor proxied test

*  Enable feedgen tests

*  Fix test for admin/get-record

*  Move note to comment for subject status

*  Accept comments in mute event

*  Remap flag event to ack event

* 🐛 Add legacyRef in report union selection

* @atproto/api 0.6.24-next.0

* @atproto/api 0.6.24-next.1

*  Adjust migration export and add index for blobCids column

*  Maintin action ids when migrating

*  Paginate events using createdAt timestamp

*  Update snapshot for pds test with events cursor update

*  Use only events for snapshot testing

*  Use only events for snapshot in the remaining test

* relative paths to lexicons for build

* fix bsky periodic event reversal in service entrypoint

*  Allow comments in takedown and label

*  Only import reports on consecutive run of the migration script

*  Adjust moderation property of blob entries

* determine latest reports to migrate

*  Process new reports for subject status

*  Process unresolved reports on first migration run

* fix transaction error, process just unresolved reports, make reported-at updates safe for reruns

* tidy

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-11-30 17:53:56 +01:00
Daniel Holmgren
7edad62c12
Put canReply state on post viewer state instead of thread viewer state (#1882)
* switch canReply from thread to post view

* tweaks & fix up tests

* update snaps

* fix more snaps

* hydrate feed items for getPosts & searchPosts

* fix another snapshot

* getPosts test

* canReply -> blockedByGate & DRY up code

* blockedByGate -> excludedByGate

* replyDisabled
2023-11-27 20:14:20 -06:00
devin ivy
95d33f7b11
Support unauthed usage of feeds (#1884)
* update local feedgens to not require a viewer where possible

* update getFeed to use optional auth

* test feeds w/ optional auth
2023-11-27 15:30:09 -05:00
devin ivy
f232371934
Provide proper error for updating to an email that's already in use (#1770)
provide proper error for updating to an email already in use
2023-11-21 18:09:07 -05:00
github-actions[bot]
b532c502af
Version packages (#1877)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-11-21 15:47:43 -06:00
Daniel Holmgren
e1b5f2537a
Prevent signature malleability (#1839)
* prevent signature malleability

* tidy & allow toggle

* fix in xrpc-server

* add changeset

* add test vecotrs
2023-11-21 15:30:40 -06:00
Emily Liu
59f70dbe80
Tweak lexicon descriptions (#1859)
* Tweak lexicon descriptions

* Tweak birthdate

* Update lexicons/app/bsky/actor/getProfiles.json

Co-authored-by: devin ivy <devinivy@gmail.com>

* Regenerate packages from lexicons

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-11-20 17:41:25 -05:00
devin ivy
c28754f681
Upgrade sharp on pds and appview (#1864)
upgrade sharp to 0.32.6
2023-11-16 18:21:57 -05:00
Daniel Holmgren
e6fb39dc84
Bugfix: fetchLabels (#1862)
* fix fetch labels

* tidy

* update cast to parseInt
2023-11-16 12:27:46 -06:00
Daniel Holmgren
b05130db68
Add temp.fetchLabels method (#1858)
* add temp.fetchLabels route

* update description
2023-11-16 11:07:52 -06:00
Daniel Holmgren
1ef6da191b
Pass along hitsTotal and cursor from searchPostsSkeleton (#1857)
pass along hitsTotal and cursor from searchPostsSkeleton
2023-11-15 15:31:14 -06:00
Daniel Holmgren
599cb449b5
Bugfix: don't pass the viewer as a param to searchPosts skeleton (#1855)
dont pass the viewer as a param to searchPosts skeleton
2023-11-14 22:18:47 -06:00
dan
681e35d4ef
Add a note to README about depending on jq and docker (#1854) 2023-11-14 22:37:15 -05:00
Daniel Holmgren
a434d586dd
Add searchPosts api to appview (#1845)
* proxy search posts to search service

* add search posts proxy

* tidy

* add type annotations
2023-11-14 18:14:22 -06:00
Daniel Holmgren
74b7fdf754
Randomize suggestions (#1844)
* randomize suggestions

* fix snap

* cursor fix

* pr feedback
2023-11-14 18:14:08 -06:00
Foysal Ahamed
697f5d3627
Expose emailConfirmedAt field from admin getRepo (#1757)
*  Expose emailConfirmedAt field from admin getRepo

* ♻️ Fix typing for repo result

* 🧹 Cleanup unnecessary import

*  Adapt to the new pds based get account info method

* 🧹 Cleanup unused pds util
2023-11-08 00:26:31 +01:00
devin ivy
bebc4bac8c
Simplify moderation fanout to configured url (#1804)
* simplify moderation fanout to configured url

* fix bsky mod fanout tests
2023-11-05 00:33:14 -04:00
Daniel Holmgren
a3d81dd911
Prevent thread loops (#1813)
* prevent thread loops

* include original uri

* tidy

* tweak

* last tweak

* last tweak i swear

* wording
2023-11-03 16:55:50 -05:00
Eric Bailey
bba9388ea9
Add a test for links that end in .php (#1810) 2023-11-02 17:29:27 -05:00
github-actions[bot]
b90f2c4ef8
Version packages (#1808)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-11-02 16:30:44 -04:00
devin ivy
772736a010
Add did doc output to getSession for session resumption (#1806)
* add optional did doc output to getSession lexicon

* add did doc output to getSession on pds, update client to use it

* api test fixes

* api changeset

* tidy
2023-11-02 16:16:26 -04:00
github-actions[bot]
3598898c03
Version packages (#1802)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-11-02 19:58:57 +00:00
dan
ce49743d7f
Add changeset for #1790 (#1801) 2023-11-02 19:52:36 +00:00
dan
b5a77babdb
Adjust types to use LexiconDoc (#1807) 2023-11-02 19:51:40 +00:00
bnewbold
84e2d4d2b6
Update atproto source license to MIT/Apache2 (#1788)
* update LICENSE files to MIT/Apache2

* README: update top-level license

* update package-level READMEs with license

* changeset: license update
2023-11-02 00:45:13 -07:00
Daniel Holmgren
cf848e87ab
Tweak schemas for entryway createAccount (#1797)
* tweak scheams

* require email & password
2023-11-01 14:56:57 -05:00
devin ivy
a161f815de
Make plc op unknown object rather than bytes (#1792)
make plc op unknown object rather than bytes
2023-11-01 11:54:00 -04:00
dan
cae59094c7
Don't validate known lexicons at runtime (#1790)
* Make lexicon validation DEV-only

* Apply code review suggestions
2023-11-01 15:42:37 +00:00
devin ivy
8637c367fe
Respect updated service auth keys (#1765)
* bust key cache when verifying service auth

* unit tests for xrpc auth

* fix

* support option for verifying non-low-s signatures

* fix verifyJwt tests
2023-10-31 18:09:02 -04:00
devin ivy
2df6f2b836
Support S3-compatible credentials for pds blobstore (#1787)
* support s3 credentials for pds blobstore

* tidy
2023-10-31 00:00:00 -04:00
YOCKOW
b28fdb2ca4
PDS: Allow configuring non-AWS S3 blob storage. (#1729)
* PDS: Allow configuring non-AWS S3 blob storage.

See https://github.com/bluesky-social/atproto/issues/1583

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-10-30 19:27:36 -04:00
Daniel Holmgren
fcb19c9c51
Simplify PDS moderation (#1723)
* spec out new simple pds mod routes

* introduce new admin state endpoints

* wire up routes

* clean up pds

* revoke refresh tokens

* getUserAccountInfo

* pr tidy

* fixing some tests

* fixing up more tests

* fanout takedowns to pds

* fanout admin reqs to pds

* tidy

* more tidy & add more pds moderation tests

* getUserAccountInfo -> getAccountInfo

* dont hydrate pds info on searchRepos

* fix build

* port admin tests to bsky package

* clean up old snaps

* tests on fanout

* tweak naming

* missed a rename

* tidy renames

* fix lex name

* tidy & move snap

* fix build

* cleanup repeat process all

* skip actor search test

* fix bday paradox

* tidy up pds service auth

* rm skipped test

* retry http

* tidy

* improve fanout error handling

* fix test

* return signing key in did-web

* more tests

* tidy serivce auth checks

* change takedownId col to takedownRef

* build branch

* fix bsky test

* add service key to indexer

* move signing key to api entry

* dont build
2023-10-30 16:56:17 -05:00
bnewbold
ec0dfdc8f5
lexicon: maximum report "reason" length of 1000 chars (graphemes) (#1171)
* lexicon: maximum report length of 500 chars (graphemes)

* lexicon: bump maximum report size to 1000 chars

* lexicon: bump max report size again to 2k graphemes

* make codegen
2023-10-30 09:28:42 -07:00
github-actions[bot]
ea04096b02
Version packages (#1774)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-10-27 14:51:50 -04:00
Paul Frazee
9c98a5baaf
Modlist updates: add "blockingByList" information and some utility functions to the sdk (#1779)
* Add blockingByList to app.bsky.actor.defs#viewerState

* Add blocking-by-list behaviors to moderation sdk

* Add modlist helper functions to bsky-agent

* codegen

* hydrate blockingByList in profile viewer state

* ignore self-mutes and self-blocks in read path

* format

* changeset

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-10-27 14:35:36 -04:00
Devin Ivy
35d108ce94 bump changeset 2023-10-27 12:40:02 -04:00
Daniel Holmgren
22622dcdb2
Lengthen view maintainer interval (#1775)
* lengthen view maintainer interval

* dont build branch
2023-10-26 18:37:33 -05:00
Devin Ivy
0587ad1dbb remove services and dev-env from changeset 2023-10-26 18:35:59 -04:00
devin ivy
46b108cb86
Facilitate authing w/ PDS based on DID doc (#1727)
* lexicon for did doc w/ auth credentials

* include did doc w/ session when configured. configure on dev-env.

* Add dynamic PDS URL adoption to the client

* remove usage of did doc field from getsession in client

* dry-up did doc type and validation

* remove explicit dep on zod by identity package

* move more did doc parsing to common-web

* go back to strings

* rollback breaking changes to identity package

* add changeset

---------

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: dholms <dtholmgren@gmail.com>
2023-10-26 18:29:51 -04:00
devin ivy
696df1e488
Lexicons to support bring-your-own-did w/ per-repo signing keys (#1739)
* lexicons for per-repo signing keys

* fix

* rename lexicon

* fix filename

* rename getSigningKey to reserveSigningKey
2023-10-26 17:15:12 -04:00
Daniel Holmgren
bb039d8e4c
Dedupe did cache refreshes (#1773)
* dedupe refreshes to did cache

* handle expired cache entries as well

* apply in pds as well

* changeset
2023-10-25 15:53:08 -05:00