979 Commits

Author SHA1 Message Date
Foysal Ahamed
bae2ce3809
Introduce #reviewOptional as reviewState for non-impactful events on a subject (#2235)
* 🚧 Working through an nullable review state

*  Update snapshots on some tests

*  Update snapshots on some tests

*  Add test for reviewOptional status mutation

* ♻️ Rename reviewOptional -> reviewNone

*  Use FOR UPDATE to respect db transactions
2024-03-06 15:30:23 -05:00
github-actions[bot]
06353c6636
Version packages (#2282)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-06 16:58:19 +00:00
dan
192223f127
Show only replies from people you follow in Following by default (#2279)
* Show only replies from people you follow by default in Following

* Patch
2024-03-06 16:55:58 +00:00
Daniel Holmgren
582109cb03
Ozone ACLs (#2252)
* tidy bsky auth

* hook up new auth verifier

* update auth throughout ozone

* handle mod signing keys

* fix bad var

* fix key parsing in pds

* fix admin auth test

* rename test

* update did doc id values

* null creds string -> `none`

* fix fetchLabels auth check

*  Add a couple more proxied requests that we use in ozone ui

* Add runit to the services/bsky Dockerfile (#2254)

add runit to the services/bsky Dockerfile

* Improve tag detection (#2260)

* Allow tags to lead with and contain only numbers

* Break tags on other whitespace characters

* Export regexes from rich text detection

* Add test

* Add test

* Disallow number-only tags

* Avoid combining enclosing screen chars

* Allow full-width number sign

* Clarify tests

* Fix punctuation edge case

* Reorder

* Simplify, add another test

* Another test, comment

* Version packages (#2261)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* 🐛 Increment attempt count after each attempt to push ozone event (#2239)

* Ozone delegates email sending to actor's pds (#2272)

* ozone delegates email sending to user's pds

* lexicon: add content field to mod email event

* test email sending via mod event

* fix auth verifier method

* better error handling for get account infos

* fix labeler service id

* fix iss on auth headers

* fix dev-env ozone did

* fix tests & another jwt issuer

* ozone: fix ip check

* fix aud check on pds mod service auth

* tidy

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
2024-03-05 19:04:57 -06:00
devin ivy
c7e6ef0907
Ozone delegates email sending to actor's pds (#2272)
* ozone delegates email sending to user's pds

* lexicon: add content field to mod email event

* test email sending via mod event
2024-03-05 13:27:25 -06:00
github-actions[bot]
27ff43b223
Version packages (#2261)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-01 13:03:04 -06:00
github-actions[bot]
6fe00a456e
Version packages (#2250)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-29 13:47:25 -06:00
github-actions[bot]
1b0b4f93ac
Version packages (#2219)
* Version packages

* Give Mary credit

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
2024-02-29 11:14:25 -06:00
Daniel Holmgren
f65de89eed
Feature: Appview v2 (#1924)
* add buf & connectrpc, codegen client

* lint

* prettier ignore

* fix prettier ignore

* tidy & add tests

* filler commit

* rm filler

* server boilerplate

* follows impl

* posts impl

* posts & likes impl

* repost impl

* profiles & handle null values

* list impl

* mutes impl

* blocks impl

* misc

* feed gen impl

* label impl

* notifs impl

* feeds impl

* threads impl

* early sketchwork

* wip

* stub out thick client

* in-progress work on hydrator

* tweak

* hydrate profile labels, detail lists

* feedgen hydration

* protobuf tweaks

* more protobuf tweaks

* wip

* snake case

* moar snake case

* tidy actor hydration

* tidy parsing

* type fixes, renaming, comments in hydrator

* hydrate list items and likes

* hydrate notifications

* feed hydration

* graph & label hydration

* more record protobufs

* pluralize

* tweak pbs

* use new methods

* Setup dataplane grpc client/mock server (#1921)

* add buf & connectrpc, codegen client

* lint

* prettier ignore

* fix prettier ignore

* tidy & add tests

* add record getter mocks

* post view hydration

* fix up mock dataplane to match new protos

* missed one

* wire up dataplane in ctx & dev-env

* adding some basic views

* feed hydration, add blocks to post hydration

* pass over notification hydration

* tidy

* merge

* implement getProfile

* hydrate post aggregation and viewer state

* fix

* fix codegen

* get some tests passing!

* add takedowns & some like bugfixing

* all profile tests passing!

* likes test

* follow endpoints using data plane

* reorg follow block rules

* reposts

* post views!

* implement getList w/ dataplane caveat

* adjust dataplane getListMembers to return listitem uris

* implement getListMutes and -Blocks w/ dataplane

* suggestions

* timeline

* misc view fixes

* view fixes for mutes, self-mute/block

* author feed

* feed gen routes

* tidy

* misc block/mute fixes

* list feed & actor likes

* implement getLists, fix some empty cursors

* implement getMutes, empty profile description fix

* implement getBlocks, block application fix

* implement getSuggestedFollowsByActor, needs some fixes

* feed generation

* search routes

* threads

* tidy

* fix some snaps

* fix getSuggestedFollowsByActor

* implement listNotifications

* implement getUnreadCount w/ dataplane

* implement notifications.updateSeen w/ dataplane

* 3rd party blocking tests

* blocked profile viewer

* add search mocks

* refactor getFeed

* createPipeline -> createPipelineNew

* basic replygating functionality on dataplane w/o filtering violating replies

* hack threadgates into dataplane, apply gates

* deterministic thread orders in dataplane

* misc cleanup around dataplane

* upgrade typescript to v5.3

* update typescript linter deps

* sync bsky proto, codegen

* update dataplane, sync with bsky proto updates

* remove indexer, ingester, daemon, moderation services from appview

* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane

* remove bsky services. db and indexing logic into mock dataplane.

* remove tests not needed by appview v2, misc reorg

* add basic in-mem repo subscription to dataplane mock

* fix dev-env, bsky tests, bsky build

* cull bsky service entrypoint

* add bsky service readme

* build

* tidy

* tidy, fix pds proxy tests

* fix

* fix bsky entrypoint deps

* support http2 grpc client

* build

* fix dataplane bad tls config/default

* support multiple dataplane urls, retry when unavailable

* build

* tidy/fix

* move dataplane mock tests into their own dir

* cover label hydration through timeline test

* bring back labels in appview tests

* remove unused db primary/replica/coordinator from bsky dataplane

* bsky proto add cids to contracts, buf codegen

* sync-up bsky data-plane w/ codegen updates

* start using dataplane interaction endpoints

* add file

* avoid overfetching from dataplane, plumb feed items and cids

* pass refs through for post viewer state

* switch list feeds to use feed item in dataplane

* handle not found err on get-thread dataplane call

* support use of search service rather than dataplane methods

* mark some appview v2 todos

* tidy

* still use dataplane on search endpoints when search service is not configured

* fix pds test

* fix up bsky tests & snaps

* tidy migrations

* fix appview-v2 docker build

* Support label issuer tied to appview v2 (#2033)

support label issuer tied to appview

* Appview v2: handle empty cursor on list notifications (#2017)

handle empty cursor on appview listnotifs

* Update appview v2 to use author feed enum (#2047)

* update bsky protos with author feed enum, misc feed item changes

* support new author feed enums in dataplane

* fix build

* Appview v2: utilize sorted-at field in bsky protos (#2050)

utilize new sorted-at field in bsky protos

* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts

* Appview v2, sync w/ changes to protos (#2071)

* sync bsky protos

* sync-up bsky implementation w/ proto changes

* Appview v2 initial implementation for getPopularFeedGenerators (#2072)

add an initial implementation for getPopularFeedGenerators on appview v2

* merge

* fixes

* fix feed tests

* fix bsync mock

* format

* remove unused config

* fix lockfile

* another lockfile fix

* fix duplicate type

* fix dupplicate test

* Appview v2 handling clearly bad cursors (#2092)

* make mock dataplane cursors different from v1 cursors

* fail open on clearly bad appview cursors

* fix pds appview proxy snaps

* Appview v2 no notifs seen behavior (#2096)

* alter behavior for presenting notifications w/ no last-seen time

* fix pds proxy tests

* Appview v2 dataplane retries based on client host (#2098)

choose dataplane client for retries based on host when possible/relevant

* don't apply negated labels

* display suspensions on actor profile in appview v2

* Appview v2 use dataplane for identity lookups (#2095)

* update bsky proto w/ identity methods

* setup identity endpoints on mock dataplane

* move from idresolver to dataplane for identity lookups on appview

* tidy

* Appview v2: apply safe takedown refs to records, actors (#2107)

apply safe takedown refs to records, actors

* Fix timing on appview v2 repo rev header (#2113)

fix timing on appview repo rev

* fix post thread responses

* Appview v2 don't apply 3p self blocks (#2112)

do not apply 3p self-blocks

* Appview v2 search for feed generators (#2118)

* add protos for feedgen search

* support feed search on getPopularFeedGenerators

* Appview v2 config tidy (#2117)

* remove mod and triage roles from appview

* rename cdn and search config

* remove custom feed harness from appview v2

* Appview v2: don't apply missing modlists (#2122)

* dont apply missing mod lists

* update mock dataplane

* Update packages/bsky/src/hydration/hydrator.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* refactor & document a bit better

* fix up other routes

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* Appview v2 enforce post thread root boundary (#2120)

* enforce post thread root boundary

* test thread root boundary

* Appview v2 fix admin environment variable (#2137)

fix admin env in appview v2

* Remove re-pagination from getSuggestions (#2145)

* remove re-pagination from getSuggestions

* fix test

* Adjust wording for account suspension (#2153)

adjust wording for account suspension

* Appview v2: fix not-found and blocked uris in threads (#2201)

* fix uris of not-found and blocked posts in threads

* update snaps

*  Show author feed of takendown author to admins only (#2197)

* fold in cid, auth, tracing, node version changes

* remove dead config from bsky service entrypoint

* build

* remove ozone test codepaths for appview v2

* tidy, docs fix

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2024-02-27 15:22:55 -05:00
bnewbold
d643b5bb13
lex: tweak deprecation note (#2222)
* lex: tweak deprecation note

The motivation with this is that the docs site hides deprecated
endpoint, and the behavior was updated to only do this when
"deprecation" is the first word of the description (to avoid hiding some
endpoints which just talk about deprecation).

* make codegen
2024-02-23 14:59:47 -08:00
github-actions[bot]
433d47d6a4
Version packages (#2216)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-22 09:46:27 -06:00
Eric Bailey
514aab92d2
Add missing getPreferences union return types (#2215)
* Add missing getPreferences union types

* Add changeset
2024-02-22 09:43:58 -06:00
Daniel Holmgren
6d38eea8f2
Config for blob upload limit (#2213)
config var for blob upload limit
2024-02-22 08:58:18 -06:00
Daniel Holmgren
87d59ff541
Admin: updateAccountPassword (#2212)
* add update account password route

* add test
2024-02-21 23:58:25 -06:00
github-actions[bot]
6d30e2c144
Version packages (#2202)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-21 19:38:02 -06:00
Eric Bailey
b60719480f
Muted items prefs (#2195)
* Muted items prefs

* Add hidden posts

* Enhance

* Update to use smart objects

* Add 'any'

* Codegen

* Enhance

* Muted words methods

* Dry it up

* Format

* Add hidden posts methods

* Who codegens the codegens

* Sanitize tags, always compare bare strings

* Moar test

* Simplify

* Add test

* Add changeset
2024-02-21 19:32:09 -06:00
devin ivy
9f90203f20
Optional service config in pds distribution (#2211)
* make appview and mod services optional on pds

* pds: allow configuring a reporting service optionally separate from an administrative mod service

* tidy
2024-02-21 20:14:45 -05:00
Daniel Holmgren
4e0271bddf
Firehose identity cleanup (#2210)
* missed a couple

* fix tests
2024-02-21 15:59:44 -06:00
Daniel Holmgren
6b8d1c641e
Send identity evts on handle change (#2209)
missed a couple
2024-02-21 15:50:05 -06:00
Daniel Holmgren
6dfc899d99
Enable creation of unknown record types (#2171)
* start to allow 3p lexicons

* tests

* tidy

* tests + ensure no legacy blob ref

* increase the depth were willing to go when searching for blobs
2024-02-21 15:11:15 -06:00
Daniel Holmgren
1a12c7e34b
Firehose Identity event (#2208)
* add new identity event

* add note for tombstone

* send identity evts

* add emission of identity evts

* emit handle event on activate account & fix subscribeRepos

* add time to evt

* update indexer for tests

* rm logs
2024-02-21 15:10:12 -06:00
Daniel Holmgren
30b05a7d4b
Account migration (#2179)
* draft of account migration lexicons

* format

* clean up schemas

* codegen

* second pass on schemas

* small fix

* move around checkImportStatus

* re-codegen

* getServiceAuth

* getServiceAuth impl

* importRepo impl

* handle uploadBlob for import

* allow bringing your own did on createAccount

* working on test flow

* fleshing out flow

* fix up sendPlcOP

* small fixes

* activate/deactivate account

* full flow working!

* schema tweaks

* format

* update schemas

* moar codegen

* match impl to new schemas

* email flow for signed plc operation

* add email flow for plc operations

* impl plc op request email

* proxy to entryway

* tidy activate account

* integrating account deactivated state

* fix up tests

* friendly parse on optional did auth

* admin activate/deactivate routes

* proxy relevant requests to entryway

* remove admin activation routes

* do not proxy acitvaition to entryway

* cfg for disallowing imports

* buff up test

* refactor listMissingBlobs a bit

* add validDid & activated to accoutn status

* emit event on account activation

* test creating a post after migrating

* account deactivation tests

* test name

* tests on plc operations

* fix recommended did creds

* codegen

* turn off accepting imports on createAccount

* undo prev change

* increment version

* build branch

* pr feedback

* handle errs in p-queue

* handle blob upload outside of txn

* Clean old temp account migration lexicons (#2187)

* clean old temp lexicons

* rm old test

* fix agent session test

* fix bsky test

* dont build branch
2024-02-20 19:29:49 -06:00
Daniel Holmgren
4c511b3d9d
Account migration schemas (#2170)
* draft of account migration lexicons

* format

* clean up schemas

* second pass on schemas

* small fix

* move around checkImportStatus

* getServiceAuth

* schema tweaks

* format

* update schemas

* email flow for signed plc operation

* refactor listMissingBlobs a bit

* codegen

* return did in describeServer

* changeset
2024-02-20 19:23:56 -06:00
github-actions[bot]
1cce9ddb34
Version packages (#2193)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-19 11:27:59 -05:00
Foysal Ahamed
d81573232b
Allow filtering moderation queue by language (#2161)
*  Store languages on subjects from record data

*  Fetch lang from author feed for repo subject

*  Regenerate lex

*  Add lang to profile subject

*  Move lang to flag

*  Add comment to flag event

*  Update pds test snapshot

* 🐛 Fix broken import

*  Rename flag to tag

*  Update snapshot for event by id

*  Bring back skipped test

*  Move tags to dedicated columns

*  Update test snapshots, use logger

*  Change lang:unknown to lang:und

*  Update test snapshots

* 🧹 Cleanup

*  Add filter params for tags on queryModerationEvents endpoint

*  Add tags property to subject status model

*  Update test code

*  Add filter to get subjects that do not have certain tags
2024-02-19 10:35:03 -05:00
github-actions[bot]
6e6b94be10
Version packages (#2184)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-15 20:22:33 -06:00
Daniel Holmgren
8c94979f73
Timeline index pref (#2188)
* add following index pref

* added changeset

* change pref name

* update changeset
2024-02-15 20:12:39 -06:00
Devin Ivy
b400fae24e sanity check on did part 2024-02-13 11:28:19 -05:00
bnewbold
9579bec720
update descriptions in Lexicons (#2110)
* lexicons: update descriptions in com.atproto.*

* lexicons: update descriptions in app.bsky.*

* Apply suggestions from code review

Thanks Emily!

Co-authored-by: Emily Liu <emilyliu7321@gmail.com>

* codegen description-only Lexicon updates

---------

Co-authored-by: Emily Liu <emilyliu7321@gmail.com>
2024-02-11 16:06:58 -08:00
devin ivy
d0be052e12
Support node v20 on pds distribution (#2157)
* upgrade node to v20, better-sqlite3 to v9

* fix tests, deps for node v20

* build

* add io_uring setting to pds dockerfile
2024-02-09 19:05:29 -05:00
Daniel Holmgren
758de8743d
PDS pipethrough (#2150)
* initial impl

* re-codegen

* tweak pipethrough

* wip

* read after write pipethrough

* integrate read after write

* setup pds pipethrough logic

* pipe through repo rev header

* fix up tests

* apply pipethrough to most pds routes

* apply pipethrough to remaining pds routes

* fix/tidy

* getRecord

* final details for passthrough in xrpc-server

* build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-02-07 20:40:43 -05:00
github-actions[bot]
77118a66a7
Version packages (#2140)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-06 10:42:24 -05:00
Foysal Ahamed
e4ec7af036
Add date range and comment filter to queryModerationEvents endpoint (#2124)
*  Add date range and comment filter to queryModerationEvents endpoint

*  Add report type and label filters and tests

* tidy tests

*  commentKeyword -> comment and make comment and hasComment independent

*  Fix multiple label filter query

* 📝 Add changesets

* codegen tidy

* add changeset

* remove unused changeset

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-02-05 19:10:44 -05:00
Foysal Ahamed
aa15789e11
🐛 Forward sendEmail event to moderationAgent instead of appview (#2125) 2024-02-04 23:18:27 -05:00
Daniel Holmgren
aaee2d0dc2
Email rate limits followup (#2133)
email rate limits followup
2024-02-03 19:06:07 -06:00
devin ivy
dd021b8b66
Remove all email tokens on email change (#2132)
remove all email tokens regardless of whether token was used to change email
2024-02-03 18:03:57 -05:00
Daniel Holmgren
b6b854afb9
Clear email tokens on email update (#2131)
delete email tokens on email update
2024-02-03 16:52:29 -06:00
Daniel Holmgren
4b1c8af6e3
Mount checkSignupQueue route on pds (#2116)
mount checkSignupQueue route on pds
2024-01-30 17:57:32 -06:00
Daniel Holmgren
f023494802
Remove appview feedgen (#2106)
* remove appview feedgen

* deprecate getTimelineSkeleton

* fix dev-env build

* rm unused test
2024-01-30 16:19:36 -06:00
github-actions[bot]
61bf9fbe7f
Version packages (#2097)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-01-25 14:20:32 -06:00
Daniel Holmgren
8994d3633a
Signup queueing - Client & PDS (#2090)
* schema + codegen

* port session refresh

* pds updates

* typo

* changeset
2024-01-25 14:16:30 -06:00
github-actions[bot]
79ec5ad95d
Version packages (#2093)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-01-25 13:22:42 -06:00
Eric Bailey
4171c04ad8
Add interest tags to preferences (#2086)
* Add interestsPref

* Codegen

* Update lex

* Add method to agent, test

* Codegen

* Format

* Remove console

* Update lex, codegen, update tests

* Update tests

* Format

* Add changeset

* Update property name
2024-01-25 12:05:10 -06:00
Daniel Holmgren
43eb4054a6
Fix typo in getRelationships (#2091)
fix typo in getRelationships + codegen
2024-01-24 19:23:01 -06:00
github-actions[bot]
ffd6c25cc0
Version packages (#2082)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-01-24 13:36:00 -06:00
Daniel Holmgren
dac5c9e202
Tagged suggestions (#2077)
* add tagged suggestions

* fix codegen

* rm unused err
2024-01-24 09:04:09 -06:00
github-actions[bot]
f069c67527
Version packages (#2065)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-01-19 17:24:16 -05:00
Foysal Ahamed
15f38560b9
Manage communication templates for moderation purposes (#2045)
*  Add initial lexicons to manage communication templates

*  All 3 endpoints are functional

*  Add list and delete endpoints

*  Add tests for communication template CRUD

* 🔒 Allow only admins to create and update templates

* 🧹 Cleanup according to PR review

*  Make updatedBy and createdBy optional in lexicon

*  Typo

*  Allow string id and update lexicon language

*  Fix tests

*  content -> contentMarkdown

*  Change column name in db table

* add changeset

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-01-19 17:06:37 -05:00
github-actions[bot]
48298d545f
Version packages (#2063)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-01-18 18:09:50 -06:00
Daniel Holmgren
b906c0cbea
Add uploadBlob rate limit (#2052)
uploadBlob ratelimit
2024-01-18 16:16:10 -06:00