2067 Commits

Author SHA1 Message Date
Devin Ivy
5560b7a25a Revert "lexicons: more string limits (#1994)"
This reverts commit ad0d976188d1f07401b9675b5c6045c91e82a84e.
2024-01-03 16:41:07 -05:00
Foysal Ahamed
5e7b0136da
Allow appealing a moderator decision through special report type (#1969)
*  Allow appealing a moderator decision through special report type

*  Allow querying subjects by appealed status

*  Move to appealed boolean state column

*  Remove leftover

*  Move appealed status to new boolean column

*  Throw when non-author attempts to appeal a subject

* 🚨 Appease the linter gods

* build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-01-02 19:17:42 -05:00
bnewbold
ad0d976188
lexicons: more string limits (#1994)
* limit external embed strings sizes

* set a (very large) alt text length limit

The motivation is to have *some* size limit on every string in post
records, to maximize interoperation. For example, we currently have a
CBOR library rejecting some records because of too-long strings.

We don't want to limit the ability of folks to be very descriptive in
alt text, specifically, so chose what seems to be a very large limit. If
this is not large enough, based on feedback, we can bump it even higher.
As context this is the largest string length limit in all of our
lexicons.

* make thumbnail URL fields format=uri

This mostly results in checks against the string being empty, or
unlimited size.

* codegen: string limits

* add changeset for string format lex updates
2024-01-03 00:24:32 +01:00
devin ivy
28609519a8
Pin alpine version temporarily for pds docker build (#1976)
* pin alpine version temporarily for pds docker build

* fix

* add note
2023-12-29 11:35:59 -05:00
devin ivy
a2a07f1e63
Experimental node clustering for bsky frontends (#1985)
* experimental node clustering for bsky frontends

* build
2023-12-28 00:15:15 -05:00
bnewbold
50f209e650
bsky: add top-level and robots.txt endpoints (#1979)
These provide helpful context (instead of just a generic 404).

The robots.txt in particular will show up for public.api.bsky.app
2023-12-21 18:01:21 +01:00
Foysal Ahamed
905743de9f
Optionally allow comment when sending email to be added to moderation event (#1981) 2023-12-20 09:49:21 -05:00
Foysal Ahamed
f365dc54dd
Create a mod event when automod applies labels to a record (#1927) 2023-12-20 09:47:51 -05:00
Daniel Holmgren
aadd35e1bd
Better handle 3rd party blocks in feeds (#1677)
* better handle 3rd party blocks inf eeds

* show anchor post in threads

* test
2023-12-19 11:43:16 -06:00
devin ivy
16b31e3ce2
Proper validation of entryway op in account creation (#1975)
* ensure entryway plc rotation key

* update plc lib on pds, test
2023-12-18 12:02:17 -05:00
devin ivy
80161e3c62
Optimize timeline for limit=1 (#1971)
* optimize timeline skeleton w/ limit=1

* build
2023-12-15 15:45:31 -05:00
Foysal Ahamed
7d818b8f19
Allow moderators along with admins to toggle invite codes (#1928)
*  Allow moderators along with admins to toggle invite codes

*  Adjust test to use triage token
2023-12-14 17:29:13 -05:00
github-actions[bot]
2012796c3b
Version packages (#1967)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-13 18:17:15 -08:00
Paul Frazee
8f3f43cb40
Fix: elevate !no-unauthenticated label to affect the entire account when on the profile (#1966)
* Fix: elevate !no-unauthenticated label to affect the entire account when on the profile

* lint

* Add changeset

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-12-13 18:13:19 -08:00
github-actions[bot]
8d4a87c0a8
Version packages (#1963)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-12 15:35:52 -08:00
Paul Frazee
7dec9df3b5
Bump api package (#1962) 2023-12-12 15:30:17 -08:00
Daniel Holmgren
89b4d3d354
Report separate error on getProfile for account suspension (#1960)
* report separate error on getProfile for account suspension

* Update packages/bsky/src/services/moderation/index.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-12-12 15:40:38 -06:00
Daniel Holmgren
0c54951f27
Add seenAt time to listNotifications (#1959) 2023-12-12 15:37:06 -06:00
devin ivy
2a9914363f
Stop running what's hot view maintainer on appview (#1953)
stop running what's hot view maintainer on appview
2023-12-11 15:20:34 -05:00
github-actions[bot]
f3c1272e0e
Version packages (#1946)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-08 15:35:01 -06:00
Eric Bailey
ffe39aae83
Include own replies to own post roots via new filter posts_and_author_threads (#1776)
* Include own replies to own post roots in post_no_replies

* Updates

* Simplify

* snaps

* snaps

* Remove unused import

* Add new threads filter to getAuthorFeed

* Implement new filter

* Typo
2023-12-08 15:32:06 -06:00
github-actions[bot]
8f84207a75
Version packages (#1945)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-08 14:14:34 -06:00
Eric Bailey
60deea1762
Strip trailing colons from link detection, add test (#1944) 2023-12-08 11:44:43 -06:00
devin ivy
842e183925
Add labels to actor search typeahead results (#1940)
add labels to appview actor search typeahead
2023-12-06 19:21:06 -05:00
github-actions[bot]
cbc94aca13
Version packages (#1938)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-06 12:23:41 -08:00
Paul Frazee
45352f9b6d
Bump api package for no-unauthenticated PR (#1937)
Bump api
2023-12-06 12:21:59 -08:00
Paul Frazee
cea5189c0c
Add !no-unauthenticated imperative label (#1926)
* Add !no-logged-out imperative label

* Add profile behavior docs and tests

* Tweak copy

* Prettier pass

* Rename no-logged-out label to no-authenticated
2023-12-06 12:15:37 -08:00
Daniel Holmgren
5038b50428
Fail open on did cache (#1934)
fail open on did cache
2023-12-05 18:08:41 -06:00
Daniel Holmgren
10f934b6c5
Appview rate limits parse cidr block (#1932)
appview rate limits parse cidr block
2023-12-05 17:18:24 -06:00
Daniel Holmgren
90564eeed2
Tweak rate limit cfg (#1931)
tweak rate limit cfg
2023-12-05 16:52:15 -06:00
Daniel Holmgren
e8473cb6be
Set trust proxy on appview for xff headers (#1930)
set trust proxy on appview
2023-12-05 16:37:47 -06:00
Daniel Holmgren
cf53d2025c
Transfer with-friends & best-of-follows to feed generator (#1919)
* transfer feeds

* remove unused feeds
2023-12-05 16:33:48 -06:00
Daniel Holmgren
af3e33a68b
Add basic application rate limits to appview (#1902)
* cache did docs in redis

* drop table

* expire from redis

* fix tests

* add cache class

* update api

* refactor

* filter negative labels

* fix up dev-env

* refactor did cache to use new redis cache class

* tidy

* ensure caching negatives

* redis cache tests

* remove timeout on did cache

* fix ns in test

* rename driver

* add timeout & fail open

* add test for timeout & fail open

* add basic rate limits

* tidy

* small pr feedback

* refactor caches

* bugfixg

* test for caching negative values

* little more to cache

* wire up cache cfg

* switch from redis scratch to redis

* fix build issues

* use different redis clients for tests

* fix test

* fix flaky test

* fix build issue

* use separate db for redis cache
2023-12-05 16:33:28 -06:00
Daniel Holmgren
3110a2a704
Cache tweaks (#1929)
* cache tweaks

* tweak

* another fix

* tweaks to expiry

* handle empty setMulti

* dont build
2023-12-05 15:57:05 -06:00
dholms
9abdf54aff Fix typo in service entry 2023-12-05 14:39:53 -06:00
dholms
249f3165a3 Import redis in service entry 2023-12-05 14:22:34 -06:00
Daniel Holmgren
e566bef1cc
Cache labels in Redis (#1897)
* cache did docs in redis

* drop table

* expire from redis

* fix tests

* add cache class

* update api

* refactor

* filter negative labels

* fix up dev-env

* refactor did cache to use new redis cache class

* tidy

* ensure caching negatives

* redis cache tests

* remove timeout on did cache

* fix ns in test

* rename driver

* add timeout & fail open

* add test for timeout & fail open

* small pr feedback

* refactor caches

* bugfixg

* test for caching negative values

* little more to cache

* wire up cache cfg

* switch from redis scratch to redis

* fix build issues

* use different redis clients for tests

* fix test

* fix flaky test

* use separate db for redis cache
2023-12-05 13:53:05 -06:00
Foysal Ahamed
7e74f00a28
♻️ Cleanup linter warnings (#1907) 2023-12-05 19:42:29 +01:00
Matthieu Sieben
50c1366704
fix(bsky): cf image invalidator not taken into account when bunny used (#1823) 2023-12-05 11:51:04 -05:00
Matthieu Sieben
9cb81426dd
pre-compile handlebar templates at build time (#1833)
feat(pds): pre-compile handlebar templates at build time
2023-12-05 11:48:22 -05:00
Matthieu Sieben
d54a7e25e5
feat: infer definition type from "types" argument in getDefOrThrow (#1812)
feat(lex-cli): infer definition type from "types" argument in getDefOrThrow
2023-12-05 11:43:20 -05:00
devin ivy
49e7f98fdb
Config to start notifications daemon from a specific did (#1922)
config to start notifications daemon from a specific did
2023-12-05 10:39:57 -05:00
Daniel Holmgren
f9fd3e68ca
Feature branch: PDS v2 (#1789)
* cleanup repeat process all

* wip

* skip actor search test

* skip actor search test

* tweak processAll

* decrease wait to 1 sec

* repo_blob -> record_blob

* simplify backlink linkTo

* return repo_root to one row

* sequence before updating repo_root

* invite code forUser -> forAccount

* ipld_block -> repo_block

* use lru-cache fetchMethod

* move did_cache to own db

* better error handling on did cache

* drop did_handle

* fix sequencer wait time

* debug

* debug

* more debug

* check something

* fix bday paradox

* fix bday paradox

* tidy up pds service auth

* rm skipped test

* retry http

* tidy

* improve fanout error handling

* fix test

* return signing key in did-web

* more tests

* tidy serivce auth checks

* user_account -> account

* remove inviteNote

* keypair per repo

* use an lru cache for keypairs as well

* clean up repo

* wip

* wrap up accoutn manager

* tidy

* tidy

* fix tests

* fix disabled codes

* fix appview tests

* add note

* set pragmas

* tidy account manager getDb

* rename pref transactor

* user pref -> account pref

* handle blob imports

* tidy imports

* add reserveSigningKey

* wip transferAccount

* clean up transferAccount

* tests

* tidy

* tidy

* configure entryway url on pds

* handle entryway in pds admin endpoints

* make importRepo temp

* fix imports

* make email optional on pds when using entryway

* handle diffs

* handle pds entryway usage for server, identity, admin endpoints

* pds support for credentials from entryway

* setup pds tests w/ entryway service

* tidy

* tidy

* update entryway version

* wip

* test handle updates w/ entryway

* split account table into two

* tidy

* tweak scripts

* tidy tests

* tidy

* better config for actorstore & dbs

* clean up cfg more

* reorg actorstore fs layout

* handle erros on actor db create

* pr tidy & fix accoutn deletion test

* pr feedback

* fix bad merge

* unskip test

* fix subscribe repos tests

* tidy repo root tables

* tidy

* fix tests

* tidy delete tokens

* tidy account getters

* tidy

* bulk deletesg

* increase chunk size

* handle racing refreshes

* wip

* fix auth test

* invert import flow

* clean up actor store on create account failure

* tweak sequencer

* prevent invite code races on createAccount

* rm note

* add back in race protection on getAccountInviteCodes

* start feature branch

* deleted app migration table

* patch up new auth test

* rm note

* g

* create accoutn delegated from entryway

* tidy

* fix test

* change plcOp type to unknown

* small fixes

* sync up w entryway branch

* Use proper error when authed account is not found (#1799)

provide proper error when account not found in access-takedown check

* build branch

* build on ghcr

* tweak service file

* tweak service file

* change where we save reserved keys

* no tmp dir in blobstore either

* fix blobstore temp location again

* handle repeat record_blobs

* create account before submitting plc op & undo if fail

* small tweak

* limit the number of local records

* push out empty commit on transfer

* fix issue with record_blob

* add push blob endpoint

* Set and validate token audiences on pds v2 (#1793)

set and validate token audience on pds v2

* merge

* include entryway did on tests

* build branch

* fix cache issue

* xrpc server blob limit

* put correct bytes

* add auth to routes

* handle quarantining/unquarantining a blob that does not exist

* tidy

* fix transfer tests

* fix email request routes for entryway

* PDS v2 entryway account deletion (#1819)

* add admin lexicon for account deletion

* implement admin account deletion endpoint

* fix entryway proxying on account email checks

* proxy to entryway for acct deletion

* read-after-write sanity check

* tweak

* wip

* finish refactor

* fix test schema

* application retry logic for busy

* pr feedback

* rm lru-cache

* fix test pg schema

* fix transfer test

* Sqlite instrumentation for pds v2 (#1838)

* sqlite instrumentation

* build

* remove build

* dont reimport blobs

* send ticks during import

* close on error

* catch handle validation error

* add log

* fix test

* return emailConfirmedAt on getAccountInfo

* Upgrade sharp on pds v2 (#1863)

upgrade sharp to 0.32.6

* read all bytes before parsing car

* Async car reader (#1867)

* asynchronously read in car

* dont buffer car

* tweak

* Gracefully handle indexing of invalid records (#1853)

* gracefully handle indexing of invalid records

* fix repo tests

* Fix role auth for access-or-role verifier, getBlob check on actor takedowns (#1869)

fix role auth for access-or-role verifier, fix getBlob actor takedown check

* better cleanup of actor-stores

* add ability to not ensure leaves

* tidy

* allow did:web transfer

* Migration utility for actor-store (#1873)

beginnings of helper for migrating all actors

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* base case for findBlobRefs

* App-level retries for sqlite on pds (#1871)

* revamp retry helper to be more flexible re: backoff strategies

* sqlite timeout helper

* ensure sqlite wal on db creation/migration rather than every open

* layer retries for sqlite on writes outside transactions on pds

* tidy

* fix up lockfile

* tidy

* fix lex codegen

* fix timing bug in threadgate test

* No-op update handling (#1916)

do no produce commits on no-op updates

* Retry on all SQLITE_BUSY error codes (#1917)

retry on all sqlite_busy error codes

* Pds v2 ensure sqlite ready (#1918)

ensure sqlite is ready before making queries

* try something

* tidy

* dont build branch

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-12-04 18:00:09 -06:00
devin ivy
cad30a7cc8
Cleanup outdated notifications in appview, add daemon for similar tasks (#1893)
* initial notification tidy logic

* helper for maintenance across all appview users

* tiny reorg

* add bsky daemon to tidy notifications

* tidy, add bsky daemon service entrypoint

* test notifs tidy daemon, add stats

* tidy

* crash failed notification daemon

* fix notification tidy constants
2023-12-01 19:01:31 -05:00
Daniel Holmgren
6d21cc1b01
Add flag for running db migrations on appview (#1913)
* add flag for running db migrations on appview

* lint

* another fix
2023-12-01 17:29:26 -06:00
devin ivy
9cec13ee46
Do not generate notifs when post violates threadgate (#1901)
* do not generate notifs when post violates threadgate

* don't count threadgate-violating replies, style
2023-12-01 18:15:27 -05:00
github-actions[bot]
1f3fad2829
Version packages (#1909)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-12-01 18:14:01 -05:00
devin ivy
378fc6132f
Additional @atproto/api 0.6.24 changeset (#1912)
api changeset
2023-12-01 18:10:23 -05:00
Daniel Holmgren
a4e5abf5f4
Fix snapshots for list items (#1911)
fix snapshots for list items
2023-12-01 15:07:17 -06:00
intrnl
8d9b1f70cd
Attach record URI to listItemView (#1758)
* feat: attach record uri to listitemview

* chore: attempt to update test snapshots
2023-12-01 14:28:54 -06:00