1610 Commits

Author SHA1 Message Date
Daniel Holmgren
17ce804ffa
Tweak createSession rate limit key (#1614)
tweak create session rl key
2023-09-15 18:11:48 -05:00
Daniel Holmgren
1cec2dd7d2
Write rate limits (#1578)
* get rate limit ip correctly

* add write rate-limits
2023-09-15 17:35:11 -05:00
Daniel Holmgren
a3a32de0ff
Allow bypass on ratelimit ip (#1613)
allow bypass on ratelimit ip
2023-09-15 17:16:57 -05:00
github-actions[bot]
38f8f44c41
Version packages (#1612)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-09-15 15:49:31 -05:00
Eric Bailey
b1dc355504
fix syntax docs (#1611) 2023-09-15 14:05:55 -05:00
Daniel Holmgren
ba78d225b1
Temporarily disable filtering invalidReplyRoots (#1609)
temporarily disable invalidReplyRoot check
2023-09-15 11:23:33 -05:00
github-actions[bot]
d7d7a043de
Version packages (#1602)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-09-14 17:53:14 -05:00
Daniel Holmgren
79867d2725
Remove default value for post table invalid attrs (#1601)
remove default value for post table attrs
2023-09-14 16:43:04 -05:00
Eric Bailey
cf6002a1bd
order by like.indexedAt in app view (#1592)
* order by like.indexedAt

* use keyset for ordering

* simplify

* ok ok ok I get it now

* Update packages/bsky/src/api/app/bsky/feed/getActorLikes.ts

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

---------

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
2023-09-14 15:38:27 -05:00
Daniel Holmgren
9879ca97b7
Interaction Gating (#1561)
* lexicons for block lists

* reorg blockset functionality into graph service, impl block/mute filtering

* apply filterBlocksAndMutes() throughout appview except feeds

* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application

* impl for grabbing block/mute details by did pair

* refactor getActorInfos away, use actor service

* experiment with moving getFeedGenerators over to a pipeline

* move getPostThread over to a pipeline

* move feeds over to pipelines

* move suggestions and likes over to pipelines

* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread

* remove old block/mute checks

* unify post presentation logic

* move profiles endpoints over to pipelines

* tidy

* tidy

* misc fixes

* unify some profile hydration/presentation in appview

* profile detail, split hydration and presentation, misc fixes

* unify feed hydration w/ profile hydration

* unify hydration step for embeds, tidy application of labels

* setup indexing of list-blocks in bsky appview

* apply list-blocks, impl getListBlocks, tidy getList, tests

* tidy

* update pds proxy snaps

* update pds proxy snaps

* fix snap

* make algos return feed items, save work in getFeed

* misc changes, tidy

* tidy

* fix aturi import

* initial lexicons for interaction-gating

* add interactions view to post views

* codegen

* model bad reply/interaction check state on posts

* initial impl for checking bad reply or interaction on write

* omit invalid interactions from post thread

* support not-found list in interaction view

* hydrate can-reply state on threads

* present interaction views on posts

* misc fixes, update snaps

* tidy/reorg

* tidy

* split interaction gating into separate record in lexicon

* switch interaction-gating impl to use separate record type

* allow checking reply gate w/ root post deletion

* fix

* initial gating tests

* tighten gated reply views, tests

* reply-gating list rule tests

* allow custom post rkeys within window

* hoist actors out of composeThread()

* tidy

* update thread gate lexicons, codegen

* lex fix

* rename gate to threadgate in bsky, update views

* lex fix

* improve terminology around reply validation

* fix down migration

* remove thread gates on actor unindexing

* add back .prettierignore

* tidy

* run ci on all prs

* syntax

* run ci on all prs

* format

* fix snap

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-09-14 15:24:48 -05:00
Daniel Holmgren
77749ab40c
getPopular hotfix (#1599)
dont pass all params
2023-09-14 14:42:09 -05:00
Daniel Holmgren
36ba295333
Remove appview proxy runtime flags (#1590)
* remove appview proxy runtime flags

* clean up proxy tests
2023-09-14 13:55:12 -05:00
Daniel Holmgren
1c42157c59
Improve xrpc server error handling (#1597)
improve xrpc server error handling
2023-09-14 13:54:47 -05:00
Daniel Holmgren
578757cb44
List feeds (#1557)
* lexicons for block lists

* reorg blockset functionality into graph service, impl block/mute filtering

* apply filterBlocksAndMutes() throughout appview except feeds

* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application

* impl for grabbing block/mute details by did pair

* refactor getActorInfos away, use actor service

* experiment with moving getFeedGenerators over to a pipeline

* move getPostThread over to a pipeline

* move feeds over to pipelines

* move suggestions and likes over to pipelines

* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread

* remove old block/mute checks

* unify post presentation logic

* move profiles endpoints over to pipelines

* tidy

* tidy

* misc fixes

* unify some profile hydration/presentation in appview

* profile detail, split hydration and presentation, misc fixes

* unify feed hydration w/ profile hydration

* unify hydration step for embeds, tidy application of labels

* setup indexing of list-blocks in bsky appview

* apply list-blocks, impl getListBlocks, tidy getList, tests

* tidy

* update pds proxy snaps

* update pds proxy snaps

* fix snap

* make algos return feed items, save work in getFeed

* misc changes, tidy

* tidy

* fix aturi import

* lex

* list purpose

* lex gen

* add route

* add proxy route

* seed client helpers

* tests

* mutes and blocks

* proxy test

* snapshot

* hoist actors out of composeThread()

* tidy

* tidy

* run ci on all prs

* format

* format

* fix snap name

* fix snapsh

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-09-14 11:19:04 -05:00
Eric Bailey
3877210e7f
add getSuggestedFollowsByActor (#1553)
* add getSuggestedFollowsByActor lex

* remove pagination

* codegen

* add pds route

* add app view route

* first pass at likes-based suggested actors, plus tests

* format

* backfill with suggested_follow table

* combine actors queries

* fall back to popular follows, handle backfill differently

* revert seed change, update test

* lower likes threshold

* cleanup

* remove todo

* format

* optimize queries

* cover mute lists

* clean up into pipeline steps

* add changeset
2023-09-13 15:56:12 -05:00
bnewbold
abc6cf9ab4
interop test files (#1529)
* initial interop-test-files

* crypto: switch signature-fixtures.json to a symlink

* syntax: test against interop files

* prettier

* Update interop-test-files/README.md

Co-authored-by: Eric Bailey <git@esb.lol>

* disable prettier on test vectors

---------

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-13 11:23:33 -05:00
Daniel Holmgren
0533fab68e
Get rate limit ip correctly (#1577)
* get rate limit ip correctly

* comment
2023-09-12 12:38:20 -05:00
Daniel Holmgren
07bb0da0cf
Remove temp.upgradeRepoVersion (#1588)
remove temp.upgradeRepoVersion
2023-09-12 12:37:57 -05:00
Daniel Holmgren
3cdc72a0a7
Fix getRepo since (#1579)
* fix schemas

* test

* format
2023-09-12 12:37:45 -05:00
Daniel Holmgren
1053c6f26c
Prevent user pref races (#1576)
prevent user pref races
2023-09-12 12:37:36 -05:00
devin ivy
6f419790ac
Feature: block lists (#1531)
* lexicons for block lists

* reorg blockset functionality into graph service, impl block/mute filtering

* apply filterBlocksAndMutes() throughout appview except feeds

* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application

* impl for grabbing block/mute details by did pair

* refactor getActorInfos away, use actor service

* experiment with moving getFeedGenerators over to a pipeline

* move getPostThread over to a pipeline

* move feeds over to pipelines

* move suggestions and likes over to pipelines

* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread

* remove old block/mute checks

* unify post presentation logic

* move profiles endpoints over to pipelines

* tidy

* tidy

* misc fixes

* unify some profile hydration/presentation in appview

* profile detail, split hydration and presentation, misc fixes

* unify feed hydration w/ profile hydration

* unify hydration step for embeds, tidy application of labels

* setup indexing of list-blocks in bsky appview

* apply list-blocks, impl getListBlocks, tidy getList, tests

* tidy

* update pds proxy snaps

* update pds proxy snaps

* fix snap

* make algos return feed items, save work in getFeed

* misc changes, tidy

* tidy

* fix aturi import

* hoist actors out of composeThread()

* tidy

* run ci on all prs

* format

* build

* proxy graph.getListBlocks

* remove unneeded index

* build pds

* setup noop listblock indexer on pds

* remove build

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-12 12:07:31 -04:00
Daniel Holmgren
fdf6a4642f
Enable appview proxy in dev-env full network (#1580)
proxy runtime flags
2023-09-11 19:32:04 -05:00
Daniel Holmgren
d4ebba874e
Increase timeline threshold (#1573)
* increase threshold

* branch name

* fix

* rm build cfg
2023-09-11 18:49:00 -05:00
Paul Frazee
841e906101 @atproto/api@0.6.12 2023-09-11 13:24:08 -07:00
Paul Frazee
4536967280
Fixes and updates to the preferences API (#1575)
* Fix to handle duplicate preference key entries

* Add personal details preference API to sdk

* Add Array.prototype.findLast() type declaration

* Move interface declaration to ensure it's included in other package builds
2023-09-11 13:23:24 -07:00
Daniel Holmgren
b3046c348f
Remove legacy repo sync impl (#1570)
remove legacy repo sync impl
2023-09-08 15:25:30 -05:00
Daniel Holmgren
f8f2d04c1d
Move fuzzy matcher to appview (#1566)
* impl on appview

* tests + clean up pds

* tidy cfg
2023-09-08 15:08:27 -05:00
Paul Frazee
d1df1f3232 @atproto/api@0.6.11 2023-09-08 10:37:24 -07:00
Paul Frazee
44b721b6ae
Add personal-details user preference with birth date (#1565)
* Add personal-details user preference with birth date

* Add personal details pref to settings union
2023-09-08 10:35:49 -07:00
Eric Bailey
15411f274b
maintain feed order (#1559)
* maintain feed order

* clean up order, protect against undefined values

* format
2023-09-07 14:10:38 -05:00
Eric Bailey
a7c42cfe39
Clean up prettier/eslint scripts (#1514)
* add prettierignore, hoist script

* upgrade prettier

* bump prettier eslint deps

* format all files

* hoist prettier check

* remove unused scripts, hoist lint:fix

* remove npm-run-all, unused

* hoist lint

* remove lint scripts

* improve lint scripts

* remove prettierignores

* downgrade prettier deps to fix codgen

* reformat all files

* update makefile, format

* fix locklife

* final format

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-06 19:27:50 -05:00
Daniel Holmgren
f023cfb009
Abyss Integration (#1545)
* add blob scanner

* introduce auto moderator

* tidy

* tests

* more tests & tidy

* timeout & logs

* tweak cfg

* push takedowns to pds

* build branch

* update snaps

* add cloudfront invalidator to indexer

* tweak log

* dont build branch
2023-09-06 17:35:39 -05:00
Daniel Holmgren
a948d53969
Only do read after write on own author feed (#1544)
only munge on own feed
2023-09-06 10:08:18 -05:00
David Buchanan
90b0a9d809
RFC: Support image aspect ratio metadata (#1306)
* Add aspectRatio field to embed image lexicon

* lexicon codegen

* pass aspectRatio through to imagesEmbed views

* simplify aspectRatio logic, port to bsky package

* re-run lexicon codegen
2023-09-06 10:08:06 -05:00
Eric Bailey
90e8325e5f
pnpm setup (#1532)
* use pnpm

* fix dependency issues, replace yarn and lerna scripts

* remove the main/dist scripts

* update Dockerfiles

* use pnpm

* fix dependency issues, replace yarn and lerna scripts

* remove the main/dist scripts

* update Dockerfiles

* update bin script

* remove unused zod dep

* fix type errors in pds

* add types prop to packages

* remove unused, bump lock

* fix test running

* build before test

* fix pino types

* format

* pds depends on dev-env in test

* refer to src instead of built packages

* pds relies on bsky in test too

* remove yarn.lock

* add -r flag to root test

* test push to aws

* remove docker test

* add publishConfig to new package

* move services to top level dir

(cherry picked from commit f5012bec33435a4473e9960066807623334f3aff)

* update workflow paths

(cherry picked from commit 5c70f0176d381ca35d6be10cfa173e22373a5b5d)

* add main-to-dist script

* use script in all packages, remove old Dockerfiles

* remove old bsky service

* remove newline

* test container builds

* Revert "test container builds"

This reverts commit c228611f5e8e1624d4b124be4976c49590130f43.

* remove unused config

* test build containers

* pnpm in syntax

* bump dd-trace

* shamefully hoist

* even more shame

* hoist, externalize deps

* clean install for prod and smaller containers

* dont build branches

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-05 18:45:49 -05:00
Foysal Ahamed
f7186f03a9
Improve matching in results from searchRepos (#1507)
*  Better results from searchRepos when matching by handle

*  Adjust tests for repo search result order

* 🧹 Cleanup

* ♻️ Refactor to re-use search query builder

* 🧹 Cleanup

*  Add test for pagination in search result
2023-09-05 10:56:19 -04:00
Foysal Ahamed
4fb32e1e06
Add missing labels that are available on the moderation tool (#1515)
* Add missing labels that are available on the moderation tool

* Remove unused label

* More cleanup

* 🧹 Get rid of ncii as well

*  Add new label in json instead of generated file 🤦🏽‍♂️
2023-09-05 10:49:55 -04:00
Eric Bailey
16662b220d
@atproto/api@0.6.9 (#1546)
* @atproto/api@0.6.9

* @atproto/api@0.6.10
2023-09-01 10:43:30 -05:00
Eric Bailey
f84027fe8e
add getFeedSuggestions (#1542)
* add getFeedSuggestions lex

* move to feed namespace

* codegen

* add table and migration

* add endpoints

* remove comment

* add test

* fix order

* rename

* just remove proxy check
2023-08-31 17:19:34 -05:00
Daniel Holmgren
9a0fd91f5c
Limit rev cache (#1543)
* disable rev cache

* disable rev cache

* use limit instead of disabling

* dont build branch
2023-08-31 15:35:47 -05:00
Daniel Holmgren
f8c89e54ca
Legacy checkouts (#1539)
* serve legacy repos & checkouts

* tidy

* check lock before formatting commit

* tidy locks

* clean up db test

* check error on failed commit format

* be more generous with new blocks

* Update packages/pds/src/db/index.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* force upgrade & ensure lock

* make it a bit smarter

* add log

* dont build branch

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-08-31 11:03:22 -05:00
Devin Ivy
f9d6673da9 @atproto/repo@0.3.0 2023-08-30 00:14:06 -04:00
Paul Frazee
6d765f7f61 @atproto/api@0.6.8 2023-08-29 17:13:35 -07:00
Paul Frazee
8db9077c68 @atproto/api@0.6.7 2023-08-29 17:10:13 -07:00
Paul Frazee
2b8cbcee1f
Add preferences helpers to SDK (#1537) 2023-08-29 17:09:46 -07:00
Daniel Holmgren
ad1fcf1387
Repo history rewrite (#1479)
* logical changes in repo

* tests

* tweak commit data

* building pds

* patching up some tests

* tidy + more tests

* patch up bsky

* clean up db

* db migration

* small patches

* fix up another test

* reinclude prevs

* api & lex updates

* add back in deprecated routes

* backward compatibility for commit v2

* add upgrade repo version root

* move namespace

* migration test

* patch up a few more tests

* remove deprecated rebase routes

* tweak api

* sprinkle rev around a few more places

* getCurrent -> getLatestCommit

* fix test

* pr feedback & tidy

* fix up block pagination

* tidy again

* add in a tets

* add cursor to listBlobs

* getRepo rev -> since

* clean up proofs test

* dont change getHead

* tweak migrate route

* build branch

* hit index in block range query

* check for dupe record refs

* tidy

* set prev to null

* dont build branch
2023-08-29 19:07:21 -05:00
Daniel Holmgren
70c8159644
Fix ambiguous reference to repoRev (#1534)
fix ambiguous column
2023-08-29 14:30:26 -05:00
dholms
913c912bc4 v0.2.2 2023-08-28 19:10:53 -05:00
dholms
d0a3d70700 v0.2.1 2023-08-28 19:04:30 -05:00
Daniel Holmgren
89917d7fc6
Bump pkg versions (#1525)
* v0.2.0

* bump xrpc
2023-08-28 15:00:24 -05:00