8 Commits

Author SHA1 Message Date
Daniel Holmgren
ad1fcf1387
Repo history rewrite (#1479)
* logical changes in repo

* tests

* tweak commit data

* building pds

* patching up some tests

* tidy + more tests

* patch up bsky

* clean up db

* db migration

* small patches

* fix up another test

* reinclude prevs

* api & lex updates

* add back in deprecated routes

* backward compatibility for commit v2

* add upgrade repo version root

* move namespace

* migration test

* patch up a few more tests

* remove deprecated rebase routes

* tweak api

* sprinkle rev around a few more places

* getCurrent -> getLatestCommit

* fix test

* pr feedback & tidy

* fix up block pagination

* tidy again

* add in a tets

* add cursor to listBlobs

* getRepo rev -> since

* clean up proofs test

* dont change getHead

* tweak migrate route

* build branch

* hit index in block range query

* check for dupe record refs

* tidy

* set prev to null

* dont build branch
2023-08-29 19:07:21 -05:00
Daniel Holmgren
9baf9b23d8
Repo v2 (#635)
* fanout to 4

* move meta to root

* combine repoRoot & commit

* finish up commits & tests

* cleanup

* tidy

* better diff

* fixed up mst tests for bytes & fanout 4

* added tests for allowable chars

* length limit & check on deserialize

* tidy

* better leading zeros fn

* pr feedback

* working on migration

* Setup app migration machinery

* Skip appMigration() tests on sqlite

* migration working

* make an app_migration table

* patch up test

* sanity check

* wrapped up

* working

* remove run on repov2

* delete seqs

* logs & static migration names

* cleanup

* missed a file

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-03-09 18:49:09 -05:00
bnewbold
f08c08a675
MST review and tests (#556)
* mst: more test coverage

In particular, ports the older fanout=32 corner-case tests to fanout=16.

I generated new paths with correct level and ordering, but haven't
actually verified that the resulting structures have the shape of the
ASCII diagrams in text.

* mst: additional comments, including TODO questions

* mst: default new MST layer to 'null' not 0

Looks like all current call sites of this 'create()' function override
the layer already. This is just making it explicit that the default,
null, means the layer has not been computed yet, distinct from being
layer=0.

* mst: remove one TODO and reformat another

From review.
2023-02-23 10:47:16 -08:00
Daniel Holmgren
75e14ae102
Feature branch: storage & sync revamp (#446)
* wip

* fleshing out repo storage

* fleshing out sql storage

* cleaning things up

* fix up tests

* dumb bug - commit log reversed

* rm staging in favor of commiting diffs to blockstore

* clean up benches

* fixing up sql storage

* some caching for sql repo store

* pr feedback

* migration

* wip

* migraiton test

* unclear param

* sql repo storage tests

* rm unused code

* fix up some diff code

* pr feedback

* enum for action types

* missed some

* wip

* ripping out auth lib

* more auth cleanup

* another lurker

* wip better sync primitives

* wip

* improving diffs & sync

* tests working!

* actually implemented checkout lol

* simplify interface & improve error handling

* writing sql storage code

* fixing up tests

* testing & bugfixes

* checkouts return records instead of cids

* one last refactor lol

* missed one

* handle other cid codecs on incoming car verification

* tests + tricky bugs

* unneeded blockstore method

* trim mst on del instead of save

* cleanup comment

* dont resolve did for every commit

* use "commit" instead of "root"

* getRoot -> getHead

* pr feedback

* very silly bug fix

* improve sync output

* reorging + sync of particular records

* serve & verify proofs. also rename some ipld methods

* fix up sync issue in mst

* find reachable records form carfile

* getRecord xrpc method

* pr feedback

* better migration test

* check migraiton result

* fixing up a couple things for pg

* explicit migrateTo

* async exceptions

* ipld car mimetype + remove updateRepo

* Update module publish scripts (#478)

* Update pds package publishing scripts

* Update auth package publishing scripts

* Update crypto package publishing scripts

* Update did-resolver package publishing scripts

* Update handle package publishing scripts

* Update xrpc-server package publishing scripts

* Update common package publishing scripts

* Update plc package publishing scripts

* Update uri package publishing scripts

* Update repo package publishing scripts

* Sort "suggested follows" by number of posts (#477)

* return suggestions by post count

* pr feedback

* fix up PG pagination issue

* partiion commit-history & commit-blocks by user did

* fix migration type

* switch pk indexes

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-01-26 18:09:55 -06:00
Daniel Holmgren
083f53078a
Fix a few MST bugs (#364)
* fix MST stage & getPointer operations

* test

* comments

* fix pointer issue on higher level splits
2022-11-22 14:37:38 -06:00
Daniel Holmgren
c207947bd5
Stagable blockstore (#351)
* stagable blockstore + remove mutable repo

* pds wip

* nearly there

* bugfix

* assertion ordering

* del old code

* save staged if no callback

* removing level

* pr fixup
2022-11-17 09:51:36 -06:00
dholms
c9a6609b93 fixed up common & repo tests 2022-09-13 11:15:56 -05:00
dholms
94babb38b7 split common and repo out into separate packages 2022-09-12 21:01:56 -05:00