Skip to content

feat(importer) Implement Rabin Fingerprintingย #56

Closed

Description

@whyrusleeping

Our current file chunking scheme is just splitting on blocks of a certain size. This is bad for a variety of reasons. @jbenet and I think it will be best to use rabin fingerprinting as our default method for file chunking. The downside is that, other than a brief description on wikipedia, there isnt much to go on for the algorithm itself, so its gonna be a bit of fun figuring this out.

Activity

  1. changed the title [-]Implement Rabin Fingerprinting[/-] [+]feat(importer) Implement Rabin Fingerprinting[/+] on Sep 13, 2014
  2. btc commented on Sep 24, 2014

    @btc
    Contributor

    Is there an open PR/branch associated with this?

  3. whyrusleeping commented on Sep 25, 2014

    @whyrusleeping
    MemberAuthor

    uhm... i beleive its in master at this point, called 'MaybeRabin' in importer/rabin.go (calling it maybeRabin because i need someone else to verify that the algorithm is correct. (Although it appears to work as expected)

  4. added a commit that references this issue on Nov 18, 2021
  5. added 2 commits that reference this issue on Feb 25, 2022
  6. added a commit that references this issue on Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

kind/enhancementA net-new feature or improvement to an existing feature

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions