ipfs,flatfs datastore

FlatFS

FlatFS is using sharded directories and flat files to store raw block contents on disk. It is not a general-purpose datastore and has several important restrictions.

FlatFS keys are severely restricted. Only keys that match /[0-9A-Z±=]+ are allowed. That is, keys may only contain upper-case alpha-numeric characters, ‘-’, ‘+’, '’, and ‘=’. This is because values are written directly to the filesystem without encoding.

Get

func (fs *Datastore) Get(key datastore.Key) (value []byte, err error) {
	// Can't exist in datastore.
	if !keyIsValid(key) {
		return nil, datastore.ErrNotFound
	}

	_, path := fs.encode(key)
	data, err := ioutil.ReadFile(path)
	if err != nil {
		if os.IsNotExist(err) {
			return nil, datastore.ErrNotFound
		}
		// no specific error to return, so just pass it through
		return nil, err
	}
	return data, nil
}

call stack

github.com/ipfs/go-ds-flatfs.(*Datastore).Get at flatfs.go:589
github.com/ipfs/go-ds-measure.(*measure).Get at measure.go:198
github.com/ipfs/go-datastore/mount.(*Datastore).Get at mount.go:261
github.com/ipfs/go-ds-measure.(*measure).Get at measure.go:198
github.com/ipfs/go-datastore/retrystore.(*Datastore).Get.func1 at retrystore.go:62
github.com/ipfs/go-datastore/retrystore.(*Datastore).runOp at retrystore.go:29
github.com/ipfs/go-datastore/retrystore.(*Datastore).Get at retrystore.go:60
github.com/ipfs/go-datastore/keytransform.(*Datastore).Get at keytransform.go:47
github.com/ipfs/go-ipfs-blockstore.(*blockstore).Get at blockstore.go:123
github.com/ipfs/go-ipfs/thirdparty/verifbs.(*VerifBS).Get at verifbs.go:61
github.com/ipfs/go-ipfs-blockstore.(*arccache).Get at arc_cache.go:118
github.com/ipfs/go-ipfs-blockstore.(*idstore).Get at idstore.go:57
github.com/ipfs/go-ipfs/thirdparty/cidv0v1.(*blockstore).Get at blockstore.go:31
<autogenerated>:2
github.com/ipfs/go-blockservice.getBlock at blockservice.go:228
github.com/ipfs/go-blockservice.(*blockService).GetBlock at blockservice.go:215
...

Procedure:

BlockService.Get -> CidBlockStore.Get -> ARCBlockstore.Get -> VerifBS.Get -> BasicBlockStore.Get -> KeyTransformDS.Get -> RetryDS.Get -> MeasureDS.Get -> MountDS.Get -> FlatFS.Get

• idstore wraps a BlockStore to add support for identity hashes
• arccache wraps a BlockStore with an Adaptive Replacement Cache (ARC) for block Cids. This provides lock access-time improvements, allowing to short-cut many searches without query-ing the underlying datastore.
• VerifyBS wraps blockstore to add validation of the hash. See verifcid.ValidateCid
• Basic Blockstore, performs Cid to Key convertion. The Cid will be encoded by base32, like this: /CIQJ5F7UR2GASG7T4S5B2LG6ECQYLQSGPQRG3PFXNQFKZD7BLB4M2YY
• KeyTransformDS wraps a given datastore with a KeyTransform function. The resulting wrapped datastore will use the transform on all Datastore operations. Key will look like: /blocks/CIQJ5F7UR2GASG7T4S5B2LG6ECQYLQSGPQRG3PFXNQFKZD7BLB4M2YY
• Retry DS, wraps a Batching datastore with a user-provided TempErrorFunc which determines if an error is a temporal error and thus, worth retrying-, an amount of Retries which specify how many times to retry an operation after a temporal error- and a base Delay, which is multiplied by the current retry and performs a pause before attempting the operation again.
• Measure DS wraps datastore which can record metrics using github.com/ipfs/go-metrics-interface
• MountDS provides a Datastore that has other Datastores mounted at various key prefixes and is threadsafe
• FlatFS