Skip to content

ipfs add: to-files with wrap creates literal single-character dot folders #10611

Closed

Description

@ProximaNova

Checklist

Installation method

dist.ipfs.tech or ipfs-update

Version

Kubo version: 0.32.1
Repo version: 16
System version: amd64/linux
Golang version: go1.23.3

Config

{
  "API": {
    "HTTPHeaders": {
      "Access-Control-Allow-Methods": [
        "PUT",
        "POST",
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "http://10.0.0.231:5001",
        "http://localhost:3000",
        "http://127.0.0.1:5001",
        "https://webui.ipfs.io"
      ]
    }
  },
  "Addresses": {
    "API": "/ip4/10.0.0.231/tcp/5001",
    "Announce": null,
    "AppendAnnounce": null,
    "Gateway": "/ip4/10.0.0.231/tcp/8080",
    "NoAnnounce": null,
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/webrtc-direct",
      "/ip4/0.0.0.0/udp/4001/quic-v1",
      "/ip4/0.0.0.0/udp/4001/quic-v1/webtransport",
      "/ip6/::/udp/4001/webrtc-direct",
      "/ip6/::/udp/4001/quic-v1",
      "/ip6/::/udp/4001/quic-v1/webtransport"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
  ],
  "DNS": {
    "Resolvers": {}
  },
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "99999999999999999999h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 908070605040302000,
    "StorageMax": "12345678910123456789GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true
    }
  },
  "Experimental": {
    "FilestoreEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "OptimisticProvide": false,
    "OptimisticProvideJobsPoolSize": 0,
    "P2pHttpProxy": false,
    "StrategicProviding": true,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "APICommands": [],
    "DeserializedResponses": null,
    "DisableHTMLErrors": null,
    "ExposeRoutingAPI": null,
    "HTTPHeaders": {},
    "NoDNSLink": false,
    "NoFetch": true,
    "PathPrefixes": [],
    "PublicGateways": null,
    "RootRedirect": ""
  },
  "Identity": {
    "PeerID": "12D3KooWLowyP9yZLaxzactf2WAhR6cT1nahBmeRM23zPQR3CP4T"
  },
  "Internal": {},
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Ipwb": {
    "Replay": {
      "Host": "localhost",
      "Index": "/ipfs/bafkreicddpbk7q3kgranv3kchhijt24kf6f2tsow37uk34x2bu7mrfzghu",
      "Port": 2016
    }
  },
  "Migration": {
    "DownloadSources": [],
    "Keep": ""
  },
  "Mounts": {
    "FuseAllowOther": true,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": null
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {
    "Interval": "0"
  },
  "Routing": {
    "AcceleratedDHTClient": false,
    "Methods": null,
    "Routers": null
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {},
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "RelayClient": {},
    "RelayService": {},
    "ResourceMgr": {},
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Description

ipfs add --help does not indicate that these two options don't work so well together:
-w, --wrap-with-directory : bool - Wrap files with a directory object.
--to-files : string - Add reference to Files API (MFS) at the provided path.

Background: say that with every ipfs add command that I run, I want to add the file or folder to MFS. A month ago I added folder "a" (... --to-files=/tf/0/ ...) then today I add a different folder named "a". This is a conflict because an entry for "/tf/0/a" already exists. One would think that the solution is to only add CIDs with to-files+wrap; the expected outcome would be that that would copy the final wrap-with-directory CID to MFS. Instead what you get is...

Bug: it writes a folder literally named "." to MFS. Dealing with paths in MFS, a gateway, a WebUI, or IPFS = each does not interpret "." literally ("." and ".." are both valid linknames in IPLD/IPFS/UnixFS). So you can't do this, for example:
$ ipfs ls -s /ipfs/QmXiw5bwsuKVYEwDwLTZTu71eM43gqTGngQHFFV4pja9R5/weird/./a/
instead you have to find the CID of "." by looking in folder "weird".

I did two tests, second test:

$ ipfs add -rHw --cid-version=1 --chunker=size-1048576 --to-files=/tf/ "/home/u/Downloads/a/R"
[2 files]
added bafybeia25pwibtmoioab5apa65w6sgfnr54ranjjeg5j4ixa7gflnrueb4 R
added bafybeiakjgbiwmv24bxwjawgg3non235rlbprrctk24l6hmoi6aacw6qsy 
[===] 100.00%
$ # 1.35-GiB folder in mfs: /ipfs/QmaeewH1ZmmQwLNNC2pQ12TNf5sqRkqQ9UMdSi9WD9gyXB

The --to-files option takes the name of the final CID. It doesn't account for the case where the final CID has no name (like with -w). I think this is a bug because I don't think this is the intended or expected outcome. (I searched the issues for '"to-files" "wrap"' and saw nothing.)

BTW, if you didn't know, "." non-literally means "current directory" in various systems, so if you run "ls ." in GNU/Linux it will show you the contents of the current working directory. More helpful example: "stat ./-a.txt" vs "stat -a.txt". With this bug, a folder named . shows up as "." in ipfs, MFS, and gateway(s); it shows up as "[name of current folder]" in WebUI at http://127.0.0.1:5001/webui -> ... -> http://127.0.0.1:5001/ipfs/bafybeibgic2ex3fvzkinhy6k6aqyv3zy2o7bkbsmrzvzka24xetv7eeadm/#/files/path/to/singleDotDir (literal directory "." is falsely named "singleDotDir").

Activity

  1. added
    kind/bugA bug in existing code (including security flaws)
    need/triageNeeds initial labeling and prioritization
    on Dec 3, 2024
  2. self-assigned this
    on Dec 3, 2024
  3. added 2 commits that reference this issue on Dec 3, 2024
    9378dbb
    b320438
  4. hsanjuan commented on Dec 3, 2024

    @hsanjuan
    Contributor

    I sent a fix at #10612. I think to-files should not allow empty-named files in general as it leads to these kinds of things.

  5. added a commit that references this issue on Dec 3, 2024
    433444b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

kind/bugA bug in existing code (including security flaws)need/triageNeeds initial labeling and prioritization

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions