Skip to content

Segmentation fault ipfs daemon --mountΒ #3032

Closed

Description

@karlek

Version/Platform/Processor information (from ipfs version --all):

go-ipfs version: 0.4.3-dev-1c41cac
Repo version: 4
System version: amd64/linux
Golang version: devel +495e3c6 Tue May 17 04:02:11 2016 +0000

Type (bug, feature, meta, test failure, question): bug
Area (api, commands, daemon, fuse, etc): daemon
Priority (from P0: functioning, to P4: operations on fire): P2

Description:
I ran a ipfs get over night on a 13~GiB large hash, and when I woke up it hade crashed with a segmentation fault after 20%.

The ipfs get produced this output: Error: unexpected EOF

This was the output from the ipfs daemon:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x0]

goroutine 10460213 [running]:
panic(0xb5ce40, 0xc4200140d0)
    /home/_/go/src/runtime/panic.go:500 +0x18c
github.com/ipfs/go-ipfs/routing/dht.(*IpfsDHT).findProvidersAsyncRoutine(0xc4201d66e0, 0x104e7e0, 0xc42968e340, 0xc43b082030, 0x22, 0x3, 0xc43b31cf60)
    /home/_/Desktop/privgo/src/github.com/ipfs/go-ipfs/routing/dht/routing.go:385 +0x5bc
created by github.com/ipfs/go-ipfs/routing/dht.(*IpfsDHT).FindProvidersAsync
    /home/_/Desktop/privgo/src/github.com/ipfs/go-ipfs/routing/dht/routing.go:306 +0x16e

Activity

  1. whyrusleeping commented on Aug 3, 2016

    @whyrusleeping
    Member

    @karlek thanks for reporting! After this happened, was the daemon still running? or did the process exit?

  2. karlek commented on Aug 3, 2016

    @karlek
    Author

    The deamon didn't continue running, it crashed with the panic.
    Sorry was out drinking 🍺

  3. whyrusleeping commented on Aug 3, 2016

    @whyrusleeping
    Member

    No worries, this one is tricky. Thanks for the info!

  4. whyrusleeping commented on Aug 4, 2016

    @whyrusleeping
    Member

    @karlek what version of go are you running? It doesnt appear to be a release version.

  5. whyrusleeping commented on Aug 4, 2016

    @whyrusleeping
    Member

    My first instinct is that the error is a 'hidden nil' where its a typed nil pointer returned through an interface. This would make it past the err != nil check on routing.go:380 and panic on line 385, but the panic trace should look different...

    Proof of concept of the mentioned 'hidden nil' bug:

    package main
    
    import (
        "fmt"
    )
    
    type myErr struct {
        a int
    }
    
    func (e *myErr) Error() string {
        return fmt.Sprint(e.a)
    }
    
    func main() {
        var err error
        err = (*myErr)(nil)
    
        if err != nil {
            fmt.Println(err.Error())
        }
    }

    Results in:

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x401043]
    
    goroutine 1 [running]:
    panic(0x48ff40, 0xc42000a0c0)
            /home/whyrusleeping/go/src/runtime/panic.go:500 +0x1a1
    main.(*myErr).Error(0x0, 0x0, 0x0)
            /tmp/x8HM4.go:12 +0x43
    main.main()
            /tmp/x8HM4.go:20 +0x48
    exit status 2
    

    Where we can clearly see that the panic is a result of the error being nil.

  6. whyrusleeping commented on Aug 4, 2016

    @whyrusleeping
    Member

    Pushing a PR that tries to debug and diagnose my suspicions here, It will print an error message and not crash your daemon. :)

  7. added a commit that references this issue on Aug 4, 2016
  8. karlek commented on Aug 4, 2016

    @karlek
    Author

    @whyrusleeping I use an old development version golang; haven't downloaded the latest version in a while.

  9. whyrusleeping commented on Aug 4, 2016

    @whyrusleeping
    Member

    @karlek if you don't mind, could you update to a newer release version of go and rebuild ipfs? The possibility exists that your old version of go has a bug in it thats causing this issue (especially since its not a release build).

  10. karlek commented on Aug 4, 2016

    @karlek
    Author

    Absolutely. Will write here if the bug reappear.

  11. Kubuxu commented on Aug 4, 2016

    @Kubuxu
    Member

    The crash should be solved but we added error log in place it would appear. We would love to know what is causing such an unusual behaviour, so if you could report back if the logs contain info about this bug, we would really appreciate it.

  12. karlek commented on Aug 4, 2016

    @karlek
    Author

    I'm a new user of ipfs, and haven't figured out how to read the output the ipfs log tail yet.
    How do I supply you the logs, i.e which commands should I run to extract the relevant information?

  13. Kubuxu commented on Aug 5, 2016

    @Kubuxu
    Member

    It will be in output of ipfs daemon.

  14. karlek commented on Sep 29, 2016

    @karlek
    Author

    I haven't encountered this bug since it happened. Just to let you know!

  15. whyrusleeping commented on Sep 29, 2016

    @whyrusleeping
    Member

    @karlek Thanks for pinging back on this. I think that i'm going to go ahead and call it closed. Likely cause being a development version of the go compiler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions