For this particular case, I use a ncurses based file manager: ranger
Instead of z, I use https://github.com/wting/autojump, which is written in Python. This has the benefit that you can call it from outside the shell, too.
Since I also love ranger as a file manager, I wrote my own integration of both: https://github.com/fdw/ranger-autojump . It remembers where I went in ranger, and I can also call autojump in ranger.