What does HackerNews think of stack-overflow-import?

> Today, with a little effort I could generate a fully working code of a rnn based time series forecasting using this tool.

Interesting. IME even for toy problems it generally spits out code which fails to do what I requested for many inputs or at all. Using this tool to solve a problem requires not only that I understand what it spits out but also that I understand how to actually solve the problem, so that I can work out which parts of the rubbish it spits out are broken and ask it to iterate on those.

It doesn't seem frightening or particularly transformative; I'm not even convinced that using it could save more time than not. It's not doing anything radically different to https://github.com/drathier/stack-overflow-import and the latter works better.

The worst part is that just as with tools like Grammarly, the people who would most benefit from a properly working version of such a tool are exactly the people least well placed to understand when and how the output of the tool is wrong.

I welcome evil players attempting to use it: their evil plans will self-destruct in hilarious ways.

See also https://github.com/drathier/stack-overflow-import for stupid Python import tricks.

"from stackoverflow import quick_sort will go through the search results of [python] quick sort looking for the largest code block that doesn’t syntax error in the highest voted answer from the highest voted question and return it as a module. If that answer doesn’t have any valid python code, it checks the next highest voted answer for code blocks."

I once implemented a custom importer as part of a system where the Python interpreter never touched the filesystem.

A friend suggested we try do this after seeing stack-overflow-import (https://github.com/drathier/stack-overflow-import), which queries stackoverflow for some matching result and loads it. For small functions, it's surprisingly effective. You still need Copilot access to use this, unfortunately.

Some other examples:

  >>> from copilot import rock_paper_scissors
  >>> rock_paper_scissors('Rock', 'Scissors')
  'Player 1 wins'

  >>> from copilot import get_current_git_commit
  >>> get_current_git_commit()
  b'29c7e9b138a9247c598b613ca378d103de878e2a\n'

  >>> from copilot import answer_to_everything
  >>> answer_to_everything(1)
  42

Practically, it works through two separate copilot calls. The first one asks copilot to predict the arguments a function would take, then the second one asks it to predict the body of the function. The returned code is then parsed, both to ensure that it is actual valid Python (it almost always is, although it sometimes spits out Python 2 or cuts off due to length limits), and to remove any extra statements that are not part of the function (sometimes it likes to include an `if __name__ == "__main__"` for example).

The function is then wrapped in an error handler that will attempt to include a module if the snippet references an undefined variable. Copilot tends to use the correct libraries, but forgets to import them so this step is necessary to get most things working. We experimented with prompting an `import ` inside the function body, which worked decent but delivered some wonky results when the code needed more than one import or when it needed none at all.

Often enough to inspire this: https://github.com/drathier/stack-overflow-import

Re: #4

https://github.com/drathier/stack-overflow-import

Relevant Python library (https://github.com/drathier/stack-overflow-import) and previous HN discussion (https://news.ycombinator.com/item?id=12868946).