Skip to content

Latest commit

 

History

History
296 lines (206 loc) · 7.46 KB

index.md

File metadata and controls

296 lines (206 loc) · 7.46 KB

Getting the Most Out of AI Coding Assistants

James Collier Marine Guyot

Who are you?

Briefly introduce yourself:

  • What is your name?
  • What do you work on?
  • Why did you join the course?

Workshop goals

  • What AI powered tools are available?

    • What can these tools do well?
    • What don't they do well?

  • Tips and tricks, pros and cons, lots of discussion

  • Ethical, security, and privacy issues

  • Practical applications (try things out)

What is "AI"?

  • A chat interface? Is ELIZA an AI?
  • Statistical inference
  • Machine learning
  • Symbolic reasoning
  • Large Language Models (LLMs)

What is an LLM?

  • A statistical model of language => multi-dimensional distribution
  • Model inferred ("trained") from a "large" volume of data
  • Training result is a high-dimensional statistical distribution
  • Sampling from the distribution produces a "token"
  • Training often optimizes for observing "human like" sequences of sampled tokens

Software development tasks

  • Building programs from scratch
  • Adding features to existing programs
  • Fixing bugs in existing programs
  • Refactoring existing programs
  • Collaborating with other programmers
  • Testing
  • Deployment
  • Maintenance
  • Code review
  • Learning new tools/libraries/techniques/...
  • Trying to understand code
  • Design

What tools do we already use for these tasks?

  • Editors (with LSP)
  • Static analysis (linters, type checkers)
  • Dynamic analysis (valgrind, profilers)
  • Tests
  • Version control
  • Build tools
  • Debuggers
  • Code generation
  • Logging and monitoring
  • Dependency management
  • Project management
  • Documentation
  • Databases

What can AI powered tools do?

Code generation

                       --{{0}}--

In-editor autocomplete

!?In editor auto-complete

Generate from comment description

!?Generate code from comment

Generate in a chat

Write a docstring for the Token class and all of it's methods

!?Generate code in chat

Generate code from unit tests

Generate a function called make_tokens based on the tests provided in test_make_tokens

!?Generate code from tests

Generate unit tests from code

Generate pytest style unit tests for the make_tokens function and the Token class

!?Generate tests from code

Translation between languages (e.g. Python -> Perl)

Convert the tokenize function into idiomatic Perl

!?Converl Python to Perl

Translation between technologies (e.g. SQL -> SQLAlchemy)

See bad_sql_results.txt

Pair programming

  • Refactoring
  • Explaining code

!?Explain and refactor

Search engine

What libraries are available to parse YAML in Python? What are their pros and cons?

!?What YAML libraries are available in Python

An example of bad results

What libraries are available to decode YAML documents in the Elm language?

!?Example of bad search engine results

Code review

Review the code in inference.py and make concise suggestions about quality, security, readability, and performance.

!?Code review

Bug fixing

There's a bug in the program where it doesn't print the last generated word. Can you suggest fixes?

!?Code review

Ethics, security, and privacy

Brainstorm

Demo: ChatGPT

Step 1: Set custom context in ChatGPT.

you are a machine learning developper in python. You use lightning-pytorch. You work in research field.

!?ChatGPT set custom context

Step 2: Describe the problem.

I have eye tracking time-series data for patients with a certain age. I want to classify this time series based on age. Please suggest a list of good deep learning models.

Step 3: Generate the python code to test this model.

Generate python code with the LSTM model

!?ChatGPT Machine learning code

Demo: q

!?q tool

Your turn: Explore the tools

Goal: Make a real-world open-source contribution (or equivalent).

Now is a good time to install anything you would like to try out locally. Also a good time to log into (or create accounts for) any services you might want to use:

Use the tools for real

Try to understand the advantages and limitations of each tool.

Some possibilities you could consider:

Do it again

  1. Go to Replit and sign up
  2. Import a github project using the link: https://github.com/vibbits/ai-coding-course

You will need to find out:

  • How do I run a Haskell project?
  • How do I build a Haskell project?
  • What does the code do?
  • Can you add tests to the project?
  • Code review the project
  • Does the code have bugs?
  • Can you add a feature?

Discussion

  1. What worked well? Which use-case/tool combinations worked well?

  2. Maybe useful / situational / it depends

  3. Definitely not useful?

Conclusion

Do not give online models private or sensitive information

Do not blindly trust the output

Explore your options: things in this domain change very quickly

Are you planning to use any of the tools in this workshop?