Generative AI - Part 7 - Code Generation and Other Specific Tasks
Investigation
Code Generation
Code generation is a specific topic of interest to software developers, so I wanted to look at the generative AI capabilities in this area separately.
Autocode
This tool can build Javascript-powered bots, scripts and APIs with code generation and instant deployment. Apart from standard services to build these specific web parts, it also has a Software Development Environment (SDE) that has connectors for many popular APIs such as Discord, Airtable, Github, Gmail and of course OpenAI. Thus it is possible to use the ChatGPT API in the bots. There are also connectors to some other AI platforms such as Stability AI and Notion.
Thus it is not really an AI Code Generator but a platform to facilitate the use of AI-powered bots and scripts, using some of the available AI platforms.
It has Standard, Enterprise and Business modes priced per seat.
Airtable
Airtable is an application development platform that makes use of AI to build custom applications on top of databases. Its AI feature uses OpenAI’s generative capabilities to power most of the UI elements.
Stepsize
Stepsize AI analyses a product team's Jira or Linear activity to generate digestible updates on the progress of teams and projects every week. Thus a generative AI system focusing on software products.
Mintlify
This is a system that automates software documentation. Works with Visual Studio Code and IntelliJ.
Phind
This is touted as the search engine for software engineers. The user describes what to find in a prompt and the software returns typical uses and code samples. I tried it with a commonly used view in ScrollView and got some useful code excerpts. It also provides the source addresses for the suggested code samples.
Bloop
This is a tool which connects to the GitHub account of the user and uses that as a basis to respond to coding prompts. It combines GPT-4 with semantic search. Here is an example where I asked the tool to get me some coding for word tokenisation using Swift.
REPL.it AI
REPL.it is a standard Python environment that can also be used on the web and provides the user with an interactive testbed for programming tasks. I’ve used it previously for some software training, where REPL.it was used to present and solve software assignments.
REPL.it has now introduced an AI mode. It is possible to use it in autocomplete mode and has an embedded GPT in the REPL.it prompt.
Imagica
This is an interesting application that starts a project with a textual prompt from the user and builds an AI app that does the task described in the prompt.
I tried it with the concept of the cosine similarity between words, so I provided a prompt asking for this. Imagica produced nodes for an AI application that calculates what I described, presumably using the definition of the task that can be found on the Internet.
There are lots of templates that could be directly used to create specific applications (maths applications, language and translation etc.)
Code Llama
This is a Meta initiative for code generation. It has been produced by performing specific training on a Llama 2 model with a large code dataset. It supports several programming languages like C++ and Python.
Codeium
A set of extensions that can be installed in several Integrated Development Environments to have real-time code or comment suggestions, based on an LLM for code. It supports several languages. I tried it for XCode and Swift with some success.
CV Preparation
Kickresume.ai
This is a web-based application that claims to use OpenAI’s ChatGPT 3.5 as a conversation tool. The workflow is based on the user selecting one of the resume templates (only a few are available in free mode, much more in paid modes), and answering a few questions in dialogue/chat style.
This is quite useful when filling up the resume, since the application uses ChatGPT fo fill sections based on the field of study and other parameters you supply. the resulting resume can be downloaded as a PDF file.
The AI-generated sections use typical statements based on the profession and skills, thus giving you a partially complete CV with the right keywords used, then you can add the specifics the AI will not be able to generate. It also has some additional perks, such as being able to generate potential interview questions for the user.
Interior Design
RoomGPT
A free and open-source tool which lets the user upload a picture of a room, select some designs and produces furnished rooms in the style/design selected.
Character Chatbots
Character.ai
This is a strange site (in Beta for now). It is used to create chatbots for different “characters”. The web site says it is based on LLMs etc., the standard technologies, but there are no specifics on the used models, apart from the information that these are specifically trained for chat. The site mentions limited context windows, so the typical restrictions about the model “forgetting things” apply.
I tried an adventure game character, which seemed to fine-tune nicely based on my responses and I got a lot of fun out of my dialogue with it, remembering the time when I used “The Adventure Game” on a Univac 1100 computer around 1982. It sounds very similar to the original adventure game in terms of tone and language, but of course generated by the LLM behind the site.
There are lots of different characters. I did not have time to check all of them, but tried a few. A Pair Programmer character suggested a few SwiftUI views when I asked for specific tasks to be performed. It could be useful, but the site has caveats about the characters not always producing meaningful stuff, so I’m not sure if there was any fine-tuning or reinforcement training applied. The site also allows you to create your own characters and there is a community built around it.
Music Generation
This is an area with comprehensive research and has been active for a long time. The methods used are similar to the ones used for text generation, but of course the domain is really complex. You can find an extensive survey in this paper.
Google Music LM
A relatively new experimental system from Google and is accepting users to test the system. I have put my name on the waiting list but have not yet received a positive response.
It receives a text prompt from the user and produces actual music. The prompts could be very detailed and descriptive (e.g. “A fusion of reggaeton and electronic dance music, with a spacey, otherworldly sound. Induces the experience of being lost in space, and the music would be designed to evoke a sense of wonder and awe, while being danceable.”) and more detail could result in higher-quality music.
Soundraw
Soundraw can produce original music from a list of genres, moods, themes, tempos and instruments. They do not explain the AI techniques they use, but judging from their description, it looks like they have a dataset of sampled music from each genre/mood and use it to generate music similar to the user selection.
Artificial Intelligence Virtual Artist (AIVA)
This seems to be the AI music generator of choice. It uses deep learning to analyze and learn from a vast database of music, enabling it to extrapolate data in order to create original music pieces in a variety of different styles and genres. The user selects a generation profile (which could be a genre or a style of music), then the key signature and duration.
The generated music sounds really good, but I haven’t used it long enough to notice any repetition or stagnancy. I’ve attached two samples below.
MuseNet
This is an OpenAI research tool that can generate 4-minute compositions. The underlying model is similar to GPT-2 (so a pretty old version of OpenAI’s models). The user can provide some notes from an existing piece and ask for a particular style.
OpenAI has also produced a “cosine similarity graph” for composers, giving us an idea of how similar several composers are.
Here is an example where Musenet was given the prompt “Bon Jovi and the first 6 notes of Chopin Op. 27, No. 2” and generated the linked music.
(… to be continued)