Generative AI, LLMs & Prompt Engineering : INTRODUCTION
Hello,
In this post, I am going to talk about Generative AI, large language models and prompt engineering. This is going to be an introduction post with general concepts and fundamentals. Let’s get it started !
Generative AI
In the beginning, the AI technologies were using for understanding & providing the information. As the time passed, the goal became creating new contents, interpreting the information. GenAI, is a machine learning type, which involves all the algorithms that can create new content based on its learnings. As the name indicates, genAI, is the artificial intelligence technology that can generate new contents based on its trained models. It imitates the creativity of human brain. It tries to generate new content that is also consistent and logical. Please pay attention to word “consistent”. We will talk about it very often in this post.
GenAI creates text, sound and picture… Even some pictures produced by an artificial intelligence are sold as an art piece, they are outcomes of genAI. The critical point is here is the importance of the models using to train the ai. GenAI uses large language models.
Large Language Models : They are a class of foundation models that trained on enormous amount of data to perform a wide range of tasks. For more info about LLMs, you can check the links end of the post.
The difference of genAI and AI is : traditional AI is designed to achieve a goal in a better & cheaper way than a human does. Gen AI is more wide-ranging. Its goal is to create something newer, something that training model doesn’t have.
Before explaining how it works, let me talk about the practical side. Why do we need genAI ? What are the products that use genAI technology ?
GenAI increases productivity and efficiency in a serious way. As I said in the beginning, a company can use genAI to make some processes more automatic. It can be a part of the company’s mostly used apps and can provide flexibility. As it learns and creates new info / content, people can achieve their goals for work faster. Marketing materials, reports of analyze, presentations, mails and even social media contents ! We all know the power of social media, so does the companies ! Instead of creating each content with a big effort, so much time and human creativity, Gen AI makes things fast and easier but also it gives unique results — of course if you are using a good one & good prompts ! Let me give you one example that I use everyday and loving ! I think we all use office apps (most of all use teams) and I think Microsoft apps are a big part of most of the companies. At this point, I hope you met with Copilot ! Copilot is a Digital assistant which aims to provide personalized help to users. It also combines the power of large language models with your data in Microsoft 365 apps to turn your words into the most powerful productivity tool. For example, use Copilot for your Microsoft Teams meetings and let it summarize meetings or draft action items for you & your team members.
If you haven’t tried it yet, I highly recommend to give a chance. You can read more in the link. Another example of gen AI is Codex, which also powers GitHub Copilot.
As a software engineer, I really love to talk about where & how I use genAI in my daily work and how we all can use it. Because I think new technologies mean so much to us since we want to improve ourselves and stay up-to-date when investigating best solutions / approaches for the products. And believe me, it’s funny to see people’s reaction who believe you will replaced by it soon when you say you love AI !
GenAI helps us to write code faster. It analysis our habits and behaves through it. It fastens the writing process of code by its completions, suggestions and helps to write cleaner code. Besides, it provides help on test codes, automatization of complex algorithms and refactoring. It can make deployment and pipeline processes faster and easier.
I want to leave a not for the discussion I mentioned a few lines ago. GenAI is literally amazing. With my both identities as a software engineer and as a person who uses new technologies on a daily basis for different purposes I can say : it can make things faster, easier and effortless for software developers & engineers but can’t do all of our work. Because we think about the products, problems and concepts, declaring the conditions, constraints, edge cases and exceptions not only for the present and for the future. We try to provide best solution by considering all of these points. Designing, developing, maintaining and testing… Our job is not only about coding or creating algorithms. We have so many hats and we wear them asynchronously. As for now, AI can provide help, and fast but can’t do everything we do in the way that we do ! (But of course it also depends what kind of an engineer you are and how much you improve yourself — this is another discussion). You can easily realize that by confusing ChatGPT a bit with your inputs and see how it gives you unexpected / wrong results — even for simple scenarios sometimes.
“AI can not only boost our analytic and decision-
making abilities but also heighten creativity.”
— HARVARD BUSINESS REVIEW
Now, let’s talk about how genAI works :
Generative AI models work by using neural networks to identify patterns from large sets of data, then create new content.
-A big data cluster is needed to train the model. It can be made of texts, pictures etc.
-After the model is ready, it needs to be trained. → Foundation Models
-As next step, when the model is trained it creates new content. The new content is a compound of the model that is used in training model.
To make results more reliable, consistent and detailed, reprocessing might be needed.
The difficulty of this learning process is : data. The genAI models need relatable and high-quality data set in training process. Collecting this kind of data set — specially in health sector- is very difficult and expensive. Also, to prevent production of similar outputs there is need of serious variety of data. This also requires hard work.
Some of the more complex models are difficult and expensive and time consuming when it comes to calculation. It requires important resources and specialty and this can be overcast for small-ranged businesses.
There is also an ethic side of using genAI, but it’s a huge part and this is a more technical post so I’m not going to talk about it deeply. Gen AI, can be used for creating fake news, fake contents etc. I’m leaving some links if you’re interested and also I will publish a separate content about it.
What is Prompt Engineering ?
I loved this picture and wanted to add exact right here. Imo, it represent prompt engineering very well. Please don’t forget to take a look after reading the definition if you didn’t know what it is :)
Prompt engineering is the process of designing and developing prompts that given to the ai systems. This process helps ai models to create more relatable and consistent contents. It has high importance to give expected result through user’s input. A prompt engineer prepares and designs prompts of NLP (Natural Language processing) model’s. It involves formulation of prompts to get more specific and meaningful outputs.
ChatGPT’s answer to question : “What is prompt engineering ?”
“Prompt engineering” refers to the process of designing and refining prompts to interact with language models effectively, particularly in the context of models like OpenAI’s GPT (Generative Pre-trained Transformer) series. Here’s how it works:
- Understanding the Task: Prompt engineering begins with a clear understanding of the task you want the language model to perform. Whether it’s text completion, translation, summarization, or any other NLP task, defining the task precisely is crucial.
- Crafting Prompts: Prompts are the input provided to the language model to generate the desired output. Crafting prompts involves selecting the right words, phrases, or questions that convey the task to the model effectively. This often requires experimentation and iteration to find the most effective prompts.
- Testing and Refinement: After crafting initial prompts, they are tested against the language model to see how well they perform the desired task. This iterative process involves refining prompts based on the model’s responses and adjusting them to improve performance.
- Considering Model Biases and Limitations: Language models, like GPT, can exhibit biases and limitations in their outputs. Prompt engineering involves considering these factors and designing prompts to mitigate biases and ensure accurate and unbiased responses.
- Iterative Improvement: Prompt engineering is an ongoing process that may involve continuous refinement based on feedback, new data, or changes in the task requirements.
Prompt engineering is essential for effectively leveraging language models in various applications, ensuring that they produce accurate, relevant, and contextually appropriate outputs for the intended task.
Here are some key concepts and potential approaches for building a General AI:
- Learning and Adaptation: A key aspect of AGI would be its ability to learn from and adapt to its environment.
This could involve techniques such as reinforcement learning, where the AI learns to take actions to maximize
some notion of cumulative reward. - Reasoning and Problem-Solving: AGI would need advanced reasoning and problem-solving abilities to tackle
a wide range of tasks. This might involve techniques from symbolic AI, such as logic-based reasoning or
symbolic manipulation, combined with probabilistic reasoning methods. - Perception and Understanding: AGI would need to perceive and understand its environment, including natural
language understanding, computer vision, and sensory processing. This could involve deep learning
techniques for processing sensory data and extracting relevant information. - Autonomy and Decision-Making: AGI would have a degree of autonomy and decision-making capability,
allowing it to make complex decisions in real-time based on its understanding of the environment and its goals.
This might involve hierarchical planning and decision-making architectures. - Self-Awareness and Consciousness: Some researchers speculate that AGI may require some form of self-
awareness or consciousness to truly exhibit human-like intelligence. However, the nature of consciousness
and whether it's necessary for AGI remains a topic of philosophical debate. - Safety and Ethical Considerations: Building AGI also raises important questions about safety, ethics, and
control. Researchers are exploring ways to ensure that AGI systems are aligned with human values, safe to deploy, and controllable by their creators.
With this post, I wanted to make an entrance to the AI world, which I started to work deeply and loved so far. I hope you liked the content and please share any suggestion or feedback with me. See you in next post ! :)
Links to have an eye on :
https://learn.microsoft.com/en-us/ai/playbook/technology-guidance/generative-ai/working-with-llms/
https://www.elastic.co/what-is/large-language-models
https://www.oracle.com/artificial-intelligence/generative-ai/what-is-generative-ai/
https://www.ibm.com/topics/large-language-models
https://www.cloudflare.com/learning/ai/what-is-large-language-model
