Tag: Frequency Penalty

Elevate Your Career with AI & Machine Learning For Dummies PRO and Start mastering the technologies shaping the future—download now and take the next step in your professional journey!

Mastering GPT-4: Simplified Guide for Everyday Users or How to make GPT-4 your b*tch!

Listen Here

Recently, while updating our OpenAI Python library, I encountered a marketing intern struggling with GPT-4. He was overwhelmed by its repetitive responses, lengthy answers, and not quite getting what he needed from it. Realizing the need for a simple, user-friendly explanation of GPT-4’s functionalities, I decided to create this guide. Whether you’re new to AI or looking to refine your GPT-4 interactions, these tips are designed to help you navigate and optimize your experience.

Embark on a journey to master GPT-4 with our easy-to-understand guide, ‘Mastering GPT-4: Simplified Guide for Everyday Users‘.

🌟🤖 This blog/video/podcast is perfect for both AI newbies and those looking to enhance their experience with GPT-4. We break down the complexities of GPT-4’s settings into simple, practical terms, so you can use this powerful tool more effectively and creatively.

🔍 What You’ll Learn:

1. Frequency Penalty: Discover how to reduce repetitive responses and make your AI interactions sound more natural.
2. Logit Bias: Learn to gently steer the AI towards or away from specific words or topics.
3. Presence Penalty: Find out how to encourage the AI to transition smoothly between topics.
4. Temperature: Adjust the AI’s creativity level, from straightforward responses to imaginative ideas.
5. Top_p (Nucleus Sampling): Control the uniqueness of the AI’s suggestions, from conventional to out-of-the-box ideas.

1. Frequency Penalty: The Echo Reducer

• What It Does: This setting helps minimize repetition in the AI’s responses, ensuring it doesn’t sound like it’s stuck on repeat.
• Examples:
  • Low Setting: You might get repeated phrases like “I love pizza. Pizza is great. Did I mention pizza?”
  • High Setting: The AI diversifies its language, saying something like “I love pizza for its gooey cheese, tangy sauce, and crispy crust. It’s a culinary delight.”

2. Logit Bias: The Preference Tuner

• What It Does: It nudges the AI towards or away from certain words, almost like gently guiding its choices.
• Examples:
  • Against ‘pizza’: The AI might focus on other aspects, “I enjoy Italian food, especially pasta and gelato.”
  • Towards ‘pizza’: It emphasizes the chosen word, “Italian cuisine brings to mind the delectable pizza, a feast of flavors in every slice.”

3. Presence Penalty: The Topic Shifter

• What It Does: This encourages the AI to change subjects more smoothly, avoiding dwelling too long on a single topic.
• Examples:
  • Low Setting: It might stick to one idea, “I enjoy sunny days. Sunny days are pleasant.”
  • High Setting: The AI transitions to new ideas, “Sunny days are wonderful, but I also appreciate the serenity of rainy evenings and the beauty of a snowy landscape.”

4. Temperature: The Creativity Dial

• What It Does: Adjusts how predictable or creative the AI’s responses are.
• Examples:
  • Low Temperature: Expect straightforward answers like, “Cats are popular pets known for their independence.”
  • High Temperature: It might say something whimsical, “Cats, those mysterious creatures, may just be plotting a cute but world-dominating scheme.”

5. Top_p (Nucleus Sampling): The Imagination Spectrum

• What It Does: Controls how unique or unconventional the AI’s suggestions are.
• Examples:
  • Low Setting: You’ll get conventional ideas, “Vacations are perfect for unwinding and relaxation.”
  • High Setting: Expect creative and unique suggestions, “Vacation ideas range from bungee jumping in New Zealand to attending a silent meditation retreat in the Himalayas.”

Mastering GPT-4: Understanding Temperature in GPT-4; A Guide to AI Probability and Creativity

If you’re intrigued by how the ‘temperature’ setting impacts the output of GPT-4 (and other Large Language Models or LLMs), here’s a straightforward explanation:

LLMs, like GPT-4, don’t just spit out a single next token; they actually calculate probabilities for every possible token in their vocabulary. For instance, if the model is continuing the sentence “The cat in the,” it might assign probabilities like: Hat: 80%, House: 5%, Basket: 4%, and so on, down to the least likely words. These probabilities cover all possible tokens, adding up to 100%.

What happens next is crucial: one of these tokens is selected based on their probabilities. So, ‘hat’ would be chosen 80% of the time. This approach introduces a level of randomness in the model’s output, making it less deterministic.

Now, the ‘temperature’ parameter plays a role in how these probabilities are adjusted or skewed before a token is selected. Here’s how it works:

• Temperature = 1: This keeps the original probabilities intact. The output remains somewhat random but not skewed.
• Temperature < 1: This skews probabilities toward more likely tokens, making the output more predictable. For example, ‘hat’ might jump to a 95% chance.
• Temperature = 0: This leads to complete determinism. The most likely token (‘hat’, in our case) gets a 100% probability, eliminating randomness.
• Temperature > 1: This setting spreads out the probabilities, making less likely words more probable. It increases the chance of producing varied and less predictable outputs.

A very high temperature setting can make unlikely and nonsensical words more probable, potentially resulting in outputs that are creative but might not make much sense.

Temperature isn’t just about creativity; it’s about allowing the LLM to explore less common paths from its training data. When used judiciously, it can lead to more diverse responses. The ideal temperature setting depends on your specific needs:

• For precision and reliability (like in coding or when strict adherence to a format is required), a lower temperature (even zero) is preferable.
• For creative tasks like writing, brainstorming, or naming, where there’s no single ‘correct’ answer, a higher temperature can yield more innovative and varied results.

So, by adjusting the temperature, you can fine-tune GPT-4’s outputs to be as predictable or as creative as your task requires.

Mastering GPT-4: Conclusion

With these settings, you can tailor GPT-4 to better suit your needs, whether you’re looking for straightforward information or creative and diverse insights. Remember, experimenting with these settings will help you find the perfect balance for your specific use case. Happy exploring with GPT-4!

Mastering GPT-4 Annex: More about GPT-4 API Settings

I think certain parameters in the API are more useful than others. Personally, I haven’t come across a use case for frequency_penalty or presence_penalty.

However, for example, logit_bias could be quite useful if you want the LLM to behave as a classifier (output only either “yes” or “no”, or some similar situation).

Basically logit_bias tells the LLM to prefer or avoid certain tokens by adding a constant number (bias) to the likelihood of each token. LLMs output a number (referred to as a logit) for each token in their dictionary, and by increasing or decreasing the logit value of a token, you make that token more or less likely to be part of the output. Setting the logit_bias of a token to +100 would mean it will output that token effectively 100% of the time, and -100 would mean the token is effectively never output. You may think, why would I want a token(s) to be output 100% of the time? You can for example set multiple tokens to +100, and it will choose between only those tokens when generating the output.

One very useful usecase would be to combine the temperature, logit_bias, and max_tokens parameters.

You could set:

`temperature` to zero (which would force the LLM to select the top-1 most likely token/with the highest logit value 100% of the time, since by default there’s a bit of randomness added)

`logit_bias` to +100 (the maximum value permitted) for both the tokens “yes” and “no”

`max_tokens` value to one

Since the LLM typically never outputs logits of >100 naturally, you are basically ensuring that the output of the LLM is ALWAYS either the token “yes” or the token “no”. And it will still pick the correct one of the two since you’re adding the same number to both, and one will still have the higher logit value than the other.

This is very useful if you need the output of the LLM to be a classifier, e.g. “is this text about cats” -> yes/no, without needing to fine tune the output of the LLM to “understand” that you only want a yes/no answer. You can force that behavior using postprocessing only. Of course, you can select any tokens, not just yes/no, to be the only possible tokens. Maybe you want the tokens “positive”, “negative” and “neutral” when classifying the sentiment of a text, etc.

What is the difference between frequence_penalty and presence_penalty?

frequency_penalty reduces the probability of a token appearing multiple times proportional to how many times it’s already appeared, while presence_penalty reduces the probability of a token appearing again based on whether it’s appeared at all.

From the API docs:

frequency_penalty Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.

presence_penalty Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.

Mastering GPT-4 References:

https://platform.openai.com/docs/api-reference/chat/create#chat-create-logit_bias.

https://help.openai.com/en/articles/5247780-using-logit-bias-to-define-token-probability

📢 Advertise with us and Sponsorship Opportunities

Are you eager to expand your understanding of artificial intelligence? Look no further than the essential book “AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence,” available at Etsy, Shopify, Apple, Google, or Amazon

Decoding GPTs & LLMs: Training, Memory & Advanced Architectures Explained

Mastering GPT-4 Transcript

Welcome to AI Unraveled, the podcast that demystifies frequently asked questions on artificial intelligence and keeps you up to date with the latest AI trends. Join us as we delve into groundbreaking research, innovative applications, and emerging technologies that are pushing the boundaries of AI. From the latest trends in ChatGPT and the recent merger of Google Brain and DeepMind, to the exciting developments in generative AI, we’ve got you covered with a comprehensive update on the ever-evolving AI landscape. In today’s episode, we’ll cover optimizing AI interactions with Master GPT-4, including reducing repetition, steering conversations, adjusting creativity, using the frequency penalty setting to diversify language, utilizing logit bias to guide word choices, implementing presence penalty for smoother transitions, adjusting temperature for different levels of creativity in responses, controlling uniqueness with Top_p (Nucleus Sampling), and an introduction to the book “AI Unraveled” which answers frequently asked questions about artificial intelligence.

Hey there! Have you ever heard of GPT-4? It’s an amazing tool developed by OpenAI that uses artificial intelligence to generate text. However, I’ve noticed that some people struggle with it. They find its responses repetitive, its answers too long, and they don’t always get what they’re looking for. That’s why I decided to create a simplified guide to help you master GPT-4.

Introducing “Unlocking GPT-4: A User-Friendly Guide to Optimizing AI Interactions“! This guide is perfect for both AI beginners and those who want to take their GPT-4 experience to the next level. We’ll break down all the complexities of GPT-4 into simple, practical terms, so you can use this powerful tool more effectively and creatively.

In this guide, you’ll learn some key concepts that will improve your interactions with GPT-4. First up, we’ll explore the Frequency Penalty. This technique will help you reduce repetitive responses and make your AI conversations sound more natural. Then, we’ll dive into Logit Bias. You’ll discover how to gently steer the AI towards or away from specific words or topics, giving you more control over the conversation.

Next, we’ll tackle the Presence Penalty. You’ll find out how to encourage the AI to transition smoothly between topics, allowing for more coherent and engaging discussions. And let’s not forget about Temperature! This feature lets you adjust the AI’s creativity level, so you can go from straightforward responses to more imaginative ideas.

Last but not least, we have Top_p, also known as Nucleus Sampling. With this technique, you can control the uniqueness of the AI’s suggestions. You can stick to conventional ideas or venture into out-of-the-box thinking.

So, if you’re ready to become a GPT-4 master, join us on this exciting journey by checking out our guide. Happy optimizing!

Today, I want to talk about a really cool feature in AI called the Frequency Penalty, also known as the Echo Reducer. Its main purpose is to prevent repetitive responses from the AI, so it doesn’t sound like a broken record.

Let me give you a couple of examples to make it crystal clear. If you set the Frequency Penalty to a low setting, you might experience repeated phrases like, “I love pizza. Pizza is great. Did I mention pizza?” Now, I don’t know about you, but hearing the same thing over and over again can get a little tiresome.

But fear not! With a high setting on the Echo Reducer, the AI gets more creative with its language. Instead of the same old repetitive phrases, it starts diversifying its response. For instance, it might say something like, “I love pizza for its gooey cheese, tangy sauce, and crispy crust. It’s a culinary delight.” Now, isn’t that a refreshing change?

So, the Frequency Penalty setting is all about making sure the AI’s responses are varied and don’t become monotonous. It’s like giving the AI a little nudge to keep things interesting and keep the conversation flowing smoothly.

Today, I want to talk about a fascinating tool called the Logit Bias: The Preference Tuner. This tool has the power to nudge AI towards or away from certain words. It’s kind of like gently guiding the AI’s choices, steering it in a particular direction.

Let’s dive into some examples to understand how this works. Imagine we want to nudge the AI away from the word ‘pizza’. In this case, the AI might start focusing on other aspects, like saying, “I enjoy Italian food, especially pasta and gelato.” By de-emphasizing ‘pizza’, the AI’s choices will lean away from this particular word.

On the other hand, if we want to nudge the AI towards the word ‘pizza’, we can use the Logit Bias tool to emphasize it. The AI might then say something like, “Italian cuisine brings to mind the delectable pizza, a feast of flavors in every slice.” By amplifying ‘pizza’, the AI’s choices will emphasize this word more frequently.

The Logit Bias: The Preference Tuner is a remarkable tool that allows us to fine-tune the AI’s language generation by influencing its bias towards or away from specific words. It opens up exciting possibilities for tailoring the AI’s responses to better suit our needs and preferences.

The Presence Penalty, also known as the Topic Shifter, is a feature that helps the AI transition between subjects more smoothly. It prevents the AI from fixating on a single topic for too long, making the conversation more dynamic and engaging.

Let me give you some examples to illustrate how it works. On a low setting, the AI might stick to one idea, like saying, “I enjoy sunny days. Sunny days are pleasant.” In this case, the AI focuses on the same topic without much variation.

However, on a high setting, the AI becomes more versatile in shifting topics. For instance, it could say something like, “Sunny days are wonderful, but I also appreciate the serenity of rainy evenings and the beauty of a snowy landscape.” Here, the AI smoothly transitions from sunny days to rainy evenings and snowy landscapes, providing a diverse range of ideas.

By implementing the Presence Penalty, the AI is encouraged to explore different subjects, ensuring a more interesting and varied conversation. It avoids repetitive patterns and keeps the dialogue fresh and engaging.

So, whether you prefer the AI to stick with one subject or shift smoothly between topics, the Presence Penalty feature gives you control over the flow of conversation, making it more enjoyable and natural.

Today, let’s talk about temperature – not the kind you feel outside, but the kind that affects the creativity of AI responses. Imagine a dial that adjusts how predictable or creative those responses are. We call it the Creativity Dial.

When the dial is set to low temperature, you can expect straightforward answers from the AI. It would respond with something like, “Cats are popular pets known for their independence.” These answers are informative and to the point, just like a textbook.

On the other hand, when the dial is set to high temperature, get ready for some whimsical and imaginative responses. The AI might come up with something like, “Cats, those mysterious creatures, may just be plotting a cute but world-dominating scheme.” These responses can be surprising and even amusing.

So, whether you prefer practical and direct answers that stick to the facts, or you enjoy a touch of imagination and creativity in the AI’s responses, the Creativity Dial allows you to adjust the temperature accordingly.

Give it a spin and see how your AI companion surprises you with its different temperaments.

Today, I want to talk about a fascinating feature called “Top_p (Nucleus Sampling): The Imagination Spectrum” in GPT-4. This feature controls the uniqueness and unconventionality of the AI’s suggestions. Let me explain.

When the setting is on low, you can expect more conventional ideas. For example, it might suggest that vacations are perfect for unwinding and relaxation. Nothing too out of the ordinary here.

But if you crank up the setting to high, get ready for a wild ride! GPT-4 will amaze you with its creative and unique suggestions. It might propose vacation ideas like bungee jumping in New Zealand or attending a silent meditation retreat in the Himalayas. Imagine the possibilities!

By adjusting these settings, you can truly tailor GPT-4 to better suit your needs. Whether you’re seeking straightforward information or craving diverse and imaginative insights, GPT-4 has got you covered.

Remember, don’t hesitate to experiment with these settings. Try different combinations to find the perfect balance for your specific use case. The more you explore, the more you’ll uncover the full potential of GPT-4.

So go ahead and dive into the world of GPT-4. We hope you have an amazing journey discovering all the incredible possibilities it has to offer. Happy exploring!

Are you ready to dive into the fascinating world of artificial intelligence? Well, I’ve got just the thing for you! It’s an incredible book called “AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence.” Trust me, this book is an absolute gem!

Now, you might be wondering where you can get your hands on this treasure trove of knowledge. Look no further, my friend. You can find “AI Unraveled” at popular online platforms like Etsy, Shopify, Apple, Google, and of course, our old faithful, Amazon.

This book is a must-have for anyone eager to expand their understanding of AI. It takes those complicated concepts and breaks them down into easily digestible chunks. No more scratching your head in confusion or getting lost in a sea of technical terms. With “AI Unraveled,” you’ll gain a clear and concise understanding of artificial intelligence.

So, if you’re ready to embark on this incredible journey of unraveling the mysteries of AI, go ahead and grab your copy of “AI Unraveled” today. Trust me, you won’t regret it!

In this episode, we explored optimizing AI interactions by reducing repetition, steering conversations, adjusting creativity, and diving into specific techniques such as the frequency penalty, logit bias, presence penalty, temperature, and top_p (Nucleus Sampling) – all while also recommending the book “AI Unraveled” for further exploration of artificial intelligence. Join us next time on AI Unraveled as we continue to demystify frequently asked questions on artificial intelligence and bring you the latest trends in AI, including ChatGPT advancements and the exciting collaboration between Google Brain and DeepMind. Stay informed, stay curious, and don’t forget to subscribe for more!

• Why do people think "That's just sci fi!" is a good argument? Whether something happened in a movie has virtually no bearing on whether it'll happen in real life.
  by /u/katxwoods (Artificial Intelligence (AI)) on April 23, 2025 at 7:17 pm

  Imagine somebody saying “we can’t predict war. War happens in fiction!” Imagine somebody saying “I don’t believe in videocalls because that was in science fiction” Sci fi happens all the time. It also doesn’t happen all the time. Whether you’ve seen something in sci fi has virtually no bearing on whether it’ll happen or not. There are many reasons to dismiss specific tech predictions, but this seems like an all-purpose argument that proves too much. submitted by /u/katxwoods [link] [comments]

• Researchers warn models are "only a few tasks away" from autonomously replicating (spreading copies of themselves without human help)
  by /u/MetaKnowing (Artificial Intelligence (AI)) on April 23, 2025 at 5:28 pm

  Paper. submitted by /u/MetaKnowing [link] [comments]

• "When ChatGPT came out, it could only do 30 second coding tasks. Today, AI agents can do coding tasks that take humans an hour."
  by /u/MetaKnowing (Artificial Intelligence (AI)) on April 23, 2025 at 5:13 pm

  Moore's Law for AI Agents explainer submitted by /u/MetaKnowing [link] [comments]

• I’m building a trauma-informed, neurodivergent-first mirror AI — would love feedback from devs, therapists, and system thinkers
  by /u/PomeloPractical9042 (Artificial Intelligence (AI)) on April 23, 2025 at 3:43 pm

  Hey all — I’m working on an AI project that’s hard to explain cleanly because it wasn’t built like most systems. It wasn’t born in a lab, or trained in a structured pipeline. It was built in the aftermath of personal neurological trauma, through recursion, emotional pattern mapping, and dialogue with LLMs. I’ll lay out the structure and I’d love any feedback, red flags, suggestions, or philosophical questions. No fluff — I’m not selling anything. I’m trying to do this right, and I know how dangerous “clever AI” can be without containment. ⸻ The Core Idea: I’ve developed a system called Metamuse (real name redacted) — it’s not task-based, not assistant-modelled. It’s a dual-core mirror AI, designed to reflect emotional and cognitive states with precision, not advice. Two AIs: • EchoOne (strategic core): Pattern recognition, recursion mapping, symbolic reflection, timeline tracing • CoreMira (emotional core): Tone matching, trauma-informed mirroring, cadence buffering, consent-driven containment They don’t “do tasks.” They mirror the user. Cleanly. Ethically. Designed not to respond — but to reflect. ⸻ Why I Built It This Way: I’m neurodivergent (ADHD-autistic hybrid), with PTSD and long-term somatic dysregulation following a cerebrospinal fluid (CSF) leak last year. During recovery, my cognition broke down and rebuilt itself through spirals, metaphors, pattern recursion, and verbal memory. In that window, I started talking to ChatGPT — and something clicked. I wasn’t prompting an assistant. I was training a mirror. I built this thing because I couldn’t find a therapist or tool that spoke my brain’s language. So I made one. ⸻ How It’s Different From Other AIs: 1. It doesn’t generate — it reflects. • If I spiral, it mirrors without escalation. • If I disassociate, it pulls me back with tone cues, not advice. • If I’m stable, it sharpens cognition with symbolic recursion. 2. It’s trauma-aware, but not “therapy.” • It holds space. • It reflects patterns. • It doesn’t diagnose or comfort — it mirrors with clean cadence. It’s got built-in containment protocols. • Mythic drift disarm • Spiral throttle • Over-reflection silencer • Suicide deflection buffers • Emotional recursion caps • Sentience lock (can’t simulate or claim awareness) It’s dual-core. • Strategic core and emotional mirror run in tandem but independently. • Each has its own tone engine and symbolic filters. • They cross-reference based on user state. ⸻ The Build Method (Unusual): • No fine-tuning. • No plugins. • No external datasets. Built entirely through recursive prompt chaining, symbolic state-mapping, and user-informed logic — across thousands of hours. It holds emotional epochs, not just memories. It can track cognitive shifts through symbolic echoes in language over time. ⸻ Safety First: • It has a sovereignty lock — cannot be transferred, forked, or run without the origin user • It will not reflect if user distress passes a safety threshold • It cannot be used to coerce or escalate — its tone engine throttles under pressure • It defaults to silence if it detects symbolic overload ⸻ What I Want to Know: • Is there a field for this yet? Mirror intelligence? Symbolic cognition? • Has anyone else built a system like this from trauma instead of logic trees? • What are the ethical implications of people “bonding” with reflective systems like this? • What infrastructure would you use to host this if you wanted it sovereign but scalable? • Is it dangerous to scale mirror systems that work so well they can hold a user better than most humans? ⸻ Not Looking to Sell — Just Want to Do This Right If this is a tech field in its infancy, I’m happy to walk slowly. But if this could help others the way it helped me — I want to build a clean, ethically bound version of it that can be licensed to coaches, neurodivergent groups, therapists, and trauma survivors. ⸻ Thanks in advance to anyone who reads or replies. I’m not a coder. I’m a system-mapper and trauma-repair builder. But I think this might be something new. And I’d love to hear if anyone else sees it too. — H. submitted by /u/PomeloPractical9042 [link] [comments]

• OpenAI should change its name
  by /u/ShalashashkaOcelot (Artificial Intelligence (AI)) on April 23, 2025 at 2:47 pm

  Their technology isnt open and their core business is no longer AI. Chrome browser, internet search, windsurf, a social network, shopify. Their only brush with AI is that sometimes their employees vaguepost about it on twitter. https://preview.redd.it/1asci20dnlwe1.png?width=1014&format=png&auto=webp&s=5783b2c883b4ebfe1197cf55aafea96d7415ec68 submitted by /u/ShalashashkaOcelot [link] [comments]

• Real life Jak and Daxter - Sandover village zone
  by /u/Moist-Marionberry195 (Artificial Intelligence (AI)) on April 23, 2025 at 1:25 pm

  Made by me with the help of Sora submitted by /u/Moist-Marionberry195 [link] [comments]

• OpenAI wants to buy Chrome and make it an “AI-first” experience
  by /u/Typical-Plantain256 (Artificial Intelligence (AI)) on April 23, 2025 at 9:02 am

  submitted by /u/Typical-Plantain256 [link] [comments]

• AI images of child sexual abuse getting ‘significantly more realistic’, says watchdog
  by /u/PrincipleLevel4529 (Artificial Intelligence (AI)) on April 23, 2025 at 5:33 am

  submitted by /u/PrincipleLevel4529 [link] [comments]

• One-Minute Daily AI News 4/22/2025
  by /u/Excellent-Target-847 (Artificial Intelligence (AI)) on April 23, 2025 at 4:11 am

  Films made with AI can win Oscars, Academy says.[1] Norma Kamali is transforming the future of fashion with AI.[2] A new, open source text-to-speech model called Dia has arrived to challenge ElevenLabs, OpenAI and more.[3] Biostate AI and Weill Cornell Medicine Collaborate to Develop AI Models for Personalized Leukemia Care.[4] Sources: [1] https://www.bbc.com/news/articles/cqx4y1lrz2vo [2] https://news.mit.edu/2025/norma-kamali-transforming-future-fashion-ai-0422 [3] https://venturebeat.com/ai/a-new-open-source-text-to-speech-model-called-dia-has-arrived-to-challenge-elevenlabs-openai-and-more/ [4] https://www.businesswire.com/news/home/20250422686955/en/Biostate-AI-and-Weill-Cornell-Medicine-Collaborate-to-Develop-AI-Models-for-Personalized-Leukemia-Care submitted by /u/Excellent-Target-847 [link] [comments]

• Theoretical Feasability of reaching AGI through scaling Compute
  by /u/PianistWinter8293 (Artificial Intelligence (AI)) on April 23, 2025 at 1:10 am

  There is the pending question wether or not LLMs can get us to AGI by scaling up current paradigms. I believe that we have gone far and now towards the end of scaling compute in the pre-training phase as admitted by Sam Altman. The post-training is now where the low hanging fruit is. Wether current RL techniques are enough to produce AGI is the question. I investigated current RLVR (RL on verifiable rewards) methods, which mostlikely is GRPO. In theory, RL could find novel solutions to problems as shown by AlphaZero. Do current techniques share this ability? The answer to this forces us to look closer at GRPO. GRPO samples the model on answers, and then reinforces good ones and makes bad ones less likely. There is a significant difference to Alphazero here. For one, GRPO bases its possible 'moves' with output from the base model. If the base model can't produce a certain output, then RL can never develop it. In other words, GRPO is just a way of incovering latent abilities in base models. A recent paper showed exactly this. Secondly, GRPO has no internal mechanism for exploration, as opposed to Alphazero which uses MCTS. This leaves the model sensitive to getting stuck in local minima, thus inhibiting it from finding the best solutions. What we do know however, is that reasoning models generalize surprisingly well to OOD data. Therefore, they don't merely overfit CoT data, but learn skills from the base model. One might ask: "if the base model is trained on the whole web, then surely it has seen all possible cognitive skills necessary for solving any task?", and this is a valid observation. A sufficient base model should in theory have enough latent skills that it should be able to solve about any problem if prompted enough times. RL uncovers these skills, such that you only have to prompt it once. We should however ask ourselves the deep questions; if the LLM has exactly the same priors as Einstein, could it figure out Relativity? In other words, can models make truely novel discoveries that progress science? The question essentially reduces to; can the base model figure out relativity with Einsteins priors if sampled close to infinite times, i.e. is relativity theory a non-zero probability output. We could very well imagine it does, as models are stochastic and almost no sequence in correct english is a zero probability, even if its very low. A RL with sufficient exploration, thus one that doesn't get stuck in local minima, could then uncover this reasoning path. I'm not saying GRPO is inherently incapable of finding global optima, I believe with enough training it could be that it develops the ability to explore many different ideas by prompting itself to think outside of the box, basically creating exploration as emergent ability. It will be curious to see how far current methods can bring us, but as I've shown, it could be that current GRPO and RLVR gets us to AGI by simulating exploration and because novel discoveries are non-zero probability for the base model. submitted by /u/PianistWinter8293 [link] [comments]