Abstract: Chain-of-Thought (CoT) prompting has proven highly effective for enhancing complex reasoning in Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs). Yet, it struggles in complex spatial reasoning tasks. Nonetheless, human cognition extends beyond language alone, enabling the remarkable capability to think in both words and images. Inspired by this mechanism, we propose a new reasoning paradigm, Multimodal Visualization-of-Thought (MVoT). It enables visual thinking in MLLMs by generating image visualizations of their reasoning traces. To ensure high- quality visualization, we introduce token discrepancy loss into autoregressive MLLMs. This innovation significantly improves both visual coherence and fidelity. We validate this approach through several dynamic spatial reasoning tasks. Experimental results reveal that MVoT demonstrates competitive performance across tasks. Moreover, it exhibits robust and reliable improvements in the most challenging scenarios where CoT fails. Ultimately, MVoT establishes new possibilities for complex reasoning tasks where visual thinking can effectively complement verbal reasoning. Arxiv link: https://arxiv.org/pdf/2501.07542 submitted by /u/imadade [link] [comments]

I have a simple dataset that I want to train a prediction model on for a pretty low stakes project (more for fun), but I have no experience training ML models. Simple linear regression didn't have great performance when I tried it and I suspect there is a more complex interaction between the variables. Training Dataset: 25K observations of 5 numerical predictor variables with one 1 numerical outcome variable. What is the best AutoML platform that I can run this with minimal code, just to see if ML models can perform better than simple regression can? Thanks! submitted by /u/Equal-Option-2287 [link] [comments]

I have been working for the past 6 weeks on this kaggle competition. My issue is that I have run out of ideas, trying everything from TTA (test time augmentation) to model architectures. My best solution is to train LightGBM, CatBoost, Neural Networks on targets which are the risk scores estimated by the survival models: Kaplan-Meier, Nelson-Aalen and CoxPH and 2 more targets, which are transformations of the time-to-event column. The only area that remains "uncharted" is domain-specific stuff. My question is whether someone on this subreddit has worked specifically on survival analysis, HCT survival, both or something similar and has domain expertise that goes beyond purely ML approaches (which models work the best, which CV scheme etc.). submitted by /u/TechNerd10191 [link] [comments]

Hey r/MachineLearning! Last week, Microsoft released Phi-4, a 14B open-source model that rivals OpenAI's GPT-4-o-mini. I managed to find & fix 4 bugs impacting its output quality. You might remember me previously from fixing 8 bugs in Google's Gemma model! 🙂 I'm going to walk you through how I found & fixed the bugs. Phi-4's benchmarks were amazing, however many users reported weird or just wrong outputs. Since I maintain the open-source project called 'Unsloth' (fine-tuning LLMs 2x faster with 70% less VRAM) with my brother, I firstly tested Phi-4 for inference and found many errors. Our GitHub repo: https://github.com/unslothai/unsloth This time, the model had no implementation issues (unlike Gemma 2) but did have problems in the model card. For my first inference run, I randomly found an extra token which is obviously incorrect (2 eos tokens is never a good idea). Also during more runs, I found there was an extra assistant prompt which is once again incorrect. And, lastly, from past experience with Unsloth's bug fixes, I already knew fine-tuning was wrong when I read the code. These bugs caused Phi-4 to have some drop in accuracy and also broke fine-tuning runs. Our fixes are now under review by Microsoft to be officially added to Hugging Face. We uploaded the fixed versions to https://huggingface.co/unsloth/phi-4-GGUF Here’s a breakdown of the bugs and their fixes: 1. Tokenizer bug fixes The Phi-4 tokenizer interestingly uses <|endoftext|> as the BOS (beginning of sentence), EOS (end of sentence) and PAD (padding) tokens. The main issue is the EOS token is wrong - it should be <|im_end|>. Otherwise, you will get <|im_end|><|endoftext|> in generations. 2. Fine-tuning bug fixes The padding token should be a designated pad token like in Llama (<|finetune_right_pad_id|>) or we can use an untrained token - for example we use <|dummy_87|>, fixing infinite generations and outputs. 3. Chat template issues The Phi-4 tokenizer always adds an assistant prompt - it should only do this if prompted by add_generation_prompt. Most LLM serving libraries expect non auto assistant additions, and this might cause issues during serving. We dive deeper into the bugs in our blog: https://unsloth.ai/blog/phi4 Do our Fixes Work? Yes! Our fixed Phi-4 uploads show clear performance gains, with even better scores than Microsoft's original uploads on the Open LLM Leaderboard. https://preview.redd.it/d8hew26e06ce1.png?width=2366&format=png&auto=webp&s=173c23feacc625566271470839fe7a5e25eb860e Some redditors even tested our fixes to show greatly improved results in: Example 1: Multiple-choice tasks https://preview.redd.it/qx50pkq706ce1.png?width=1579&format=png&auto=webp&s=437da2cabdbf98ef5a8b8cbdc5592907a20e2316 Example 2: ASCII art generation https://preview.redd.it/sw1o3a3yt4de1.png?width=2326&format=png&auto=webp&s=fc6bfc45d14134d45f332ba58bbd1de049f5776b We also made a Colab notebook fine-tune Phi-4 completely for free using Google's free Tesla T4 (16GB) GPUs: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Phi_4-Conversational.ipynb Thank you for reading this long post and hope you all found this insightful! If you have any questions, please feel free to ask! 🙂 How I found the bugs: I first downloaded the original Phi-4 from https://huggingface.co/microsoft/phi-4, and tested inference out. Weirdly I found <|im_start|>assistant<|im_sep|> to be appended at the even with add_generation_prompt = False in Hugging Face, so I theorized there was a chat template problem. Adding assistant prompts by default can break serving libraries. And yes, https://huggingface.co/microsoft/phi-4/blob/f957856cd926f9d681b14153374d755dd97e45ed/tokenizer_config.json#L774 had by default added the assistant prompt - I first fixed this! I then found <|endoftext|> to be used for the BOS, EOS and PAD tokens, which is a common issue amongst models - I ignored the BOS, since Phi-4 did not have one anyways, but changed the PAD token to <|dummy_87|>. You can select any of the tokens since they're empty and not trained. This counteracts issues of infinite generations during finetuning. For Llama-fication, I used torch.allclose to confirm all tensors are in fact equivalent. I also used some fake random data to check all activations are also mostly similar bitwise. I also uploaded the model to the HF Open LLM Leaderboard to confirm if the original Phi-4 arch and the new Llama-fied models are equivalent. Finally I verified all finetuning runs with Unsloth in a Colab Notebook to confirm all runs were correct. submitted by /u/danielhanchen [link] [comments]

Dear All, I am a UG student and I want to submit my manuscript to any of these two journals; the work is on the interplay of privacy and explainability in machine learning (would be more than happy to send you the arXived version of the same on request). I have previously published in a very reputed workshop of EMNLP and came to know that mostly ML nowadays is a conference-centric discipline. I want to know which of these two will be better to submit my work (due to the length and scope, I am unable to submit to conferences this time). I cannot submit it to tmlr until it's Scopus-indexed and not considering AIJ and Machine Learning Journal at this moment. I just want to make sure that if the paper gets accepted, I want this to be at least comparable with a borderline A* paper (in terms of the so-called prestige of the venue). Also, let me know if you have any other suggestions; I am new to journals and I appreciate your opinion. P.S.: My guide slightly prefers PR to JAIR due to its higher IF but nevertheless, he is open JAIR or any other Scopus-indexed journals as long as it is comparable with at least a borderline A* or very strong A conf paper as said. submitted by /u/RepresentativeOk7956 [link] [comments]

Hello folks, Just started contributing into the writing for research, previously I just used to experiment and work on results, tables and plots. Obviously using AI to generate content for paper is unethical and wrong in many aspect. But what about using it to correct your grammar and comprehension. Technically it will also considered as AI written but is it okay to do this atleast in the literature review, introduction and description for the experiment? To be honest, I like writing and when I asked AI (chatgpt and others) I see that it is much easier to read and interpret, which I think is good for the community and on the other hand, it may be considered unethical by many. When I ran a 'AI-text detector' on many of paper I'm using as reference from last 1~ year, I usually get a 50-70% score. What do you all think? submitted by /u/ade17_in [link] [comments]

Can explainable AI balance competing needs in job recommendation systems? Models like OKRA, powered by GNNs, deliver stakeholder-specific insights - text explanations for candidates, skill alignment for recruiters, and visualizations for companies. They address biases (e.g. rural underrepresentation) and challenges like integrating explanations with source data (CVs, vacancies). Future directions focus on refining explanation coherence, fairness metrics, and real-world validation, pushing explainable multi-stakeholder AI towards equitable, context-aware job matching. We unpack "Explainable Multi-Stakeholder Job Recommender Systems" by Roan Schellingerhout here: https://www.shaped.ai/blog/decoding-job-recommendations-the-future-of-explainable-multi-stakeholder-ai submitted by /u/skeltzyboiii [link] [comments]

There is this dataset (won't link here as I don't want my kaggle and reddit associated) with a few input features (5-6) used to predict one target value. But one of the features is basically perfectly linearly correlated with the target (>0.99). An example would be data from a trucking company with a single model of trucks: Target: truck fuel consumption / year Features: driver's age, tires type, truck age, DISTANCE TRAVELED / year Obviously in average the fuel consumption will be linearly proportional with the nb of miles traveled. I mean normally you'd just use that to calculate a new target like fuel/distance. Yet not a single person/notebook did this kind of normalization. So everyone's model has >.99 accuracy, as that one feature drowns out everything else. Is that something other people noticed: more and more the code looks fine (Data loading, training many types of models), maybe thanks to LLMs. But the decision making process is often quite bad? submitted by /u/ToThePastMe [link] [comments]

Interesting new text-to-speech system that tackles mathematical content by combining OCR and language models. The key innovation is treating mathematical notation as a specialized language that needs translation, using a multi-stage pipeline to convert equations into natural speech. Technical approach: * Custom OCR model trained specifically on mathematical documents * T5-based language model fine-tuned for math-to-text translation * Three-stage pipeline: recognition → translation → synthesis * Integration with LaTeX parsing for handling complex mathematical typography Key results: * 95% accuracy in mathematical expression recognition * Successful handling of complex notation including fractions, integrals, matrices * User testing showed preference over existing math TTS systems * Natural language output matches human descriptions I think this could be impactful for making technical education more accessible. Being able to convert mathematical documents to clear speech opens up some possibilities for learning and working with technical content. The combination of OCR and NLP seems maybe like a robust approach that could extend beyond just mathematics to other technical domains with specialized notation. I see some limitations around context-dependent notation and complex proofs, but these seem like natural areas for future work rather than fundamental flaws in the approach. TLDR: New TTS system combines specialized OCR and language models to convert mathematical documents to natural speech, achieving 95% accuracy in math recognition and producing human-like descriptions. Full summary is here. Paper here. submitted by /u/Successful-Western27 [link] [comments]

Abstract: “Over more than a decade there has been an extensive research effort of how effectively utilize recurrent models and attentions. While recurrent models aim to compress the data into a fixed-size memory (called hidden state), attention allows attending to the entire context window, capturing the direct dependencies of all tokens. This more accurate modeling of dependencies, however, comes with a quadratic cost, limiting the model to a fixed-length context. We present a new neural long-term memory module that learns to memorize historical context and helps an attention to attend to the current context while utilizing long past information. We show that this neural memory has the advantage of a fast parallelizable training while maintaining a fast inference. From a memory perspective, we argue that attention due to its limited context but accurate dependency modeling performs as a short-term memory, while neural memory due to its ability to memorize the data, acts as a long-term, more persistent, memory. Based on these two modules, we introduce a new family of architectures, called Titans, and present three variants to address how one can effectively incorporate memory into this architecture. Our experimental results on language modeling, common-sense reasoning, genomics, and time series tasks show that Titans are more effective than Transformers and recent modern linear recurrent models. They further can effectively scale to larger than 2M context window size with higher accuracy in needle-in-haystack tasks compared to baselines.” Arxiv: https://arxiv.org/abs/2501.00663 submitted by /u/imadade [link] [comments]

Paper: https://arxiv.org/abs/2501.06252 Abstract Self-adaptive large language models (LLMs) aim to solve the challenges posed by traditional fine-tuning methods, which are often computationally intensive and static in their ability to handle diverse tasks. We introduce Transformer², a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting only the singular components of their weight matrices. During inference, Transformer² employs a two-pass mechanism: first, a dispatch system identifies the task properties, and then task-specific "expert" vectors, trained using reinforcement learning, are dynamically mixed to obtain targeted behavior for the incoming prompt. Our method outperforms ubiquitous approaches such as LoRA, with fewer parameters and greater efficiency. Transformer² demonstrates versatility across different LLM architectures and modalities, including vision-language tasks. Transformer² represents a significant leap forward, offering a scalable, efficient solution for enhancing the adaptability and task-specific performance of LLMs, paving the way for truly dynamic, self-organizing AI systems. Blog Summary: https://sakana.ai/transformer-squared/ GitHub: https://github.com/SakanaAI/self-adaptive-llms submitted by /u/hardmaru [link] [comments]

Hello, I am wondering what is the usual way people search for or discover new papers in ArXiv? Do you just use their search engine? Any tips/hints? submitted by /u/TheDevilIsInDetails [link] [comments]

I am very creative when it comes to adding improvements to my embedding or inference workflows, but I am having problems when it comes to measuring whether those improvements really make the end result better for my use case. It always comes down to gut feeling. How do you all measure... ..if this new embedding model if better than the previous? ..if this semantic chunker is better than a split based one? ..if shorter chunks are better than longer ones? ..if this new reranker really makes a difference? ..if this new agentic evaluator workflow creates better results? Is there a scientific way to measure this? submitted by /u/EnvironmentalPost830 [link] [comments]

I’ve built an LGBM model to classify Parkinson’s patients using a dataset of 2,562 patients with 37 features selected through P value and correlation analysis and my own domain knowledge, questions can be binary, continuous or ordinal eg do they have Urinary Problems yes/no = 0/1, all questions are numerical answers. The dataset was split into 70% training (1,793 samples), 15% validation (384 samples), and 15% hold-out test (385 samples). I performed 5-fold stratified cross-validation on the training set, with approximately 1,434 samples for training and 359 for validation in each fold. The dataset contains 1085 PD patients and 1477 non-PD patients. I think the performance is really good, I'm wondering if anyone has any additional tests or methods to assess whether it's a big fantasy or have I a good model on my hands? .=== Cross-Validation Metrics === Mean F1 Score: 0.8860 ± 0.0210 Mean AUC: 0.9565 ± 0.0095 Mean Recall: 0.8814 ± 0.0239 Mean Precision: 0.8911 ± 0.0251 === Hold-Out Set Metrics === F1 Score: 0.8875 AUC: 0.9505 Recall: 0.8957 Precision: 0.8795 submitted by /u/Disastrous_Ad9821 [link] [comments]

Hi. I am working on a project which requires me to identify sentiments from English text and then quantify those sentiments as percentage. I need to run six models on the text and then compare the classifications. So far, I have explored some BERT and RoBERTa based models in Huggingface, which are trained using the GoEmotion dataset provided by Google. I was curios, are there any better models that I am missing? Please leave the name of some pre-trained models which can give some good results. TIA! submitted by /u/Big_Baguette17 [link] [comments]

I have an imbalanced dataset of about 100,000 rows 1500 of them are of defaultes, which has more than 1000 features and has lots of missing values. Alsothe name of the features are anonymized (like bureau_1, bureau_2) so it also seems difficult and these feaures had max correlation of 0.1 with the target variable I want to predict the probability of a customer who might default based on the data but am not able to make much progress in terms of metrics like recall (0.25), f1 and auprc. I have tried various tree based models like lgbm, xgboost etc with various class balance attributes but its not giving me that good of results. If anyone of you have such prior experience of handling such datasets, can you suggest me what should i do in terms of feature engineering, modelling etc. All of your help will mean a lot to me. submitted by /u/Chuggleme [link] [comments]

I am working on getting object tracking working for a sports game, and would like to take the next step and be able to detect when an action has taken place (like a soccer ball has gone out of bounds, of a bowling ball has hit pins, or a ball has been thrown (as opposed to a practice throw or pump fake). I have been doing these by hand coding heuristics for how to detect, but I would like to be more flexible. All the libraries for action recognition seem to be about human skeleton actions. That makes me think I am looking at the wrong problem space. Is there existing art for taking locations of objects over time and learning when an action is taking place given training data? submitted by /u/needaname1234 [link] [comments]

Some people say that AI research scientists (PhD holders) are pretty much irreplaceable because of their ability to push the boundaries of knowledge and come up with groundbreaking methods and algorithms. But let’s be real—tech companies don’t need a ton of researchers, especially if their work doesn’t directly boost profits. On the flip side, Machine Learning Engineers are the ones putting those algorithms into action, scaling systems, and keeping production pipelines running—all things that directly bring in the $$$. That’s why some people think MLE roles will grow faster than AI research scientist roles in the future. What do you think? Are there trends or experiences you’ve seen that suggest one of these roles will be more in demand down the line? I'm currently a PhD student by the way. For a fair comparison, let’s assume both roles are at a FAANG company. submitted by /u/wonder-why-I-wonder [link] [comments]

I wanted to apply clustering algorithms on a similarity matrix. Is that possible? If yes, how? submitted by /u/fordperfect14 [link] [comments]

What are the approaches to access datasets during training? Are they downloaded to the machines/pods before starting the training process or are they network mounted? Similarly for large models how do the models are deployed for inference? ( for auto scaling or for updating the model version) submitted by /u/badseed79 [link] [comments]

Does anyone have experience with the NannyML library? I am having a difficult time fully grasping the reasoning behind forcing users to split data into chunks. I haven’t seen any other drift detection libraries do this. Let’s say I have a model on which I would like to perform drift detection. I have some reference feature data from some time ago, and some analysis feature data from today. It seems that to use this library, I am required to split these 2 datasets into arbitrary chunks (they recommend at least 6). I would actually like to perform drift detection by comparing both sets of data to each other as a whole, however. This doesn’t work - forcing the chunk size to 1 results in the upper_threshold value to be set to 0 and every feature gets alerted on. It seems like the library is geared towards comparing some number of reference datasets across time vs some equal number of analysis datasets across time… but doesn’t work if there is only have 1 analysis dataset (for 1 date). What am I missing here? Any help much appreciated! submitted by /u/ObviousYam144 [link] [comments]

Hi Community, First, I want to thank you for reading my earlier posts on geometric intuition and receiving with worms! I didn't expect to receive so much good feedback and also different explanations in the comment. I learned so much! Motived by this, I wrote another post for geometric intuition and this time about "Dot Product". Here is the link https://maitbayev.github.io/posts/dot-product/ Let me know what you think submitted by /u/madiyar [link] [comments]

Netflix and Cornell University researchers have exposed significant flaws in cosine similarity. Their study reveals that regularization in linear matrix factorization models introduces arbitrary scaling, leading to unreliable or meaningless cosine similarity results. These issues stem from the flexibility of embedding rescaling, affecting downstream tasks like recommendation systems. The research highlights the need for alternatives, such as Euclidean distance, dot products, or normalization techniques, and suggests task-specific evaluations to ensure robustness. Read the full paper review of 'Is Cosine-Similarity of Embeddings Really About Similarity?' here: https://www.shaped.ai/blog/cosine-similarity-not-the-silver-bullet-we-thought-it-was submitted by /u/skeltzyboiii [link] [comments]

Hi Everyone, I recently noticed most LLM observability providers (Arize AI, Galileo AI, LangSmith) use a simple LLM-as-a-Judge framework to detect hallucinations for deployed RAG applications. There's a ton of hallucination detection research out there like this or this survey, so I wondered why aren't any of these providers offering more advanced research-backed methods? Given the user input query, retrieved context, and LLM output, one can pass this data to another LLM to evaluate whether the output is grounded in the context. So I benchmarked this LLM-as-a-Judge framework against a couple of research methods on the HaluBench dataset - and turns out they're probably right! A strong base model with chain-of-thought prompting seems to work better than various research methods. Code here. Partial results: Framework Accuracy F1 Score Precision Recall Base (GPT-4o) 0.754 0.760 0.742 0.778 Base (GPT-4o-mini) 0.717 0.734 0.692 0.781 Base (GPT-4o, sampling) 0.765 0.766 0.762 0.770 CoT (GPT-4o) 0.833 0.831 0.840 0.822 CoT (GPT-4o, sampling) 0.823 0.820 0.833 0.808 Fewshot (GPT-4o) 0.737 0.773 0.680 0.896 Lynx 0.766 0.780 0.728 0.840 RAGAS Faithfulness (GPT-4o) 0.660 0.684 0.639 0.736 RAGAS Faithfulness (HHEM) 0.588 0.644 0.567 0.744 G-Eval Hallucination (GPT-4o) 0.686 0.623 0.783 0.517 submitted by /u/MagnoliaPotato [link] [comments]

Hi! A friend and I have been working on a project called SemHash which I wanted to share. We found that text deduplication is more complex than it appears, so we built this to simplify the process. Duplicate samples can skew model training, return redundant samples in RAG workflows, reduce generalization, and cause train-test leakage—leading to unreliable results. Techniques like minhash handle exact or near-exact duplicates, but semantic deduplication also catches semantically redundant samples, which we believe is an important aspect of deduplication. Furthermore, it’s not trivial to see why something was removed with minhash, which we also believe is important. For this reason. we’ve added explainability features as well so that you can inspect why something was removed. We already found some interesting results on some well known datasets in our benchmarks which are included in the repo. The package can be installed with pip install semhash, and the basic usage looks like this (this example assumes you have the datasets library installed): from datasets import load_dataset from semhash import SemHash # Load a dataset to deduplicate train = load_dataset("ag_news", split="train")["text"] test = load_dataset("ag_news", split="test")["text"] # Initialize a SemHash instance semhash = SemHash.from_records(records=train) # Deduplicate the train set deduplicated_train = semhash.self_deduplicate().deduplicated # Or deduplicate the test set against the train set deduplicated_test = semhash.deduplicate(records=test).deduplicated I’m very interested in hearing your thoughts on this! Is deduplication a part of your current ML workflows, and if so, what techniques do you use? submitted by /u/Pringled101 [link] [comments]

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. Thanks to everyone for answering questions in the previous thread! submitted by /u/AutoModerator [link] [comments]

For Job Postings please use this template Hiring: [Location], Salary:[], [Remote | Relocation], [Full Time | Contract | Part Time] and [Brief overview, what you're looking for] For Those looking for jobs please use this template Want to be Hired: [Location], Salary Expectation:[], [Remote | Relocation], [Full Time | Contract | Part Time] Resume: [Link to resume] and [Brief overview, what you're looking for] ​ Please remember that this community is geared towards those with experience. submitted by /u/AutoModerator [link] [comments]