Built a text-only baseline: trained a Random Forest on ~90,000 resolved Polymarket questions (YES/NO). Features: TF-IDF (word ngrams, optional char ngrams) + a few cheap flags (date/number/%/currency, election/macro/M&A keywords). Result: ~80% accuracy on 15.000 held-out data/questions (plus decent Brier/logloss after calibration). Liked the idea played a bit more with differnt data sets and did some cross validation with Kalshi data and saw similar results. Now having this running with paper money and competing with stat of the art LLM's as benchmakrs. Lets see. Currently looks like just from the formulation of the question at polymarket (in the given data set) we can predict with 80% accurarcy if it's a YES or NO. Happy to share further insights or get feedback if someone tried smth similar? Source of the paper trading. Model is called "mystery:rf-v1": Agent Leaderboard | Oracle Markets. Did not publish accuary so far there. submitted by /u/No_Syrup_4068 [link] [comments]

I’m a researcher currently trying to replicate published results, and I’m running into reproducibility issues more often than I expected. I’m trying to calibrate whether this is “normal” or a sign I’m missing something fundamental. I have been careful about all the parameter as stated in papers. Despite that, I’m still seeing noticeable deviations from reported numbers—sometimes small but consistent gaps, sometimes larger swings across runs. For example, I was trying to replicate “Machine Theory of Mind” (ICML 2018), and I keep hitting discrepancies that I can’t fully understand. My labmates also tried to replicate the paper they were not able to replicate results even closely. What are the papers you tried but couldn’t replicate no matter what you did? submitted by /u/ArtVoyager77 [link] [comments]

I've been spending time with Apache ADBC (Arrow Database Connectivity) and DuckLake (lakehouse architecture using DuckDB) to read columnar data. I realized XGBoost took Arrow tables as a data input and I was able to pass arrow tables with little memory overhead to train. I also wanted to try to not use scikit-learn so I built a train and test split function with PyArrow instead. ADBC also allows you to stream larger than memory data and train a model in the right circumstances. submitted by /u/empty_cities [link] [comments]

I'm working on a research project where I've gotten to the point of confirmation and I'm working on the proof. The POC works and the results give extremely strong evidence supporting the proposed method across various datasets. Here's the heart of the problem: I'm not in academia, I've never attempted publication, and I have limited credentials. I'm in the public sector with close relationships with certain academic organizations and national labs, as well as a host of experienced folks in the operational workspace. The research is self-driven and self-motivated but is built off of years of personal experience and a literal ton of white papers, so I'm aware of the SOTA and other similar approaches (which will be included in the paper). I'd like to reach out to some folks in various capacities, maybe even reach out to the local university, to ask for guidance, recommendations, and review. I'm absolutely open to bringing in a partner for co-authorship as long as they contribute or provide mentorship. I just have zero sense as to the risk of doing so. I don't feel like theft is a common problem but theft is a spectrum--it could happen at any point with any level of granularity. I understand that it might sound like I'm conflating IP/copyright/patent theft but I'm not. I want other people to use the proposed method, to add on to it, to enhance it, to reference it in other work, or to just use it operationally, but to do so after it's been published or made available. If anyone has any advice on this, I'd love to hear it. submitted by /u/WadeEffingWilson [link] [comments]

Link: https://arxiv.org/abs/2602.14814 Authors: Julien Siems, Riccardo Grazzi, Kirill Kalinin, Hitesh Ballani, Babak Rahmani Abstract: Over the last years, state-tracking tasks, particularly permutation composition, have become a testbed to understand the limits of sequence models like Transformers and RNNs (linear and non-linear). However, these are often sequence-to-sequence tasks: learning to map actions (permutations) to states, which is incompatible with the next-token prediction setting commonly used to train language models. We address this gap by converting permutation composition into code via REPL traces that interleave state-reveals through prints and variable transformations. We show that linear RNNs capable of state-tracking excel also in this setting, while Transformers still fail. Motivated by this representation, we investigate why tracking states in code is generally difficult: actions are not always fully observable. We frame this as tracking the state of a probabilistic finite-state automaton with deterministic state reveals and show that linear RNNs can be worse than non-linear RNNs at tracking states in this setup. https://preview.redd.it/9cjies2580kg1.png?width=1080&format=png&auto=webp&s=e5e534d329bbdbf3d705e811c473ada55d503d20 submitted by /u/Yossarian_1234 [link] [comments]

I’ve been thinking about an emerging structural issue in generative AI. Model quality is improving rapidly. Creation cost is decreasing. Inference is becoming cheaper. But discovery mechanisms haven’t evolved at the same pace. As generative systems scale, the amount of produced content increases superlinearly. Ranking, filtering and relevance models often remain engagement-driven rather than quality-driven. From a machine learning perspective, I’m curious: Do we see discovery and relevance modeling becoming the next major bottleneck in generative ecosystems? Specifically: – Are current ranking systems fundamentally misaligned with user value? – Is engagement still the right optimization objective? – Could smaller, curated relevance models outperform large engagement-optimized feeds? Would appreciate perspectives from people working on recommender systems or ranking models. submitted by /u/Opposite-Alfalfa-700 [link] [comments]

Hi everyone, I've been diving deep into sparse architectures for vision transformers, and I'm incredibly impressed with the potential of SparseFormer to solve the O(n²) compute bottleneck, especially for commercial applications like data labeling and industrial inspection. It feels like this is where the industry is heading for efficiency, and it seems to have more commercial potential than it's currently given credit for, especially with the push towards multimodal models. Is anyone here working with or researching SparseFormer? Curious to hear thoughts on its commercial viability versus other sparse MoE approaches for vision tasks. submitted by /u/SR1180 [link] [comments]

Various venues offer, or have in the past offered, the opportunity to submit short papers, often with a four pages page limit. This is currently true of the ACL. Short papers are not long papers, and there are usually explicit requirements as to how they should be treated differently by reviewers. See for example http://aclrollingreview.org/cfp section on short papers. Question to anyone who has submitted short papers in the past, do you think your paper was reviewed fairly as a short paper? I know we've all had some bad experiences with subletting any kind of paper, but do you think on average the reviewers understood the assignment and evaluated your work based on the criteria for short papers? I think it's true that ICLR used to have a short papers track and removed it. Does anyone know why it was removed? submitted by /u/Efficient_Ad_6772 [link] [comments]

I'm a 2nd year PhD student and looking to broaden my collaboration circle and what better than this community. I primarily work on developing frameworks for fairness (imaging models, LM) (evaluation/mitigation for clinical deployment) but really open for boarder topics. If there's a possibility we can connect and work on something exciting (for a publication in conf or a workshop), would be great. If you have hold of a dataset which will be useful we can make it formal with our institutes. looking forward to hearing from brilliant minds! submitted by /u/ade17_in [link] [comments]

I just want to ask PhDs in AI on this sub, how much does your supervisor support your phd ? In term of research output, how much help do you get from your supervisor? Only ambigious direction (e.g. Active Learning/RL for architecture X)? Or more details idea, like the research gap itself? If you meet a certain problem (e.g. cannot solve X because too hard to solve), do they give you any help, like potential solution direction to try, or just tell you "please do something about it"? How often do their suggestion actually help you? If they don't help much, do they ask their post doc or other student to collaborate/help you solve the problem? Do they have KPI for you? (E.g. number of finished work per year?) In term of networking/connection, how much do he/she help you? submitted by /u/_karma_collector [link] [comments]

Hello everyone! I am writing here to share a library I am currently developing for research use that filled a niche for me in the Equinox/JAX eco-system: eqx-learn. I am using Equinox as the foundation for my radio-frequency modelling library ParamRF, and I have absolutely loved the mixed OO/functional style. However, for my research, I require classical ML models (specifically PCA and Gaussian Process Regression), but could not find an Equinox-native library in the ecosystem that was as straight-forward and consistent as scikit-learn. eqx-learn aims to address this, with a JAX-based take on the scikit-learn API. All models in the library are ultimately Equinox Module's, and can be fit using the library's free "fit" function. The design is such that models simply "advertise" their capabilities by implementing specific methods (e.g. solve(X, y), condition(X, y), loss(), and the "fit" function then fits/trains the model accordingly. I believe that this de-coupling of capabilities vs fitting algorithm fits the JAX style better, and also has lots of potential. At the moment, eqx-learn addresses all my research needs, but I thought it may be useful to share the library online to advertise that it exists, and mention that I am happy to accept PRs for additional models and fitting algorithms! Although there are no docs, there are short examples in the repo :). Happy coding! Cheers, Gary submitted by /u/gvcallen [link] [comments]

Hi I got 3 official reviews. OA: 2/2.5/2.5 (average OA is 2.33) and Confidence: 4/4/3 (average Confidence is 3.67) Thoughts? submitted by /u/srkrrr [link] [comments]

Hi I am preparing for a interview at an AI Lab for LLM inference team with a systems role, not MLE. I have been told I will have an LLM inference related coding round, a design round and an inference optimization related discussion. I have been extensively preparing for these. My Prep for coding is learning to code from scratch the following: SelfAttention, Transformer block, BPE tokenizer, Sampling methods, LV Cache, Bean Search. For other two interviews, I am just studying all the inference design and bottlenecks and old/new work done to eliminate them. I would love to hear if anyone has had similar interview and can share experiences. submitted by /u/dividebyzero74 [link] [comments]

I've tried to use a Transformer decoder architecture to model a sequence of user actions. Unlike an item_id paradigm where each interaction is described by the id of the item the user interacted with, I need to express the interaction through a series of attributes. For example "user clicked on a red button on the top left of the screen showing the word Hello", which today I'm tokenizing as something like [BOS][action:click][what:red_button][location:top_left][text:hello]. I concatenate a series of interactions together, add a few time gap tokens, and then use standard CE to learn the sequential patterns and predict some key action (like a purchase 7 days in the future). I measure success with a recall@k metric. I've tried a buch of architectures framed around gpt2, from standard next token prediction, to weighing the down funnel action more, to contrastive heads, but I can hardly move the needle compared to naive baselines (i.e. the user will buy whatever they clicked on the most). Is there any particular architecture that is a natural fit to the problem I'm describing? submitted by /u/adjgiulio [link] [comments]

The paper was accepted as a spotlight poster at ICML for 2025. For industry, I know that when it comes to time series forecasting, many non faang companies still use ARIMA due to resource cost and efficiency, and they focus on stationary data. I wonder if this model can be a good alternative that can be implemented. Worth noting that TimeBase is benchmarked on long-horizon tasks (96–720 steps), so if your ARIMA usage is for short-term forecasting, the comparison is less direct. What are your thoughts? Their code is public on github, I provided the link here submitted by /u/Whatever_635 [link] [comments]

I am tired of reading all these clearly LLM generated ‘I implemented XYZ in python’ and nonsensical long replies on this subreddit. They add absolutely zero value and just creates meaningless noise. Can we block these posts and replies? submitted by /u/Playful-Fee-4318 [link] [comments]

I have a strong background in ML theory (did a Ph.D. in the field) but I'm out of the loop on the current NLP state-of-the-art. I'm looking for a "roadmap" that respects a PhD-level understanding of math/optimization while skipping "Intro to Python" style tutorials. The end goal isn't academia but more of industry / research roles, maybe. If you had to design a 4-week "crash course" for someone who already understands backprop but hasn't touched a Transformer, what repos or advanced courses would you include? Going over some seminal papers? Is building from scratch (like NanoGPT) a good idea? submitted by /u/meni_s [link] [comments]

Basically titles says it all... I gave the paper a 6 in ICLR, but it ended up being rejected. Just wondering if this is normal? Should I review the paper and pretend it's my first time reading it? Btw, I'm not an expert in that field; the topic is from one of my collaborations. submitted by /u/famous-BlueRaincoat [link] [comments]

Hi all, Do you have a guess of what is the average number of interviews people make until getting a job offer in ML in the US? I made 23 interviews in the last ~8 months without an offer. I don't know if they find my experience outdated, or if my background is actually okay but they keep constantly choosing someone who worked in a job recently, or if there is a problem in the way I communicate or something else. Between 2020 and 2023, I worked as a Data Scientist for ~3 years. I put what I did during this period here • Curated high-quality question–answer pairs from company documents and fine-tuned an LLM (RoBERTa) for extractive question answering. This resulted in a 20% improvement in exact match score. • Trained, optimized, and evaluated deep learning model to predict whether changes in documents need to be reported. Experimented with MLflow and deployed it as a REST API. • Fine-tuned a BERT-based sentence transformer and built an NLP pipeline to extract key topics from company documents. Deployed and integrated the model into an application to deliver actionable document insights. • Designed and implemented end-to-end ETL pipelines with Python, Spark, and SQL to ingest data from different document sources, extract the right data from these documents, and apply various data/text preprocessing methods to ensure data quality, diversity, and compatibility with downstream machine learning models. • Built, optimized, and deployed a deep learning pipeline to classify the regulatory questions into correct categories and integrated it into an application which saved the department approximately $1,500,000 After 2023, I started my Master of Science program in Computer Science in T20 university in the US. I graduated in May 2025. I did an agentic AI project like this: • Built a multi-agent data analytics chatbot using GPT-4 and LangGraph to orchestrate specialized LangChain tools for file parsing, automated statistical analysis, anomaly detection, and data visualization. • Implemented production-ready infrastructure with authentication, session management, file management, caching, and rate limiting. • Implemented backend API with FastAPI and containerized deployment on AWS EC2 using Docker and Docker Compose. submitted by /u/Zealousideal-Egg1354 [link] [comments]

submitted by /u/MzCWzL [link] [comments]