Let’s say I have a black-box function that maps inputs to points in an N-dimensional space. The function’s output space may be finite or infinite. Given a set of sampled points obtained from different inputs, I want to estimate how much of the function’s possible output space is covered by my samples. For a simpler case, assume the function returns a single numerical value instead of a vector. By analyzing the range of observed values, I can estimate an interval that likely contains future outputs. If a newly sampled point falls outside this range, my confidence in the estimated range should decrease; if it falls within the range, my confidence should increase. What kind of estimator am I looking for? I appreciate any insights! submitted by /u/Euphoric-Ad1837 [link] [comments]

When our field first developed RNNs, they were the obvious choice for sequential tasks until vanishing/exploding gradients and the inherently unparallelizable backpropagation through time (BPTT) limited their scalability. Years of collective research addressing these issues ultimately birthed the Transformer—massively parallelizable, scalable, and easier to train, marking the revolutionary arrival of the golden age of attention. The Ignored Alternatives State Space Models and parallelizable LSTM variants emerged as potential solutions to the parallelization issues of traditional RNNs, but they sacrificed the ability to generalize to problems in the NC1 complexity class which vanilla RNNs can do, staying within TC0 like Transformers. This isn’t just theoretical—after over 3 years and billions spent optimizing hardware for transformers, these alternatives offered virtually no compelling advantage. The Chain of Thought Contradiction Fast forward to Chain of Thought prompting – suddenly we're training models with elaborate reasoning examples, often including this bizarre theatrical process where LLMs are deliberately trained to make mistakes just to demonstrate correction capabilities. It's computational theater. But DeepSeek's R1 approach is where this paradox becomes undeniable. They're using reinforcement learning to train reasoning chains, which is genuinely innovative, but... Why are we still using Transformers for what is fundamentally a recurrent reasoning process? Let me dissect this architectural mismatch: We're tokenizing chains of thought, severely restricting their expressive potential The reasoning process itself functions as a hidden state WITHOUT ground truth labels (which is actually perfect – otherwise we'd just be training glorified memorization) This scenario logically demands a BPTT-like approach – which would be completely unparallelizable even with Transformers since we lack intermediate labels – yet we're circumventing this entire problem with GRPO and somehow getting spectacular results We're essentially performing recurrent optimization while stubbornly avoiding recurrent architectures. The intellectual contradiction is mind-boggling! It's as if the entire field developed collective amnesia about the fundamental principles of sequential processing that motivated RNNs in the first place. The Billion-Dollar Blindspot Let's cut to the chase: RNNs can solve problems in the NC1 complexity class that Transformers fundamentally cannot. This isn't academic nitpicking—it's about computational expressiveness that directly impacts reasoning capabilities. A Transformer forced to use input sequences as pseudo-RNN states is crippled for reasoning: poor length generalization, inefficient information pruning, and suboptimal cache performance. Yet R1's approach—using reinforcement learning without BPTT—works brilliantly and could resurrect even basic RNNs with superior results. At inference, the process is identical: store state, sample outputs, track probabilities, then adjust based on reasoning quality. So why aren't we applying this to architectures designed for sequential reasoning? This architectural mismatch seems strikingly obvious yet remains unaddressed. Is it infrastructure lock-in? Publication pressure? Or has the field collectively forgotten why recurrent networks were created in the first place? The emperor has no clothes. The question is: who will be the first to point it out? submitted by /u/JirkaKlimes [link] [comments]

I've been diving into TULIP, a new approach for vision-language pretraining that addresses what the authors call the "seeing half a scene" problem in models like CLIP. The key insight is combining contrastive learning with masked feature prediction in a unified framework. Technical approach: * Uses a dual-encoder architecture (ViT + text transformer) similar to CLIP * Introduces "block-wise masking with patch shuffling" - a new visual masking strategy * Combines two training objectives: contrastive learning and masked feature prediction * Leverages both real image-text pairs and synthetic data from diffusion models Key results: * State-of-the-art performance across multiple benchmarks: * 70.8% on ImageNet-1K classification (ViT-B) * 77.6% box AP on COCO detection * 58.3% mIoU on ADE20K segmentation * Shows that neither contrastive learning nor masked prediction alone is sufficient * Works well even with limited text descriptions (10M image-text pairs) * Performance scales effectively with increased model size and pretraining data I think this approach represents an important shift in how we build vision-language models. By forcing models to understand both global image-text relationships and local visual feature relationships, we can create systems with more comprehensive visual understanding. The use of synthetic data to supplement real datasets is also pragmatic - it helps address data scarcity for specific concepts without requiring expensive annotation. The block-wise masking strategy seems particularly clever. Instead of randomly masking individual patches (which can be too easy for models to solve), this approach creates a more challenging pretraining task that encourages understanding of spatial relationships. TLDR: TULIP combines contrastive learning with masked feature prediction to create vision-language models that understand both whole images and their detailed components. It achieves SOTA results across multiple vision tasks and demonstrates effective use of synthetic training data. Full summary is here. Paper here. submitted by /u/Successful-Western27 [link] [comments]

Most implementations of Reinforcement Learning applied to Tetris have been based on hand-crafted feature vectors and reduction of the action space (action-grouping), while training agents on the full observation- and action-space has failed. I created a project to learn to play Tetris from raw observations, with the full action space, as a human player would without the previously mentioned assumptions. It is configurable to use any tree policy for the Monte-Carlo Tree Search, like Thompson Sampling, UCB, or other custom policies for experimentation beyond PUCT. The training script is designed in an on-policy & sequential way and an agent can be trained using a CPU or GPU on a single machine. Have a look and play around with it, it's a great way to learn about MCTS! https://github.com/Max-We/alphazero-tetris submitted by /u/Npoes [link] [comments]

We're excited to share FlashTokenizer, a high-performance tokenizer engine optimized for Large Language Model (LLM) inference serving. Developed in C++, FlashTokenizer offers unparalleled speed and accuracy, making it the fastest tokenizer library available.​ Key Features: Unmatched Speed: FlashTokenizer delivers rapid tokenization, significantly reducing latency in LLM inference tasks.​ High Accuracy: Ensures precise tokenization, maintaining the integrity of your language models.​ Easy Integration: Designed for seamless integration into existing workflows, supporting various LLM architectures.​GitHub Whether you're working on natural language processing applications or deploying LLMs at scale, FlashTokenizer is engineered to enhance performance and efficiency.​ Explore the repository and experience the speed of FlashTokenizer today:​ We welcome your feedback and contributions to further improve FlashTokenizer. https://github.com/NLPOptimize/flash-tokenizer submitted by /u/springnode [link] [comments]

Semi-supervised learning (SSL) leverages abundant unlabeled data alongside limited labeled data to enhance learning. As vision foundation models (VFMs) increasingly serve as the backbone of vision applications, it remains unclear how SSL interacts with these pre-trained models. To address this gap, we develop new SSL benchmark datasets where frozen VFMs underperform and systematically evaluate representative SSL methods. We make a surprising observation: parameter-efficient fine-tuning (PEFT) using only labeled data often matches SSL performance, even without leveraging unlabeled data. This motivates us to revisit self-training, a conceptually simple SSL baseline, where we use the supervised PEFT model to pseudo-label unlabeled data for further training. To overcome the notorious issue of noisy pseudo-labels, we propose ensembling multiple PEFT approaches and VFM backbones to produce more robust pseudo-labels. Empirical results validate the effectiveness of this simple yet powerful approach, providing actionable insights into SSL with VFMs and paving the way for more scalable and practical semi-supervised learning in the era of foundation models. Paper Link submitted by /u/oncecookedpork [link] [comments]

What are some good journals without any publication fee or processing charges? submitted by /u/_My__Real_Name_ [link] [comments]

We've working on a project to predict sentiment of client meeting transcripts into negative, neutral or positive. I'm using Siebert model currently which is roberta large variant to predict sentiment of each speaker sentences (upto 512 tokens as this is its context length) of a transcript and then applying some logic on sentences' preds we're defining whole transcript sentiment. Issue is it is giving around 70% recall and 50% precision. To tackle this we fed neutral predicted transcripts to llama3.1 8b. It improved recall to 90% but precision fell in 20-30% range. I'm looking for ideas/different approaches to tackle this issue. Any suggestions are welcome. submitted by /u/Adi-Sh [link] [comments]

BACKGROUNG: I was using some models to generate tags for music such as genre, mood, and instruments in the music (audio file). The original models were in .pb extension. The models are available on [Essentia models — Essentia 2.1-beta6-dev documentation] and the models I am using are: discogs-effnet-bs64-1 genre_discogs400-discogs-effnet-1 mtg_jamendo_instrument-discogs-effnet-1 mtg_jamendo_moodtheme-discogs-effnet-1 The input and outputs of the models are given in the respective json files which show the classes and the input/output sizes and names. The default .pb models simply use the inbuilt functions: from essentia.standard import ( MonoLoader, TensorflowPredictEffnetDiscogs, TensorflowPredict2D, ) def essentia_feature_extraction(audio_file, sample_rate): #Loading the audio file audio = MonoLoader(filename=audio_file, sampleRate=16000, resampleQuality=4)() # Embedding audio features embeddings = embedding_model(audio) result_dict = {} processed_labels = list(map(process_labels, genre_labels)) # Genre prediction genre_predictions = genre_model(embeddings) result_dict["genres"] = filter_predictions(genre_predictions, processed_labels) # Mood/Theme prediction mood_predictions = mood_model(embeddings) result_dict["moods"] = filter_predictions( mood_predictions, mood_theme_classes, threshold=0.05 ) # Instrument prediction instrument_predictions = instrument_model(embeddings) result_dict["instruments"] = filter_predictions( instrument_predictions, instrument_classes ) return result_dict THE PROBLEM: No matter what audio file I use as input, I consistently get the same output predictions for mood and instruments. The genre predictions are now usually all zero (meaning "unknown genre"). import librosa import numpy as np import tritonclient.http as httpclient def essentia_feature_extraction_triton(audio_file, sample_rate): try: audio, sr = librosa.load(audio_file, sr=16000, mono=True) audio = audio.astype(np.float32) mel_spectrogram = librosa.feature.melspectrogram( y=audio, sr=16000, n_fft=2048, hop_length=512, n_mels=128 ) mel_spectrogram = librosa.power_to_db(mel_spectrogram, ref=1.0) if mel_spectrogram.shape[1] < 96: mel_spectrogram = np.pad( mel_spectrogram, ((0, 0), (0, 96 - mel_spectrogram.shape[1])), mode="constant" ) elif mel_spectrogram.shape[1] > 96: mel_spectrogram = mel_spectrogram[:, :96] mel_spectrogram = np.expand_dims(mel_spectrogram, axis=0).astype(np.float32) with httpclient.InferenceServerClient(url=TRITON_URL) as triton_client: # --- EFFNET DISCOGS (Combined Model) --- input_name = "melspectrogram" genre_output_name = "activations" embedding_output_name = "embeddings" inputs = [httpclient.InferInput(input_name, mel_spectrogram.shape, "FP32")] inputs[0].set_data_from_numpy(mel_spectrogram) outputs = [ httpclient.InferRequestedOutput(genre_output_name), httpclient.InferRequestedOutput(embedding_output_name) ] results = triton_client.infer( model_name=EFFNET_DISCOGS_MODEL_NAME, inputs=inputs, outputs=outputs ) genre_predictions = results.as_numpy(genre_output_name) embeddings = results.as_numpy(embedding_output_name) embeddings = embeddings.astype(np.float32) # --- MOOD PREDICTION --- input_name = "embeddings" output_name = "activations" inputs = [httpclient.InferInput(input_name, embeddings.shape, "FP32")] inputs[0].set_data_from_numpy(embeddings) outputs = [httpclient.InferRequestedOutput(output_name)] mood_predictions = triton_client.infer( model_name=MOOD_MODEL_NAME, inputs=inputs, outputs=outputs ).as_numpy(output_name) # --- INSTRUMENT PREDICTION --- input_name = "embeddings" output_name = "activations" inputs = [httpclient.InferInput(input_name, embeddings.shape, "FP32")] inputs[0].set_data_from_numpy(embeddings) outputs = [httpclient.InferRequestedOutput(output_name)] instrument_predictions = triton_client.infer( model_name=INSTRUMENT_MODEL_NAME, inputs=inputs, outputs=outputs ).as_numpy(output_name) submitted by /u/NotSoAsian86 [link] [comments]

This paper explores the mathematical properties of sliding window clustering, proving several fundamental behaviors that explain why certain clustering approaches succeed or fail. The key technical contribution is a set of mathematical proofs showing that the clustering behavior of sliding windows depends critically on window size and data symmetry properties: Small windows produce flat centroids: They mathematically prove that as window size becomes small relative to signal frequency, cluster centroids approach constant functions Near-symmetric data creates meaningless clusters: When data satisfies f(t) ≈ f(-t), they show clustering becomes essentially random Large windows naturally form interval clusters: They prove that optimal clustering of large sliding windows forms intervals (contiguous chunks of the time series) Formal mathematical framework: The paper establishes theoretical foundations using properties of autocorrelation and similarity measures The main results include: Theorem 1 shows that small windows produce nearly identical, flat cluster centroids Proposition 2 demonstrates that with symmetric periodic signals, windows are assigned to clusters essentially randomly Theorem 3 establishes that with large windows, optimal clusters form intervals Several corollaries extend these results to specific clustering algorithms and data types I think this work explains phenomena many practitioners have observed empirically but couldn't fully explain. When working with sliding windows, I've often noticed that small windows produce uninformative clusters while larger ones tend to identify meaningful temporal segments. Now we have mathematical explanations for why this happens. I think these results could guide better algorithm design for time series analysis. Understanding the mathematical limitations of different window sizes should help researchers avoid approaches that are doomed to fail due to fundamental constraints rather than implementation issues. TLDR: The paper provides mathematical proofs showing that small sliding windows produce flat, meaningless clusters; nearly symmetric data makes clustering ineffective; and large windows naturally form interval-based clusters - explaining why some sliding window clustering approaches work while others fail. Full summary is here. Paper here. submitted by /u/Successful-Western27 [link] [comments]

The finance/AI world is split: Do LLMs have predictive power in trading? Some argue markets are too efficient, too noisy for AI to extract real edge. Others believe AI can uncover hidden patterns beyond human capability. Instead of debating, I built an AI-driven Options Trader to find out. 🔬 The Experiment I designed an algorithm that feeds all major LLMs with every possible data point—spanning technical indicators, news sentiment, options flow, macro signals, and cross-market correlations. Instead of cherry-picking signals, AI conducts a comprehensive cross-analysis across models. The rule is simple: ✅ If all LLMs align on a high-probability trade, we take it. ❌ If uncertainty is high or risk/reward is poor, we sit out. This isn't just another AI trading bot. It's an attempt to quantify AI’s true decision-making power in financial markets—something few have rigorously tested. 🤔 What’s the Edge? AI isn’t distracted by market noise—it operates purely on structured analysis. Instead of relying on one AI model, we use an ensemble approach for robustness. The absence of a trade is as valuable as taking one—avoiding unnecessary risk. 🔍 Research & Real-World Testing I’ll be sharing the results, insights, and unexpected findings in my QuantSignals newsletter. If you're curious about AI x Quant Trading and whether LLMs can truly generate alpha in options trading, sign up and follow this journey. 📩 Follow along here: https://open.substack.com/pub/henryzhang/p/nvda-weekly-combo-analysis-2025-03?r=14jbl6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false What do you think? Are we on the edge of an AI-driven trading revolution, or are markets simply too efficient for LLMs to win? Let’s test it—scientifically. #QuantTrading #AITrading #OptionsTrading #MachineLearning #LLM #FinanceResearch #QuantSignals submitted by /u/henryzhangpku [link] [comments]

I am working on a system that initially used RAG to fetch relevant information, but recently I found better performance using a CAG/Large-context LLM architecture where I do the following: Pull all the relevant data Use Gemini 2 Flash to take the query + the retrieved data and filter it to only the relevant data Pass the filtered data to the most performant LLM for the task to respond to the prompt. The second step helps mitigate what I’ve seen referred to as the “lost in the middle” phenomenon, and distraction. In my case scaling over time is not a major concern as the context window size stays more or less consistent. The problem, and in hindsight it’s quite obvious, is that even after being filtering, the document is still big — and for the filter LLM to output that filtered document takes up to 20s for Gemini 2 flash. That latency isn’t acceptable in the system. I have considered solutions like enumerating all the data in the context window and getting the filter LLM to only output the indices of relevant data, effectively letting us do lossless compression on the output prompt, meaning we can generate the output faster. In my testing (and I’m not sure if this is really an issue) I’ve found that this produces different results for the filter, which concerns me a bit. So I am still a bit stuck on how best to speed up the filter. I’m curious if anyone else here has tried an architecture like this with filtering large context with an LLM/is knowledgeable enough to weigh in? submitted by /u/SlackEight [link] [comments]

Hi r/MachineLearning I'm planning to fine-tune the QWQ-32B model on a custom dataset and would appreciate some guidance from those with experience. My Current Situation: I have a dataset in Alpaca format I'm unsure about the optimal fine-tuning approach for QWQ-32B I do have few questions Can QWQ-32B be effectively fine-tuned using the Alpaca format dataset, or would this be suboptimal? Should I convert my data to use the <think> format instead? If so, would generating a new dataset using DeepSeek or Claude be recommended? Does QWQ-32B support QLoRA fine-tuning, or is full fine-tuning required? I'd appreciate hearing about your experience fine-tuning QWQ-32B, including any challenges faced and helpful configurations or optimization tips. Thank you in advance for any insights! submitted by /u/aadityaura [link] [comments]

Satellite Image Dataset for Cyclone Prediction So I need a satellite image Dataset of any specific Indian state for cyclone prediction. From mausam.imd.gov.in Any idea how to create a traianable dataset from here I would really appreciate the help submitted by /u/Melodic_Bliss [link] [comments]

can anyone send some begineer freindly resources for the score based generative models all videos/blogs/papers which I see are diving directly into the mathematical explanation which is hard to grasp for me. submitted by /u/jiraiya1729 [link] [comments]

Something is odd happening to the science. There is a new paper called "Transformers without Normalization" by Jiachen Zhu, Xinlei Chen, Kaiming He, Yann LeCun, Zhuang Liu https://arxiv.org/abs/2503.10622. They are "selling" linear layer with tanh activation as a novel normalization layer. Was there any review done? It really looks like some "vibe paper review" thing. I think it should be called "parametric tanh activation, followed by useless linear layer without activation" submitted by /u/ivanstepanovftw [link] [comments]

Hey everyone, Our ICCV 2025 paper just got desk-rejected because we included the supplementary material as an appendix in the main PDF, which allegedly put us over the page limit. Given that this year, ICCV required both the main paper and supplementary material to be submitted on the same date, we inferred (apparently incorrectly) that they were meant to be in the same document. For context, in other major conferences like NeurIPS and ACL, where the supplementary deadline is the same as the main paper, it’s completely standard to include an appendix within the main PDF. So this desk rejection feels pretty unfair. Did anyone else make the same mistake? Were your papers also desk-rejected? Curious to hear how widespread this issue is. submitted by /u/hellomellow1 [link] [comments]

IPV-Bench: Evaluating Video Generation Models with Physically Impossible Scenarios Researchers have created a new benchmark called IPV-Bench to evaluate how well video generation models understand basic physics and logic. This benchmark contains 1,000 carefully crafted prompts that test models on their ability to handle physically impossible scenarios across 9 categories including gravity violations, object permanence issues, and logical contradictions. The key methodology included: - Testing models with both "create impossible" prompts (asking for impossibilities) and "avoid impossible" prompts (requesting physically plausible videos) - Evaluating videos through both automated metrics and human assessment - Testing across multiple state-of-the-art models including Sora, Morph-E, WALT, Show-1, Gen-2, Runway, Pika, and LaVie - Developing a detailed taxonomy of impossible physics scenarios Main findings: - Current SOTA models produce physically impossible content 20-40% of the time even when explicitly asked to follow physics laws - Performance was worst on "change impossibilities" and "contact impossibilities" (~50% accuracy) - Different models show different "impossibility profiles" - making distinct types of physical reasoning errors - Strong text understanding doesn't guarantee strong physical reasoning - Human evaluators easily identified these impossibilities, highlighting the gap between AI and human understanding I think this research reveals a fundamental limitation in current video generation systems - they lack the intuitive physics understanding that humans develop naturally. This matters significantly for applications where physical plausibility is important, like simulation, education, or training robotics systems. The benchmark provides a systematic way to measure progress in this area, which will be crucial as these models become more widely deployed. The taxonomy they've developed is particularly useful as it gives us a framework for thinking about different types of physical reasoning failures. I suspect we'll see this benchmark become an important tool for improving the next generation of video models. TLDR: IPV-Bench is a new benchmark testing video models' understanding of physical impossibilities. Current models frequently generate physically impossible content even when instructed not to, showing they lack true understanding of how the physical world works. Full summary is here. Paper here. submitted by /u/Successful-Western27 [link] [comments]

I am making a water leak dataset, I can't seem to agree with my team if the dataset should be balanced (500/500) or unbalanced (850/150) to reflect real world scenarios because leaks aren't that often, Can someone help? it's an Uni project and we are all sort of beginners. submitted by /u/hippobreeder3000 [link] [comments]

Dear Researchers, We are excited to invite you to submit your research to the 1st IEEE International Conference on Future Intelligent Technologies for Young Researchers (FITYR 2025), which will be held from July 21-24, 2025, in Tucson, Arizona, United States. IEEE FITYR 2025 provides a premier venue for young researchers to showcase their latest work in AI, IoT, Blockchain, Cloud Computing, and Intelligent Systems. The conference promotes collaboration and knowledge exchange among emerging scholars in the field of intelligent technologies. Topics of Interest Include (but are not limited to): Artificial Intelligence and Machine Learning Internet of Things (IoT) and Edge Computing Blockchain and Decentralized Applications Cloud Computing and Service-Oriented Architectures Cybersecurity, Privacy, and Trust in Intelligent Systems Human-Centered AI and Ethical AI Development Applications of AI in Healthcare, Smart Cities, and Robotics Paper Submission: https://easychair.org/conferences/?conf=fityr2025 Important Dates: Paper Submission Deadline: April 30, 2025 Author Notification: May 22, 2025 Final Paper Submission (Camera-ready): June 6, 2025 For more details, visit: https://conf.researchr.org/track/cisose-2025/fityr-2025 We look forward to your contributions and participation in IEEE FITYR 2025! Best regards, Steering Committee, CISOSE 2025 submitted by /u/khushi-20 [link] [comments]