Hi there! I'm a student and I'm trying to train a folder with 200 images with stylegan3 (I want to create a morphing video synchronized with music) But..I'm having some issues regarding the GPU. Can you recommend some valid alternatives? Thank you ! submitted by /u/Kash112 [link] [comments]

I'm writing a proof of concept for a RAG application for hundreds of thousands of textual records stored in a Postgres DB, using pgvector to store embeddings ( and using an HNSW index). Vector dimensions are specified correctly. Currently running experiments using varied chunk sizes for the text and comparing two different embedding models. (actual chunk size can vary a little because I am not breaking words to force a size). nomic-embed-text snowflake-arctic-embed-m-long Here's the gist experiment: 1- Create embeddings for "n" documents 2- Create a list of queries/prompts for information that is assuredly contained in SOME of those documents. Examples: What were the events that happened at "location x"? What is the John Doe's nickname? Who were the patients that checked into "hospital name"? Tell me about a requisition made by the director of sales. ... 3- For each query/prompt, I run a cosine distance query and get the the nearest 5 matching chunks. 4- After calculating the average distance for all queries/chunks, the lowest value is, in theory, the best combination of model/chunk_size. This worked SUPER well with a small sample of documents (say ≃ 200), but once I added more documents I started noticing an issue. Some of the NEW documents contain lists of literally 30k+ names. Whenever I ran a query that contains names, chunks from the lists above are returned, EVEN IF THEY DON'T CONTAIN THE NAMES, or any of the other information presented in the prompt (this happens regardless of the chose chunk size or strategy). My theory is that when a chunk containing names is embedded, the resulting embedding contain a strong vector for the semantic meaning of "name", but the vectors that differentiate that name from others can be relatively weak. A chunk containing almost nothing but references to the vector for "name" is then considered very similar to the prompt's embeddings, despite the names themselves not matching. For those of you with more experience/understanding, am I wrong in these assumptions? Would you have any suggestions/workarounds? I have some ideas but would like to see if anyone faced the same issues. submitted by /u/grudev [link] [comments]

I am trying to replace the key, query, and value in different prompts of the diffusion model for video editing. I want to understand why key, query, and value are effective and what they represent in the diffusion model. https://preview.redd.it/uoce1dh4rfvc1.png?width=1086&format=png&auto=webp&s=24d6504ca9c50d9f5924dd935204db6c15484a16 submitted by /u/Candid_Finish444 [link] [comments]

I asked a few of them (via Ollama) about WebGPU adoption and it turns out all of them are using old data. Here are the dates they gave me: wizardlm2:7b-q5_0: early 2023 LLAMA 3: August 2022 LLAMA 2: No date but gave similar answer to LLAMA 3 mistral: No date but very generic answer and I couldn't get it to divulge when it was updated last I also went online and asked ChatGPT and even it was January 2022. Are there newer models around? Edit: You can probably tell I'm new to this... submitted by /u/TheyreNorwegianMac [link] [comments]

I'm not an ML expert, but I work with some, and I've been asking around the (virtual) office, as well as interviewing scholars. Based on my research, I wrote an article you can read here. It seems to me that, while the hardware and software supporting LLMs will pretty certainly improve, the data presents a more complicated story. There's the issue of model collapse: essentially, the idea that as models approximate the distributions of original data sets with finite sampling, they will inevitably cut off the tails of those distributions. And as they begin to sample their own approximations in future model generations, this will lead to a collapse of the model (unless it can continue to tap that original data source). Then there's the issue of error propagation across generations of LLMs. Mark Kon, at Boston University, suggests tools like watermarking to help keep our datasets clean moving forward (he described the problem as a bigger mouse/bigger mousetrap situation). Mike Chambers, one of my colleagues at AWS, basically argued as much or more can be accomplished at this point by cleaning our datasets as by ingesting ever more data. One related, long term takeaway is that LLMs and other models will probably start working to ingest new categories of data (beyond text and image) before too long. And that next paradigm shift is going to happen sooner than many of us think. Thoughts? submitted by /u/thedaveperry1 [link] [comments]

Hello I am trying to implement this from a paper: First, select the first l sampling points in the sampling points of bearing faults and calculate the mean μ_rms and standard deviation σ_rms of their root mean square values, and establish a 3σ criterion- based judgment interval [μ_rms − 3σ_rms, μ_rms +3σ_rms] accordingly. 2) Second, calculate the RMS index for the l + 1 th point FPTl+1 and compare it with the decision interval in step 1. If its value is not in this range, then recalculate the judgment interval after making l =l + 1. If its value is within this range, a judgment is triggered once. 3) Finally, in order to avoid false triggers, three consecutive triggers are used as the identification basis for the final FPT, and make this time FPTl = FPT The paper title: Physics guided neural network: Remaining useful life prediction of rolling bearings using long short-term memory network through dynamic weighting of degradation process My question is: how do I get the μ_rms and σ_rms from the RMS? What I did in this case was first sample the data and then calculate the RMS on the samples. But then I recreate sequences from these RMS values (which doesn't seem logical to me) and then calculate the μ_rms and σ_rms. I do use this value I obtain to do the interval and compare it with the RMS value. But the problem is that by doing this, it triggers way too early. This is the code I have made: def find_fpt(rms_sample, sample): fpt_index = 0 trigger = 0 for i in range(len(rms_sample)): upper = np.mean(rms_sample[i] + 3 * np.std(rms_sample[i])) lower = np.mean(rms_sample[i] - 3 * np.std(rms_sample[i])) rms = np.mean(np.square(sample[i + 1]) ** 2) if upper > rms > lower: if trigger == 3: fpt_index = i break trigger += 1 else: trigger = 0 print(trigger) return fpt_index def sliding_window(data, window_size): return np.lib.stride_tricks.sliding_window_view(data, window_size) window_size = 20 list_bearing, list_rul = load_dataset_and_rul() sampling = sliding_window(list_bearing[0][::100], window_size) rms_values = np.sqrt(np.mean(np.square(sampling) ** 2, axis=1)) rms_sample = sliding_window(rms_values, window_size) fpt = find_fpt(rms_sample,sampling) submitted by /u/Papytho [link] [comments]

so i was experimenting with tabnet architecture by google https://arxiv.org/pdf/1908.07442.pdf and found that if the data has a lot of randomness and noice then only it can outperform based on my dataset, but traditional machine learning algo like xgboost, random forest do a better job at those dataset where the features are robust enough but they fail the zero shot test and the transformer show some accuracy in that, so i just wanted to check if its possible to merge both of the traditional techniques and the transformer architecture so that it can perform better at traditional ml algo datasets and also give a good zero shot accuracy. while trying to merge it i found that in the tabnet paper they assume that each feature is independent and do not provide any place for any relationship with the features itself but the Tabtransformer architecture takes it into account https://arxiv.org/pdf/2012.06678.pdf as well but doesnt have any feature selection as proposed in tabnet.... i tried to merge them but was stuck where i have to do feature selection on the basis of the dimension assigned to each feature, while this work i s done by sparsemax in the tabnet paper i cant find a way to do that... any help would be appreciated submitted by /u/Shoddy_Battle_5397 [link] [comments]

Ansys has released an AutoML product for physical simulation called Ansys Sim AI (https://www.ansys.com/fr-fr/news-center/press-releases/1-9-24-ansys-launches-simai). As a machine learning engineer, I wonder what types of models can be used to train on 3D mesh data in STL format with physical fields. How can the varying dimensions of input and output data be managed for different geometric objects? Does anyone have any ideas on this topic? submitted by /u/SatieGonzales [link] [comments]

I have been working on a project for the past couple of months, and I wanted to know if anyone had feedback or thoughts to fuel its completion. I built a lexer and parser using python and C tokens to create a language that reads a python script or file and utilizes hooks to amend or write new lines. It will be able to take even a blank Python file to write, test, and deliver a working program based on a single prompt provided initially by the user. The way it works is it uses GPTs API to call automated prompts that are built into the program. It creates a program by itself by only using 1 initial prompt by the user on the program. It is a python program with the language I named autoscripter built into it. I hope to finish it by the end of the year if not into next year. This is a very challenging project, but I believe it is the future of scripting, and I have no doubts Microsoft will release something on this sooner than later. Any thoughts? I created this first by designing a debugger that error corrected python code and realized that not only error correction could be automated, but also the entire scripting process could be left to a lot of automation. submitted by /u/starcrashing [link] [comments]

I am working on a project regarding marketing of AI powered products in Retail stores. I am trying to find some products that market ‘AI’ as the forefront feature, eg Samsung’s BeSpoke AI series, Bmw’s AI automated driving etc. Need them to be physical products so I can go to stores and do research and survey. Any kind of help is appreciated. submitted by /u/Complete-Holiday-610 [link] [comments]

Latest releases of models such as GPT-4 and Claude have a significant jump in the maximum context length (4/8k -> 128k+). The progress in terms of number of tokens that can be processed by these models sound remarkable in % terms. What has led to this? Is this something that's happened purely because of increased compute becoming available during training? Are there algorithmic advances that have led to this? submitted by /u/analyticalmonk [link] [comments]

Used to receive invitation by this time of the year. Maybe I am forgotten. submitted by /u/noname139713 [link] [comments]

I'm a recent engineering graduate who's switching roles from traditional software engineering ones to ML/AI focused ones. I've gone through an introductory probability course in my undergrad, but the recent developments such as diffusion models, or even some relatively older ones like VAEs or GANs require an advanced understanding of probability theory. I'm finding the math/concepts related to probability hard to follow when I read up on these models. Any suggestions on how to bridge the knowledge gap? submitted by /u/AffectionateCoyote86 [link] [comments]

Say I have 1000 PDF docs that I use as input to a RAG Pipeline. I want to to evaluate different steps of the RAG pipeline so that I can measure: - Which embedding models work better for my data? - Which rerankers work and are they required? - Which LLMs give the most factual and coherent answers? How do I evaluate these steps of the pipeline? Based on my research, I found that most frameworks require labels for both retrieval and generation evaluation. How do I go about creating this data using a LLM? Are there any other techniques? Some things I found: For retrieval: Use a LLM to generate synthetic ranked labels for retrieval. Which LLM should I use? What best practices should I follow? Any code that I can look at for this? For Generated Text: - Generate Synthetic labels like the above for each generation. - Use a LLM as a judge to Rate each generation based on the context it got and the question asked. Which LLMs would you recommend? What techniques worked for you guys? submitted by /u/awinml1 [link] [comments]

Hi everyone. I want to build this idea of mine for a class project, and I wanted some input from others. I want to build an AI algorithm that can play the game Drift Hunters (https://drift-hunters.co/drift-hunters-games). I imagine I have to build some reinforcement learning program, though I'm not sure exactly how to organize state representations and input data. I also imagine that I'd need my screen to be recorded for a continuous period of time to collect data. I chose this game since it's got three very basic commands(turn left, turn right, and drive forward) and the purpose of the game(which never ends) is to maximize drift score. Any ideas are much appreciated. lmk if u still need more info. Thanks everyone. submitted by /u/Valuable-Wishbone276 [link] [comments]

PDF: https://arxiv.org/abs/2404.11457 GitHub: https://github.com/KID-22/LLM-IR-Bias-Fairness-Survey ​ Abstract: With the rapid advancement of large language models (LLMs), information retrieval (IR) systems, such as search engines and recommender systems, have undergone a significant paradigm shift. This evolution, while heralding new opportunities, introduces emerging challenges, particularly in terms of biases and unfairness, which may threaten the information ecosystem. In this paper, we present a comprehensive survey of existing works on emerging and pressing bias and unfairness issues in IR systems when the integration of LLMs. We first unify bias and unfairness issues as distribution mismatch problems, providing a groundwork for categorizing various mitigation strategies through distribution alignment. Subsequently, we systematically delve into the specific bias and unfairness issues arising from three critical stages of LLMs integration into IR systems: data collection, model development, and result evaluation. In doing so, we meticulously review and analyze recent literature, focusing on the definitions, characteristics, and corresponding mitigation strategies associated with these issues. Finally, we identify and highlight some open problems and challenges for future work, aiming to inspire researchers and stakeholders in the IR field and beyond to better understand and mitigate bias and unfairness issues of IR in this LLM era. ​ https://preview.redd.it/3glvv92v6dvc1.png?width=2331&format=png&auto=webp&s=af66f2bf082620882f09ea744eda88cf06c67112 ​ https://preview.redd.it/d48pt3sw6dvc1.png?width=1126&format=png&auto=webp&s=2343460399473bde3f5e37c0bbcfdc88ffc81efb submitted by /u/KID_2_2 [link] [comments]

So, nowadays, everyone is distilling rationales gathered from a large language model to another relatively smaller model. However, I remember from the old days that we did we train the small network to match the logits of the large network when doing distillation. Is this forgotten /tried and did not work today? submitted by /u/miladink [link] [comments]

Llama-3 is making waves in the AI community. I was curious how it will perform in the medical domain, Here are the evaluation results for Llama-3 (7B and 70B) on a medical domain benchmark consisting of 9 diverse datasets https://preview.redd.it/sdwx5tglxbvc1.png?width=1464&format=png&auto=webp&s=d32585a69244d44c83e2b1e8a85301a7a8676ea2 I'll be fine-tuning, evaluating & releasing Llama-3 & different LLMs over the next few days on different Medical and Legal benchmarks. Follow the updates here: https://twitter.com/aadityaura https://preview.redd.it/9egbcayv9avc1.png?width=1344&format=png&auto=webp&s=436a972421d5568e1a544962b8cfd1c7b14efe04 submitted by /u/aadityaura [link] [comments]

I have been hunting jobs for almost 4 months now. It was after 2 years, that I opened my eyes to the outside world and in the beginning, the world fell apart because I wasn't aware of how much the industry has changed and genAI and LLMs were now mandatory things. Before, I was just limited to using chatGPT as UI. So, after preparing for so many months it felt as if I was walking in circles and running across here and there without an in-depth understanding of things. I went through around 40+ job posts and studied their requirements, (for a medium seniority DS position). So, I created a plan and then worked on each task one by one. Here, if anyone is interested, you can take a look at the important tools and libraries, that are relevant for the job hunt. Github, Notion I am open to your suggestions and edits, Happy preparation! submitted by /u/xandie985 [link] [comments]

Paper: https://arxiv.org/abs/2311.09480 Tweet: https://x.com/NickLourie/status/1770077925779337563 Code: https://github.com/nicholaslourie/opda Docs: https://nicholaslourie.github.io/opda/tutorial/usage.html Abstract: The choice of hyperparameters greatly impacts performance in natural language processing. Often, it is hard to tell if a method is better than another or just better tuned. Tuning curves fix this ambiguity by accounting for tuning effort. Specifically, they plot validation performance as a function of the number of hyperparameter choices tried so far. While several estimators exist for these curves, it is common to use point estimates, which we show fail silently and give contradictory results when given too little data. Beyond point estimates, confidence bands are necessary to rigorously establish the relationship between different approaches. We present the first method to construct valid confidence bands for tuning curves. The bands are exact, simultaneous, and distribution-free, thus they provide a robust basis for comparing methods. Empirical analysis shows that while bootstrap confidence bands, which serve as a baseline, fail to approximate their target confidence, ours achieve it exactly. We validate our design with ablations, analyze the effect of sample size, and provide guidance on comparing models with our method. To promote confident comparisons in future work, we release opda: an easy-to-use library that you can install with pip. submitted by /u/nicholaslourie [link] [comments]