I am currently engaged on a project that aims to give some insight to machine learning engineers about how their models perform on vast variety of mobile devices. It is starting to be a very popular practice to embed machine learning models within apps and use them without needing any api/network connection. You can see most examples especially for apps that use computer vision heavily. Passing each and every image to cloud for processing is simply unacceptable, data heavy and slow. With the latest improvements in the field, embedding ml models to apps gets easier and preferable. This comes with another price though. There are 1000s of mobile devices out there that come with different chipsets like Qualcomm, Exynos, Snapdragon etc. They also come with different gpu capabilities and on top of that different OS versions. All these combinations are very likely to create some uncertainty. Does my model performs the same way it does in the office's android test phone? After working on a computer vision and machine learning startup for more than 3 years as a lead mobile engineer who embedded 10s of models inside apps, answer to that question is very clear to me. No, my model will not perform same on a Xiaomi Android 11 phone as it performs on your office Samsung Android 13. And often you will not even know that. ML engineers will be highly isolated from the app environment. They can measure the performance of ml model already with their tools in the cloud when it comes to accuracy, recall etc. Which are very very important metrics. But, they already measure/evaluate that. When it comes to inference time, it heavily depends on the system it works on. It is not feasible to have each and every mobile device in the office available. To solve this issue, we have decided to develop mobile SDK and a platform for collecting/visualising some metrics. And we have decided the most important metric, at the heart of the issue, would be the inference time. I would like to ask you people if this makes sense and is reasonable. Is there other vital metrics you think a ml engineer would be interested in? The SDK we prepared collects all device related metadata( memory available, cpu usage, os, api level, battery etc.) and inference time parameter and shows charts like: OS System vs inference time Device model vs inference time Memory available vs inference time in a single session etc. submitted by /u/orcnozyrt [link] [comments]

Aka Interface / OS / system agents and LAM. Seems like many new projects are popping up in this space, curious to get your thoughts on whether these will stick around, and AI agents will become the center of every user interaction going forward. Some examples: francedot/Interface-Agent: InterfaceAgent: a versatile framework designed to create system and interface agents capable of managing mobile and desktop applications and features. (github.com) mnotgod96/AppAgent: AppAgent: Multimodal Agents as Smartphone Users, an LLM-based multimodal agent framework designed to operate smartphone apps. (github.com) microsoft/UFO: A UI-Focused Agent for Windows OS Interaction. (github.com) OpenInterpreter/open-interpreter: A natural language interface for computers (github.com)- microsoft/UFO: A UI-Focused Agent for Windows OS Interaction. (github.com) submitted by /u/Pretend-Map7430 [link] [comments]

I found this paper very interesting. We kind of make same assumptions, that the authors are making, while using covnet for computer vision. I was wondering can we extend for computer vision use cases Abstract Traditional supervised learning aims to learn an unknown mapping by fitting a function to a set of input-output pairs with a fixed dimension. The fitted function is then defined on inputs of the same dimension. However, in many settings, the unknown mapping takes inputs in any dimension; examples include graph parameters defined on graphs of any size and physics quantities defined on an arbitrary number of particles. We leverage a newly-discovered phenomenon in algebraic topology, called representation stability, to define equivariant neural networks that can be trained with data in a fixed dimension and then extended to accept inputs in any dimension. Our approach is user-friendly, requiring only the network architecture and the groups for equivariance, and can be combined with any training procedure. We provide a simple open-source implementation of our methods and offer preliminary numerical experiments. submitted by /u/No-Natural36 [link] [comments]

I'm currently engaged in a project employing the pre-trained Phi-3-mini model (utilizing ollama for execution). In this project, I've integrated RAG with ChromaDB as the vector store, and I've incorporated a local embedding model named nomic-embed-text. My objective is to inform the Phi-3 model that it's operating for XYZ company and for a specific purpose. Additionally, I need to ensure that the model is aware of everyday currency values. While I can retrieve these values daily, I'm seeking a method to notify the model only once per day. Moreover, I'm open to exploring alternative tools for the vector store and embedding model, as long as they align with the requirements of the project. I've opted for Phi-3 due to its suitability as a Small Language Model (SLM) for this task. submitted by /u/SadHat4219 [link] [comments]

Hey all, firstly, I don't want to sound disingenuous so I really hope this doesn't come off that way. I have a pretty non-traditional path to Data Science, I did a Bachelor of Commerce, and through a rotation program at a big Canadian bank got into a Data Science team, that was supportive and took me on despite me lacking technicals. I've been in the position now for 2 years, mostly working with NLP and unstructured data. Doing usual Power BI dashboard, KPI antics, all the way up to using transformer embeddings for email classification models. It has been a cool role, sadly plagued with bad management, and all at a slow bank. In recent times, the bank has gone through reorg, and we (data and analytics team) do not have the support from senior management, nor the funding that we did maybe even a year ago. Layoffs are a small possibility, but already I feel like we have been brushed to side, with not much expectations, nor no net new projects. Furthermore, my boss who had hired me might also be leaving, meaning I would be stuck with completely non-technical management, and I would learn nothing. Perhaps the one good thing is the pay, but that is making it hard for me to now find a new role. My current pay is around $95K CAD + last year, a 15% bonus. With new leadership now though, that doesn't support us, I doubt we will get a bonus as fat as that anymore. I have been interviewing with eBay for a Data Science Analyst position, working on their item buying page, and got the offer yesterday. I would report to a team in SF, but would be based in Canada. The role would be less model-building, a lot more A/B testing, from what I gathered. Pros of eBay: Data Science at Tech company gives validity to my otherwise non-technical resume (academically, at least) 1 week in office and flexibility to work from abroad (according to HR); so I can travel home internationally and not burn PTO, compared to 3 in office and no flexibility currently Hopefully newer tech stack than the bank (fr we don't even have a SQL server set up here lmfao) Been with the bank for 3 years in various roles, really tired and fed up, so would be a good change and refresh 20K sign on for first year, 10K sign on second year, conditional to me staying for a year after receiving bonuses, in addition to equity discount purchase options, and $30K USD equity package (25%/year vest) in compensation Cons: Only a 10% base-pay bump. Tech homies (though in SF) tell me to not settle for anything less than 20% bump between jobs. Not building models as much, which makes it more statistics-focused and less applied 80% focused on A/B testing from what I gathered Unsure of eBay market reputation these days, what are some companies that people end up working at after eBay? High rate of layoffs In addition to all of this, I am interviewing with Intuit and Robinhood, both for more technical Data Science positions. Those would likely pay in the 120K-140K CAD range from what I understand, eBay would be at 105k. The next step for these interviews would be the technicals, which I do not feel confident for at all (Python, SQL LeetCode Mediums + statistics and ML design questions). I haven't studied for this stuff that much, and would really be cramming. I have told eBay that I can let them know on Monday if the compensation is okay, I am tempted to go back to them and beg for $110K CAD, which would be a 15% bump, and mention that I am interested in the role, but the pay is just not working for me, especially that I am interviewing for two other positions that may pay better. So I want to request if I can continue the interview process with the other two, and get back to eBay with a final confirmation. Not sure how to phrase this, or if I should even show my hand like that. eBay HR was really nice, really felt like they were gunning for me, and the offer felt like the max they could squeeze out between Payroll and the Hiring team. I'm just confused. In light of all of this, where can/should I go? I was 65% in favour to take, 35% to reject, but given my friends/family advice that the pay is low and not worth moving, I am unsure. I'm also not sure how to buy myself time for the interveiws with RH and Intuit, and god if I can even do them. Is it really that bad to move for 10% base bump? Ignoring sign-on bonus, equity, quality of life? Lastly, I'm not sure if eBay is a boost to my career or a step down; is it a good place to work? Is it frowned upon by other companies as a "legacy tech company" or something? Thank you! submitted by /u/sg6128 [link] [comments]

https://preview.redd.it/ws4d4qiczfyc1.png?width=639&format=png&auto=webp&s=241e5ceb3d40157deed93e78faaee4116f07b195 How does mean substraction operation project a vector onto a hyperplane and likewise scaling project it onto a hypersphere in Layer Normalization? submitted by /u/ApartmentEither4838 [link] [comments]

Hi, I’m working on a ML system for network intusion detection. I’ve come across huge free datasets that have been really helpful but I’ve come to a point in my project where I need to make my own. I see the millions of simulated attacks on a network and can’t imagine that this is sone by hand. If anyone has any ideas it would be appreciated. Thanks submitted by /u/OpeningDirector1688 [link] [comments]

For people who have worked on self hosting your own models, I'm curious about your tech stack and architecture for model serving. Especially for those who are serving models larger than 30B. What optimizations and stack do you find effective for dealing with an environment where request volume is volatile (e.g can spike 10x in minutes), but responsiveness needs to be high? submitted by /u/gamerx88 [link] [comments]

In your opinion, I was wondering which is better when searching for a data job-- a portfolio of small demos or an actual product that fills a void? For example, if my community has an information need such as analysis of schools, their suspension rate and other related features, would that be better than a bunch of small projects posted to github? I'm thinking an actual product is more beneficial in showcasing one's skills, because it's an end-to-end project (e.g., data collection, data cleaning, analysis, infrastructure, integrating data updates, etc). submitted by /u/wage_slaving_sucks [link] [comments]

So I've been looking into open robotics projects lately, with the desire of building something that can use ai. I've seen the deep mind video of robots playing soccer; I have no idea what kind of hardware is needed to run that inference on a robot, but I like the idea; build a chassis, and virtually train it and then let the robot use the model. I ran into a robotics project earlier today called Stack Chan, it runs on a M5Stack iot core; the project is designed to be extensible. Suppose I wanted to bolt wheels and an arm to this stack chan; would I be able to train that with ai? Or, would I have to get something like an Nvidia SBC to do this? submitted by /u/zelkovamoon [link] [comments]

Sharing a fun little random project I worked on last week where I made multiple LLMs interact with each other pretending to be humans… submitted by /u/AvvYaa [link] [comments]

At it's essence I guess RAG is about retrieving relevant documents based on the prompt putting the documents into the context window Number 2 is very straight forward, while number 1 is where I guess more of the important stuff happens. IIRC, most often we do a similarity search here between the prompt embedding and the document embeddings, and retrieve the k-most similar documents. Ok, at this point we have k documents and put them into context. Now it's time for the LLM to give me an answer based on my prompt and the k documents, which a good LLM should be able to do given that the correct documents were retrieved. I tried doing some hobby projects with LlamaIndex but didn't get it to work so nicely. For example, I tried with NFL statistics as my data (one row per player, one column per feature) and hoped that GPT-4 together with these documents would be able to answer atleast 95% of my question correctly, but it was more like 70% which was surprisingly bad since I feel like this was a fairly basic project. Questions were of the kind "how many touchdowns did player x do in season y". Answers varied from being correct, to saying the information wasn't available, to hallucinating an incorrect answer. Hopefully I'm just doing something in suboptimal way, but it got me thinking of how widely used RAG is in production around the world. What are some applications on the market that successfully utilizes RAG? I assume something like perplexity.ai is using it, and of course all other chatbots that uses browsing in some way. An obvious application mentioned is often embedding your company documents, and then having an internal chatbot that uses RAG. Is that deployed anywhere? Not at my company, but I could see it being useful. Basically, is RAG mostly something that sounds good in theory and is currently hyped or is it actually something that is used in production around the world? submitted by /u/lapurita [link] [comments]

submitted by /u/DIAMBRA_AIArena [link] [comments]

I'm in a senior DS role right now. This is my first data job after being a professor for a few years post PhD. I'm a modeler, that's my main focus on the job, which I absolutely love. However, the client (I'm a consultant) uses SAS miner and guide, and does not use Python at all. Partially because they always have and partially for security concerns. As I build my models, realistically the biggest issue is making sure I do things that our (imo outdated) tech stack can handle. I'd love to do a sexy GNN network based model for example but right now we struggle to execute a random forest. The experience I'm getting is great, I'll be about to make some solid quantifiable improvements, and I'm not looking to move jobs in the next <3 years. However, I worry that if I go on the market in the future, my lack of experience putting Python into prod will be an issue. Hopefully at that point I'll have some promotions under my belt and will be moreso managing a team than running code. If I'm in the future applying for more senior positions, will they care so much about what tools I've been using versus my experience leading a team/communicating with the business, etc? submitted by /u/math_vet [link] [comments]

Hi, Currently I have a one note where I track different pieces of company desired goals/targets through the year. Some of the things they care about : 1) certs / continuing education 2) speaking events 3) individual contributions (projects etc) How are some of the ways you track your progress? And if you don’t…why? Any way you can resell yourself every review is great ammunition imo. submitted by /u/Texas_Badger [link] [comments]

I’m tasked with figuring out how to separate syntax and semantics for a given text. To be more concrete, is there a way to say two text convey the same idea just with a different way of expressing it. The only method I know is to use embeddings and compare the cosine similarities of it but I don’t think that cuts it. I am pretty new to NLP and any recommendation is helpful submitted by /u/justinhjy1004 [link] [comments]

Our work on analysing linear time series forecasting models was accepted to ICML. ArxiV: https://arxiv.org/abs/2403.14587 Abstract: Despite their simplicity, linear models perform well at time series forecasting, even when pitted against deeper and more expensive models. A number of variations to the linear model have been proposed, often including some form of feature normalisation that improves model generalisation. In this paper we analyse the sets of functions expressible using these linear model architectures. In so doing we show that several popular variants of linear models for time series forecasting are equivalent and functionally indistinguishable from standard, unconstrained linear regression. We characterise the model classes for each linear variant. We demonstrate that each model can be reinterpreted as unconstrained linear regression over a suitably augmented feature set, and therefore admit closed-form solutions when using a mean-squared loss function. We provide experimental evidence that the models under inspection learn nearly identical solutions, and finally demonstrate that the simpler closed form solutions are superior forecasters across 72% of test settings. Summary Several popular works have argued that linear regression is sufficient for forecasting (DLinear and FITs are examples for the discerning reader). It turns out that if you do the maths these models are essentially equivalent. We do the math and also the experiments. Perhaps most interestingly: the ordinary least squares (OLS) solution is almost always better than other linear models trained using gradient descent. Importantly: we did not do a hyper parameter search to set, for example, the regularisation coefficient. We reserve that for future work. OLS is extremely efficient - a model can be fit in the order of milliseconds if set up right. Finally, although we don't go to lengths to show this: many of our results are superior to large and complex models, begging the question of when and where such models are effective. submitted by /u/Gramious [link] [comments]

submitted by /u/vijayabhaskar96 [link] [comments]

Hello, I’m coming with question to maybe more experienced professionals or even people which are recruiting. In most job postings I see for DS, MLE, MLOps etc I see requirement of at least 3 YOE. In my personal experience I saw lot of devs with 1 YOE having much more knowledge and wider range of skills then devs with 6 YOE writing code in PHP and using only excel. I assume most people having less experience in their resume would be dropped immediately in early stage of reviewing candidates because of this factor. What’s the deal with this time boundary and is it really that important? submitted by /u/jesteartyste [link] [comments]

Hey folks, Recently spent time measuring the Time to First Token (TTFT) of various large language models (LLMs) when deployed within Docker containers, and the findings were quite interesting. For those who don't know, TTFT measures the speed from when you send a query to when you get the first response. Here are the key findings: Performance Across Token Sizes: Libraries like Triton-vLLM and vLLM are super quick (~25 milliseconds) with fewer tokens but slow down significantly (200-300 milliseconds) with more tokens. CTranslate-2 and Deepspeed-mii also slow down as you increase the token count. However, vLLM keeps things quick and efficient, even with more tokens. Handling Big Inputs: Libraries like Deepspeed-mii, vLLM, TGI, and Triton-vLLM can handle more tokens but get slower the more you push them. This shows some challenges in scaling up. Best Token Responses: While everything runs smoothly up to about 100 tokens, performance drops after 500 tokens. The ideal number of tokens for the quickest response seems to be around 20, with times ranging from about 25 to 60 milliseconds depending on the model. https://preview.redd.it/6n03xwbqddyc1.jpg?width=1600&format=pjpg&auto=webp&s=9464d6f85a2cdab685fc8e7cd7031a85600f00c1 These findings might help you pick the right models and libraries and set your expectations. Keen to hear if anyone else has tested TTFT or has tips on library performance! submitted by /u/rbgo404 [link] [comments]