Programmers are always looking for ways to make their code more efficient. One way to do this is to use a faster loop. Python is a high-level programming language that is widely used by developers and software engineers. It is known for its readability and ease of use. However, one downside of Python is that its loops can be slow. This can be a problem when you need to process large amounts of data. There are several ways to make Python loops faster. One way is to use a faster looping construct, such as C. Another way is to use an optimized library, such as NumPy. Finally, you can vectorize your code, which means converting it into a format that can be run on a GPU or other parallel computing platform. By using these techniques, you can significantly speed up your Python code.
According to Vladislav Zorov, If not talking about NumPy or something, try to use list comprehension expressions where possible. Those are handled by the C code of the Python interpreter, instead of looping in Python. Basically same idea like the NumPy solution, you just don’t want code running in Python.
Example: (Python 3.0)
lst = [n for n in range(1000000)]
def loops():
newlst = []
for n in lst:
newlst.append(n * 2)
return newlst
def lstcomp():
return [n * 2 for n in lst]
from timeit import timeit
print(timeit(loops, number=100))
#18.953254899999592 seconds
print(timeit(lstcomp, number=100))
#11.669047399991541 seconds
Or Do this in Python 2.0
Python list traversing tip:
Instead of this: for i in range(len(l)): x = l[i]
Use this for i, x in enumerate(l): …
TO keep track of indices and values inside a loop.
Twice faster, and the code looks better.
Another option is to write loops in C instead of Python. This can be done by using a Python extension module such as pyximport. By doing this, programmers can take advantage of the speed of C while still using the convenient syntax of Python.
Finally, developers can also improve the performance of their code by making use of caching. By caching values that are computed inside a loop, programmers can avoid having to recalculate them each time through the loop. By taking these steps, programmers can make their Python code more efficient and faster.
Very Important: Don’t worry about code efficiency until you find yourself needing to worry about code efficiency.
The place where you think about efficiency is within the logic of your implementations.
Get 20% off Google Google Workspace (Google Meet) Standard Plan with the following codes: 96DRHDRA9J7GTN6 Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE (Email us for more codes)
Hello everyone., on today new scraper I created the python version for the zillow scraper. https://github.com/johnbalvin/pyzill What My Project Does The library will get zillow listings and details. I didn't created a defined structured like on the Go version just because it's not as easy to maintain this kind of projects on python like on Go. It is made on pure python with HTTP requests, so no selenium, puppeteer, playwright etc. or none of those automation libraries that I hate. Target Audience This project target could be real state agents probably, so lets say you want to track the real price history of properties around an area, you can use it track it Comparison There are libraries similar outhere but they look outdated, most of the time, scraping projects need to ne on constant maintance due to changed on the page or api pip install pyzill Let me know what ou think, thanks about me: I'm full stack developer specialized on web scraping and backend, with 6-7 years of experience submitted by /u/JohnBalvin [link] [comments]
Weekly Thread: Beginner Questions 🐍 Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you. How it Works: Ask Anything: Feel free to ask any Python-related question. There are no bad questions here! Community Support: Get answers and advice from the community. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources. Guidelines: This thread is specifically for beginner questions. For more advanced queries, check out our Advanced Questions Thread. Recommended Resources: If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance. Example Questions: What is the difference between a list and a tuple? How do I read a CSV file in Python? What are Python decorators and how do I use them? How do I install a Python package using pip? What is a virtual environment and why should I use one? Let's help each other learn Python! 🌟 submitted by /u/AutoModerator [link] [comments]
I was wondering if anyone has come across anything like https://github.com/a-h/templ or https://hono.dev/guides/jsx, but for python. For context, I am familiar with jinja2, mako, etc. but find them to be unintuitive due to the loose coupling of logic (ex: database calls) and templating (ex: generating a list from the database results). Therefore, I am looking for a "inline" templating solution. submitted by /u/TheRealMrMatt [link] [comments]
I'm trying to decide whether to use typeguard or stick to assert isinstance in the places where I care. Has anyone done benchmarking testing of the overhead of using type guards "at"typeguard decorator ? submitted by /u/gerardwx [link] [comments]
Take the excellent nuitka, compile python code to C, turn it into web assembly, and you got Python in the browser, without the usual runtime overhead: https://wasmer.io/posts/py2wasm-a-python-to-wasm-compiler While the doc states you can get this effect by doing: pip install py2wasm py2wasm myprogram.py -o myprogram.wasm wasmer run myprogram.wasm You still need the wasmer WASM runtime (curl https://get.wasmer.io -sSfL | sh for Unix users, iwr https://win.wasmer.io -useb | iex for Windows user with Powershell), however. But more than that, since you need nuikta, it means you need a C compiler installed. While in Ubuntu it's a just an sudo apt install build-essential, it will require a bit more work on Windows and Mac. submitted by /u/desmoulinmichel [link] [comments]
I'm sharing some of the Python tutorials I made. I teach Ukrainian teenagers and university students Python programming and data science for free. Here is an introduction to Pandas. It shows: How to read data into a Pandas dataframe from a CSV file. Show how to print out the column names. See and set the index. Print sections of the dataframe. Do basic statistics. Create a dataframe from a dictionary. submitted by /u/cyprusgreekstudent [link] [comments]
Weekly Wednesday Thread: Advanced Questions 🐍 Dive deep into Python with our Advanced Questions thread! This space is reserved for questions about more advanced Python topics, frameworks, and best practices. How it Works: Ask Away: Post your advanced Python questions here. Expert Insights: Get answers from experienced developers. Resource Pool: Share or discover tutorials, articles, and tips. Guidelines: This thread is for advanced questions only. Beginner questions are welcome in our Daily Beginner Thread every Thursday. Questions that are not advanced may be removed and redirected to the appropriate thread. Recommended Resources: If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance. Example Questions: How can you implement a custom memory allocator in Python? What are the best practices for optimizing Cython code for heavy numerical computations? How do you set up a multi-threaded architecture using Python's Global Interpreter Lock (GIL)? Can you explain the intricacies of metaclasses and how they influence object-oriented design in Python? How would you go about implementing a distributed task queue using Celery and RabbitMQ? What are some advanced use-cases for Python's decorators? How can you achieve real-time data streaming in Python with WebSockets? What are the performance implications of using native Python data structures vs NumPy arrays for large-scale data? Best practices for securing a Flask (or similar) REST API with OAuth 2.0? What are the best practices for using Python in a microservices architecture? (..and more generally, should I even use microservices?) Let's deepen our Python knowledge together. Happy coding! 🌟 submitted by /u/AutoModerator [link] [comments]
Hi there, My team at Bytewax and I have been working on a series of hands-on guides on streaming data and I am excited to share how one can handle missing values in real-time in Python. While some parts of the guide are simplified, for example, we use a random number generator as an input source, the algorithmic part is production-ready. We are taking advantage of the Numpy library + stateful operators in Bytewax. Other input sources are available, too. https://bytewax.io/guides/handling-missing-values submitted by /u/oli_k [link] [comments]
I have started at a new job and had the idea that it would probably be clever to set up my developing environment in exactly the same way as my predecessor did. Because: This should help resolving errors quicker in the transition period His code was good and clean and it appears that he knows what he is doing we were using mostly the same tools (VScode etc.) anyways. He set up his virtual environments (VE)s with conda/mamba. I vaguely remembered that I also used to do that but then stopped for some reason and switched to the virtualenv package. But I did not remember why anymore. So I just set up my VEs in the same way, it should not really make any difference anyways (so I thought). Well, fast forward about two weeks and now I have VEs that occasionally (but not always) exist twice in the same folders under the same name (according to mamba info --envs) and that are at the same time completely empty (according to mamba list) and contain all packages I have installed anywhere, ever (according to pip list). I usually install packages via pip and I assume this may have fucked things up in combination with mamba? I'll probably switch back to virtualenv again and add a "do not use conda/mamba VEs !!!" in my notes. I am working on Windows. Is mamba better on Linux? submitted by /u/RareRandomRedditor [link] [comments]
I found this awesome AI framework for python called "Crew AI", that allows us to create assistants, or agents in the CrewAI lingo, and assign specific tasks. I started my tiny youtube channel 3 months and for a youtuber, especially a novice one like myself, there are a bunch of tasks such as checking what other similar channels are doing, which topics are trending and if a video idea is good or not. There is also the part of coming up with the video content idea itself, create a catchy title, suitable youtube tags and finding the appropriate forum to talk about! You folks get the picture. It's a great project for anyone seeking to automate tasks not just related to a youtube channel, but this could easily be adapted to your daily job tasks, or even a startup idea. I have written a full article about how the project unfolds and also a youtube video for how I did it step by step, which you are welcome to check it out! The links are as follows: The medium article: https://medium.digitalmirror.uk/create-an-ai-team-to-manage-your-youtube-channel-5dc1e6c9b31b The YouTube video: https://youtu.be/5JoVeYcxgpU And of course the source code: https://github.com/fmiguelmmartins/crewaiyoutube.git As always, if you would like to drop some feedback so I can improve with time I would be grateful! Cheers submitted by /u/fx2mx3 [link] [comments]
Just came across this list and I think it's amazing: https://techtalksweekly.substack.com/p/all-pydata-2023-talks submitted by /u/ad81923 [link] [comments]
New developers print out strings to their terminal. It’s how we learn! But printing out to the terminal isn’t what you do with most professional applications. In those cases, you log into files. Sometimes, you log into multiple locations at once. These logs may serve as an audit trail for compliance purposes or help the engineers debug what went wrong. Python Logging teaches you how to log in the Python programming language. Python is one of the most popular programming languages in the world. Python comes with a logging module that makes logging easy. What You’ll Learn In this book, you will learn how about the following: Logger objects Log levels Log handlers Formatting your logs Log configuration Logging decorators Rotating logs Logging and concurrency and more! Book formats The finished book will be made available in the following formats: paperback (at the appropriate reward level) PDF epub The paperback is a 6″ x 9″ book and is approximately 150 pages long. You can check out the book's campaign on Kickstarter. Let me know if you have any questions! submitted by /u/driscollis [link] [comments]
I have not seen updates in almost a year and there is no activity in the support forum? Is there a replacement forum? Thanks submitted by /u/br_web [link] [comments]
https://github.com/leftmove/cria My name is Anonyo, and I am a seventeen year old from Southeast Michigan. This is my second open source project. I built Cria, a Python library that allows you to run LLMs programmatically through Python. Cria is designed so there is as little friction as possible — getting started takes just five lines of code. I created this library because I was using OpenAI in my project, and kept running into rate limits. With local LLMs getting better and better, I sought to switch, but found command line configurations to be limited. Running and configuring LLMs can be trivial, but programs like ollama make it easier. The only problem I found with ollama though, was the lack of features in its Python client. To counteract this, I built Cria — a program that is concise, efficient, and most importantly programmable. The name is based off of Meta's llama, as baby llamas are called crias. Cria solves pain points of ollama - it manages the ollama server for you, saves your message history, streams responses by default, and allows easier management of multiple LLMs at once. The target audience for this project is anyone that uses AI. This library was built in a weekend, and is meant to make running LLMs easier for anyone that wants to go local. I built this for myself, but it could be used at scale as ollama runs its own server (there would of course have to be extra configuration though). In Python, while there are libraries to run LLMs, they are feature limited. Ollama provides the most extensive toolset, and so Cria uses that. Ollama has its own Python library, but I found that to be cumbersome when running multiple LLMs. There are similar libraries, but existing alternatives involve strictly the command line (from what I've seen). While this isn't my first open source project, it's my first Python library. The code may be a little rough around the edges, so I would appreciate any and all suggestions. Thank you! submitted by /u/reformedbillclinton [link] [comments]
Golem Network has just released Ray on Golem MVP, a significant milestone in integrating Ray with Golem's decentralized infrastructure! 🎉 Check out all the details about the release and next steps in our blog post: 📝 https://blog.golem.network/announcing-ray-on-golem-mvp-launch-the-recommended-way-of-running-python-code-on-the-golem-network/ 👉 Golem is looking for engaged beta testers to run diverse applications and collaborate on refining the solution. Interested? Find all the information on the brand-new Ray on Golem site: 🔗 https://www.golem.network/ray submitted by /u/GolemSM [link] [comments]
A project to build a local server without any external dependencies with main features of hot reloading browser on source modification. The idea is to make it simple, standalone and work without any setup. The project uses regular sockets, websockets, inotify and a bit of javascript on the fly embedding to allow users to achieve automatic synchronization with browser. The article describes hot to monitor file changes with direct loading of libc and requesting kernel notify on change. Free article and code: https://hereket.com/posts/linux\_live\_reload\_python\_server/ submitted by /u/tootac [link] [comments]
Hi, I'm Owen, and I've developed an AI tool for data processing. I spend a significant amount of time processing daily data from numerous Excel files, which requires me to write a lot of Python code. I have been using GitHub Copilot to assist with this, but I am exploring ways to eliminate the need to write code manually. The tool is called Tipis AI, and you can find it here: https://github.com/tipisai/tipis-fe What My Project Does Therefore, I have developed an AI that generates Python code for data processing. I've also created a Python runtime environment to execute this code. You can use this tool to process any type of files. Target Audience Any worker. Someone needs to process files. Comparison You can perform complex calculations and summarizations on files using natural language descriptions, without needing to master complex Python knowledge If you have any suggestions for AI in data processing, please let me know, and I might include them in this product.I am very interested in code generation. submitted by /u/smallSohoSolo [link] [comments]
Interactive data visualization is a powerful tool that can significantly enhance the analysis and interpretation of complex datasets. With Python, the Plotly library offers various features that can be used to create interactive publication-quality graphs. This project demonstrates how to use 2 of these awesome features: Range Slider Dropdown Menu A good dataset is essential to demonstrate this style of interactivity. For this project, we will use the UN population projection data to sort and visualize by country for various age groups. FREE ARTICLE: https://johnloewen.substack.com/p/python-plotly-combining-dropdowns-and-range-sliders-for-user-interaction-658dc6fd9c71 submitted by /u/jgloewen [link] [comments]
I made a detailed tutorial on how to work with missing data, outliers, and how to lean up data in Pandas Python. This explains how to clean up your data so that your statistical results and trainin data is not skewed. https://github.com/werowe/HypatiaAcademy/blob/master/pandas/pandas_missing_data.ipynb submitted by /u/cyprusgreekstudent [link] [comments]
For my application, Scipy's solvers are not fast enough so I am looking to speed up by using another package. These are some packages I have found so far: NumbaLSODA DifferentialEquations.jl Torchquad (Although doesn't solve ODEs, just integrates quickly, but that could be wrapped I guess) Torchdiffeq PyDSTool I will be experimenting with these packages, but I was wondering whether anyone has experience with them and has done work on them before. Also, I am wondering whether there are any packages I should check out? submitted by /u/D_vd_P [link] [comments]
Today I Learned (TIL) You learn something new every day; what did you learn today? Submit interesting and specific facts about something that you just found out here.
submitted by /u/Cactus_Jacks_Ear [link] [comments]
Reddit Science This community is a place to share and discuss new scientific research. Read about the latest advances in astronomy, biology, medicine, physics, social science, and more. Find and submit new publications and popular science coverage of current research.