Programmers are always looking for ways to make their code more efficient. One way to do this is to use a faster loop. Python is a high-level programming language that is widely used by developers and software engineers. It is known for its readability and ease of use. However, one downside of Python is that its loops can be slow. This can be a problem when you need to process large amounts of data. There are several ways to make Python loops faster. One way is to use a faster looping construct, such as C. Another way is to use an optimized library, such as NumPy. Finally, you can vectorize your code, which means converting it into a format that can be run on a GPU or other parallel computing platform. By using these techniques, you can significantly speed up your Python code.
According to Vladislav Zorov, If not talking about NumPy or something, try to use list comprehension expressions where possible. Those are handled by the C code of the Python interpreter, instead of looping in Python. Basically same idea like the NumPy solution, you just don’t want code running in Python.
Example: (Python 3.0)
lst = [n for n in range(1000000)]
def loops():
newlst = []
for n in lst:
newlst.append(n * 2)
return newlst
def lstcomp():
return [n * 2 for n in lst]
from timeit import timeit
print(timeit(loops, number=100))
#18.953254899999592 seconds
print(timeit(lstcomp, number=100))
#11.669047399991541 seconds
Or Do this in Python 2.0
Python list traversing tip:
Instead of this: for i in range(len(l)): x = l[i]
Use this for i, x in enumerate(l): …
TO keep track of indices and values inside a loop.
Twice faster, and the code looks better.
Another option is to write loops in C instead of Python. This can be done by using a Python extension module such as pyximport. By doing this, programmers can take advantage of the speed of C while still using the convenient syntax of Python.
Finally, developers can also improve the performance of their code by making use of caching. By caching values that are computed inside a loop, programmers can avoid having to recalculate them each time through the loop. By taking these steps, programmers can make their Python code more efficient and faster.
Very Important: Don’t worry about code efficiency until you find yourself needing to worry about code efficiency.
The place where you think about efficiency is within the logic of your implementations.
Get 20% off Google Google Workspace (Google Meet) Standard Plan with the following codes: 96DRHDRA9J7GTN6 Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE (Email us for more codes)
Weekly Thread: Meta Discussions and Free Talk Friday 🎙️ Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related! How it Works: Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting. Guidelines: All topics should be related to Python or the /r/python community. Be respectful and follow Reddit's Code of Conduct. Example Topics: New Python Release: What do you think about the new features in Python 3.11? Community Events: Any Python meetups or webinars coming up? Learning Resources: Found a great Python tutorial? Share it here! Job Market: How has Python impacted your career? Hot Takes: Got a controversial Python opinion? Let's hear it! Community Ideas: Something you'd like to see us do? tell us. Let's keep the conversation going. Happy discussing! 🌟 submitted by /u/AutoModerator [link] [comments]
Here is just a small wrapper to interact with the new MetaAI chat bot assistant with Python (https://www.meta.ai/), which is running the newly release Llama 3 model. Another nice thing is that its directly connected with Bing Search so you will be able to get the latest informations. https://github.com/Strvm/meta-ai-api submitted by /u/Significant-Turn4107 [link] [comments]
Groupby in itertools module of python works differently that you might have thought. In this video, I have tried to explain very clearly about this feature. You are requested to watch the video. If you like, you can share the video and subscribe my youtube channel for further hacks. https://youtu.be/sX8G8qNwxjc?si=UTdHvbDKIMfOGhpr submitted by /u/hasanul_islam [link] [comments]
Hello ! Earlier this year I made a post about my open source project PyPDFForm and got some really nice feedbacks from you guys. I have been since then continuously working on it and I'd love to share you two really cool features that were newly added to the library. The first one is, like what the title says, the library finally supports creating a subset of widgets through code now. One of the previous hard dependency for PyPDFForm was that it requires a PDF template that was prepared using another tool, namely Adobe Acrobat or some web based ones like DocFly. Well now, at least for text field, checkbox, and dropdown, you can do it through plain Python code. I personally find this a huge milestone of the project and if you are interested in this new feature checkout the documentation here. The second new feature, which is actually not quite new because this was how PyPDFForm worked back in its ancestral stage, is that now you can fill a PDF form "in place", meaning when you fill it this way, the output PDF will look like as if it's filled manually. The reason why this got removed was because of a bug related to text field, where when filled this way the text that got put into a text field will not show up unless the text field is actually clicked by mouse and selected. Fortunately I was enlightened by another user from this sub last time I posted here and he/she gave me a solution on this weird text field bug. Thanks to that, I was able to bring this old but yet new feature back to the library for those who just wants to simply fill a PDF form. Again if you are interested, it's documented here. If you are interested in any of these new features or just the library in general, feel free to go checkout the newest release of the project, try it, test it, and leave comments or suggestions. And of course if you are willing, a star on GitHub is always kindly appreciated. submitted by /u/chinapandaman [link] [comments]
https://github.com/Never-Over/chatpdb Do you ever copy code, errors, or stack traces into ChatGPT? We did, and found it frustrating to always have to manually find, copy, and paste each relevant piece of information. On top of that, being forced to change tools would switch our focus and cause us to lose our flow. That’s why we built chatpdb - a python debugger with ChatGPT! Simply use chatpdb like you would use ipdb or pdb; it’s a drop in replacement with the exact same functionality. The only addition is the new y keyword, which will trigger a response from ChatGPT. Here’s how it works: > /Programming/test-chatpdb/lib.py(2)echo_platform() 1 def echo_platform(platform: str): ----> 2 print("You are running on:" + platform) 3 ipdb> y The exception occurred because the function `echo_platform` tries to concatenate the string "You are running on:" with the `platform` variable, which is `None`. [...] In this example, chatpdb correctly diagnoses an error in the current function as being caused by a bug in the calling function. chatpdb automatically passes relevant context about your program, including the source code, stack trace, and error information if available to ChatGPT. If you have a more specific question you can also supply a prompt to y: > /Programming/test-chatpdb/lib.py(2)echo_platform() 1 def echo_platform(platform: str): ----> 2 print("You are running on:" + platform) 3 ipdb> y "Why is platform coming through as None?" The variable `platform` is coming through as `None` because the environment variable `"PLATFORM"` is not set in your system's environment variables. [...] It’s easy to install and set up: pip install chatpdb export OPENAI_API_KEY=.... import chatpdb chatpdb.set_trace() What my project does: A drop-in replacement for ipdb or pdb, with ChatGPT built in. Target audience: anyone who currently uses ChatGPT to debug or ask questions about their code. Comparison: Copilot offers similar functionality, but is a much heavier tool and harder to turn off/ignore when you don’t want it. stackexplain is probably the closest tool; however it lacks the ability to hook into your program in any other way than fully running the python process. chatpdb meets you where you are – AI tooling that’s only invoked when you need it. We hope you give it a try! We’d love any feedback or suggestions. Docs submitted by /u/the1024 [link] [comments]
Explaining one of the most common web API vulnerability classes - Broken Object Level Authorization in a practical manner. Providing a case study example based on the Damn Vulnerable RESTaurant API, including methods for identifying and preventing these vulnerabilities. https://devsec-blog.com/2024/04/web-api-security-champion-broken-object-level-authorization-owasp-top-10/ submitted by /u/theowni [link] [comments]
I recently open-sourced a package and the tooling was a bit of a struggle. I decided to write down the steps & all the tools needed to open-source a Python package in a scalable way that invite users and contributors. https://jonathanadly.com/open-sourcing-a-python-project-the-right-way-in-2024 Happy to hear your feedback! submitted by /u/Vegetable_Study3730 [link] [comments]
Does anyone know what’s going on behind the scenes of PyPI (who maintains `pip install`)? It seems like they’ve stopped processing support tickets over a month ago. I’ve hit the data limit for my package [PyBoy](https://github.com/baekalfen/pyboy), and I immediately posted a ticket to get the limit increased (as others have successfully done). But after more than a month, and several attempts at contacting the support team, I’ve heard nothing back, and I’ve run out of options. Does anyone know what’s happening, or how to get a comment from PyPI? My ticket is this one: https://github.com/pypi/support/issues/3757 submitted by /u/baekalfen [link] [comments]
Edit: Just to clarify I'm not looking to do anything risky, definitely not putting my arse on the line. I spoke to a colleague that is on the same boat and he uses Power automate desktop (company approved) a lot, very successfully. Apparently Python can be used with Power automate, so I'll see how it goes. Thanks for all the tips! I’m currently a Computer Science student and work as an Applications Support Engineer. I have been encountering a lot of manual and repetitive tasks at my job that I’m eager to automate to improve my productivity. However, theres a lot of IT restrictions on my work laptop. Even the developers on another team face similar challenges and end up using their personal devices for certain tasks due to these limitations. Previously, in a different role within the same company, I successfully automated tasks using Excel Power Query, Excel VBA, and Power Automate. I’m new to Python and would like to leverage it for automation as well, but I’m unsure how to navigate the IT restrictions. Does anyone have experience or advice on how to deal with such situations? Any workarounds or tips on using Python in a restricted IT environment would be greatly appreciated! Thank you in advance for your help! submitted by /u/Successful_Day_4547 [link] [comments]
Article link: https://rishiraj.me/articles/2024-04/python_subinterpreter_parallelism I have written an article, which should be helpful to folks at all experience levels, covering various multi-tasking paradigms in computers, and how they apply in CPython, with its unique limitations like the Global Interpreter Lock. Using this knowledge, we look at traditional ways to achieve "true parallelism" (i.e. multiple tasks running at the same time) in Python. Finally, we build a solution utilizing newer concepts in Python 3.12 to run any arbitrary pure Python code in parallel across multiple threads. All the code used to achieve this, along with the benchmarking code are available in the repository linked in the blog-post. This is my first time writing a technical post in Python. Any feedback would be really appreciated! 😊 submitted by /u/ThatsAHumanPerson [link] [comments]
Hey Python community! I created a stable diffusion pipeline to convert reference images to prompt and used it along with text prompt to generate variations of reference image. Explainer video here: https://www.youtube.com/watch?v=x9VryjEcxzk submitted by /u/Maleficent_Yak_993 [link] [comments]
Weekly Thread: Professional Use, Jobs, and Education 🏢 Welcome to this week's discussion on Python in the professional world! This is your spot to talk about job hunting, career growth, and educational resources in Python. Please note, this thread is not for recruitment. How it Works: Career Talk: Discuss using Python in your job, or the job market for Python roles. Education Q&A: Ask or answer questions about Python courses, certifications, and educational resources. Workplace Chat: Share your experiences, challenges, or success stories about using Python professionally. Guidelines: This thread is not for recruitment. For job postings, please see r/PythonJobs or the recruitment thread in the sidebar. Keep discussions relevant to Python in the professional and educational context. Example Topics: Career Paths: What kinds of roles are out there for Python developers? Certifications: Are Python certifications worth it? Course Recommendations: Any good advanced Python courses to recommend? Workplace Tools: What Python libraries are indispensable in your professional work? Interview Tips: What types of Python questions are commonly asked in interviews? Let's help each other grow in our careers and education. Happy discussing! 🌟 submitted by /u/AutoModerator [link] [comments]
I run an API (Python + Flask + Gunicorn) which has an endpoint that's calls a nested processes, which can sometimes take 6 seconds, to sometimes 60+ seconds to complete. The time it takes isn't a measure of my API's performance really, but rather its bottlenecked by an external service that the process has to rely on. Im looking for any suggested libraries that can safely handle this secondary process asynchronously with a long timeout. Ideally the API endpoint is called, and it won't have to wait for this process to finish if it takes more than 30+ seconds. Here's a mockup of what the chain of commands look like. Process 1: core.slurm.SlurmUser - addSlurmUser took: 1.6083331108093262. This process then calls core.slurm.SlurmUser - addScratch (which takes sometimes 6 seconds, sometimes 30+ seconds) Child Process 2: core.slurm.SlurmUser - addScratch took: 29.492394208908081 Any advice? Am I missing something that I can take advantage of already? The App is built using Flask + Gunicorn workers. Wondering If Celery is something that can nicely integrate with my current setup, or is it something that would replace Gunicorn in this scenario? submitted by /u/zacky2004 [link] [comments]
I’m aware there is a big open source community aspect to the python community. So my question is: “would uploading commercial software as a pip package offend the community?” I’m hesitant if this is normal, and if users would consider trials for components that would then make their project ask for a license to work after some time. Just wondering if it’s worth the effort to make products more accessible, or if it would just rub users the wrong way. submitted by /u/Normal_Antenna [link] [comments]
I'm working on a project involving a Python application with multiple interacting modules and am looking for some advice on deploying it efficiently on Google Cloud Platform. Here's a brief overview of what the application entails: Functionality Overview: • Data Fetching: One module fetches data from external websites. • Data Extraction: Another module extracts data from another GCP project. • Data Processing: Variables are processed across various modules. • CSV Output: The app generates and stores CSV files. • Scheduled Running: The application should run automatically once daily. I would greatly appreciate your insights on the following questions: What are the best GCP services to use for this type of application? Recommended Python libraries for handling web data retrieval in this case? How to proceed step by step? I intend to first deploy my Python application using placeholder inputs and simple printed outputs. Then, I plan to integrate the data fetching component and implement the creation and storage of CSV files. Does this strategy seem logical to you? Currently, I only tried to deploy my Python code using Google Cloud Functions, but I find this method impractical because I must deploy each module separately. I would prefer a solution that allows me to deploy all my code at once, where the Python modules can smoothly pass variables among themselves. Thank you in advance for your help! submitted by /u/ste042024 [link] [comments]
I got grumpy with our Scrum process, and I thought about what kinda tool I'd love to work with... so I started making it in my beloved Python. I think it's mature enough I'd like to start giving it to other people. It's called ScrumMD. It's open source, and you can already install it with pip (pip install scrummd) if you've got Python 3.10+. Documented, with tutorials on https://scrummd.readthedocs.io/en/stable/ and source on https://github.com/lkingsford/scrummd What my project does Short version is that it's some tools to support you storing all of your Scrum cards (or, I guess, other cards - tickets perhaps?) in markdown format on a local machine. There's intentionally a lot of flexibility - so, every card needs a summary, but everything else is fair game. You can configure to require fields for some collections (like needing status in stories), or limit fields (like requiring status be 'Done' or 'In Progress'). Target Audience Limited, but public. It's chief audience is software engineers who work in self-organising teams. It's for teams who use processes like Scrum, but don't need the bureaucracy layers. Honestly - I know it's niche. Heck, I won't even be using it at work myself. But, if I could, I would - because I actually like using it. Comparison I'm not aware of any CLI tools that do this kinda thing - particularly with local markdown files. The chief tool it would replace is a project/scrum management tool like Jira, or an issue tracker like the one that comes with GitHub. Both of those are far more friendly to a non-technical audience than ScrumMD is. Technical things Technically - it's pretty darned vanilla. The only non-included bits I'm using are a toml library on Python 3.10, Sphynx for Doco, Setup Tools and Pytest. I use Pylint and Mypy for my own sake. Wasn't anything technically novel going on - although it's the first time I've tried submitting anything to PyPi myself. It's been fun to try setting up CI in GitHub - I'd used a number of other CI/CD frameworks, but I think I like GitHub actions. Happy to talk through anything I've done here, and would really be genuinely pleased to be forced to talk my way through any design decisions or hear ways I could have structured it better. submitted by /u/thelochok [link] [comments]
Weekly Thread: Beginner Questions 🐍 Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you. How it Works: Ask Anything: Feel free to ask any Python-related question. There are no bad questions here! Community Support: Get answers and advice from the community. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources. Guidelines: This thread is specifically for beginner questions. For more advanced queries, check out our Advanced Questions Thread. Recommended Resources: If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance. Example Questions: What is the difference between a list and a tuple? How do I read a CSV file in Python? What are Python decorators and how do I use them? How do I install a Python package using pip? What is a virtual environment and why should I use one? Let's help each other learn Python! 🌟 submitted by /u/AutoModerator [link] [comments]
Version 1.1.1 of UXsim is released, which improves performance significantly. Main Changes in 1.1.1 Add setting to adjust vehicle logging time interval via World.vehicle_logging_timestep_interval By lowering the interval (e.g., World.vehicle_logging_timestep_interval=2), the simulation time can be reduced (~20% speed up), and we can obtain vehicle trajectory data with slightly less accuracy. The logging setting does not affect the internal simulation accuracy. Only the outputted trajectories are affected. By setting World.vehicle_logging_timestep_interval=-1, the record_log is turned off, and the simulation time can be significantly reduced (~40% speed up). This addresses Issue #58 Correct route choice behavior Vehicle.links_prefer and Vehicle.links_avoid work correctly now. UXsim UXsim is a free, open-source macroscopic and mesoscopic network traffic flow simulator written in Python. It simulates the movements of car travelers and traffic congestion in road networks. It is suitable for simulating large-scale (e.g., city-scale) traffic phenomena. UXsim is especially useful for scientific and educational purposes because of its simple, lightweight, and customizable features, but users are free to use UXsim for any purpose. submitted by /u/Balance- [link] [comments]
https://github.com/Never-Over/bridge The Problem We built bridge to solve the most frustrating part of any new project — infrastructure. Whenever you spin up a new Django project, you usually have to manually configure Postgres, background workers, a task queue, and more. The problem is amplified when you go to deploy your application — hosting providers don’t understand anything about what you’ve configured already, so you have to run through an even more complicated process to set up the same infrastructure in a deployed environment. What My Project Does bridge is a pip-installable package that spins up all of the infrastructure you need, and automatically connects it to your Django project. By adding a single line to your Django project's settings.py file, bridge configures everything for you — this means you don’t need to mess with DATABASES, BROKER_URL, or other environment variables to connect to these services. bridge also gives you the access you need to manage these services, including a database and Redis shell, as well as a Flower instance for monitoring background tasks. When you’re ready to deploy, bridge can handle that as well. By running bridge init render, bridge will write all of the configuration necessary to deploy your application on Render, including a button to trigger deploys straight from your README. If you don’t want all of these services, (say you already have a database, and just want to add background workers) bridge supports that too! It can automate everything you need and nothing you don’t. Target Audience This tool is intended for Django developers who are getting new projects off the ground. Comparison Compared to how Django functions on its own, bridge abstracts and manages all of the services you'd need to manually set up and configure. Compared to other boilerplate or starter pack repos, bridge is more lightweight and doesn't force you to change your application code. It is focused only on handling all of the required dependencies, startup, and teardown of your infrastructure. How it Works bridge is built on top of Docker, so you get fully isolated and up-to-date versions of Postgres and Redis from the beginning. Celery and Flower need to run on top of your app code, so we hook into runserver to spin these up as background processes. If you need to spin things down, bridge stop will conveniently shut down all services that it’s started. Coming Soon In the future, we want to add support for more services (jupyter, mail/mailhog etc), more hosting providers (Heroku, Railway, etc.), and more configuration (env vars, optional dependencies, etc). bridge is and always will be fully open source. Please give it a try and we’d love any feedback! Github Docs PyPI submitted by /u/the1024 [link] [comments]
I made a cheat sheet of all common operations on Python's many data structures. This include both the built-in data structures and all common standard library data structures. The time complexities of different data structures in Python If you're unfamiliar with time complexity and Big O notation, be sure to read the first section and the last two sections. I also recommend Ned Batchelder's talk/article that explains this topic more deeply. submitted by /u/treyhunner [link] [comments]
Today I Learned (TIL) You learn something new every day; what did you learn today? Submit interesting and specific facts about something that you just found out here.
submitted by /u/Make_the_music_stop [link] [comments]
Reddit Science This community is a place to share and discuss new scientific research. Read about the latest advances in astronomy, biology, medicine, physics, social science, and more. Find and submit new publications and popular science coverage of current research.