Programmers are always looking for ways to make their code more efficient. One way to do this is to use a faster loop. Python is a high-level programming language that is widely used by developers and software engineers. It is known for its readability and ease of use. However, one downside of Python is that its loops can be slow. This can be a problem when you need to process large amounts of data. There are several ways to make Python loops faster. One way is to use a faster looping construct, such as C. Another way is to use an optimized library, such as NumPy. Finally, you can vectorize your code, which means converting it into a format that can be run on a GPU or other parallel computing platform. By using these techniques, you can significantly speed up your Python code.
According to Vladislav Zorov, If not talking about NumPy or something, try to use list comprehension expressions where possible. Those are handled by the C code of the Python interpreter, instead of looping in Python. Basically same idea like the NumPy solution, you just don’t want code running in Python.
Example: (Python 3.0)
lst = [n for n in range(1000000)]
def loops():
newlst = []
for n in lst:
newlst.append(n * 2)
return newlst
def lstcomp():
return [n * 2 for n in lst]
from timeit import timeit
print(timeit(loops, number=100))
#18.953254899999592 seconds
print(timeit(lstcomp, number=100))
#11.669047399991541 seconds
Or Do this in Python 2.0
Python list traversing tip:
Instead of this: for i in range(len(l)): x = l[i]
Use this for i, x in enumerate(l): …
TO keep track of indices and values inside a loop.
Twice faster, and the code looks better.
Another option is to write loops in C instead of Python. This can be done by using a Python extension module such as pyximport. By doing this, programmers can take advantage of the speed of C while still using the convenient syntax of Python.
Finally, developers can also improve the performance of their code by making use of caching. By caching values that are computed inside a loop, programmers can avoid having to recalculate them each time through the loop. By taking these steps, programmers can make their Python code more efficient and faster.
Very Important: Don’t worry about code efficiency until you find yourself needing to worry about code efficiency.
The place where you think about efficiency is within the logic of your implementations.
Get 20% off Google Google Workspace (Google Meet) Standard Plan with the following codes: 96DRHDRA9J7GTN6 Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE (Email us for more codes)
Hello! I am looking for the best Python App for iPads because I don‘t want to buy a laptop and I have a PC at home. Any Suggestions? Thanks submitted by /u/Designer_Ad5684 [link] [comments]
Weekly Thread: Meta Discussions and Free Talk Friday 🎙️ Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related! How it Works: Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting. Guidelines: All topics should be related to Python or the /r/python community. Be respectful and follow Reddit's Code of Conduct. Example Topics: New Python Release: What do you think about the new features in Python 3.11? Community Events: Any Python meetups or webinars coming up? Learning Resources: Found a great Python tutorial? Share it here! Job Market: How has Python impacted your career? Hot Takes: Got a controversial Python opinion? Let's hear it! Community Ideas: Something you'd like to see us do? tell us. Let's keep the conversation going. Happy discussing! 🌟 submitted by /u/AutoModerator [link] [comments]
Try running this code: import numpy for x in range(60, 64): print(x) y = numpy.arange(x + 1)[-1] assert x == y assert x == int(y) assert (1 << x) == (1 << int(y)) assert (1 << x) == (1 << y) Python version is 3.11.2, NumPy is 1.24.2. Edit: forgot the import. submitted by /u/drbobb [link] [comments]
Hey everyone! I find myself in an unusual situation - a friend reached out to me if I could help his company with technical interviews for Python roles which will heavily focus on web scraping. I have conducted interviews in the past, but I haven't done any web scraping, so I'm reaching out to the community: what are some theoretical questions that you feel would be a good test of whether the candidate knows what they're doing when it comes to web scraping and ideally that you as a candidate would like to hear (engaging/interesting/not the same question for the 100th time)? Additionally - is there some best practice for practical tasks that could be completed on some live coding platform that would involve scraping? Or maybe you had a memorable (in the positive sense 🙂 ) live coding task? Lastly - are any of the live coding platforms standouts right now, ideally that could be used for free for 2-4 sessions? repl.it? Thanks in advance for your help with this - I really don't want to let my friend down 🙂 ——— As for the theoretical questions that I have right now (in abbreviated form 🙂 ), with "expected answers" in parentheses: What experience do you have with scraping? What libraries have you used? Have you used Selenium for scraping? (open ended; expect to hear about BeautifulSoup and/or Scrapy; Selenium a nice to have I guess?) Can you tell me about challenges you encountered in that project and how you overcame them? (if applicable; open ended) OR How do you deal with 400-codes, dynamic websites, captchas, anti-scraping measures? (somewhat open ended: retries/investigate for root cause of 400s; use Selenium; use appropriate tools for captchas; add delays, rotate User-Agents, use proxies, simulate browser headers etc.) How would you decide which to use on a project: BeautifulSoup or Scrapy? (Scrapy seems better for bigger/more complex projects, BS4 seems sufficient for smaller ones; open ended; not sure about this one tbh) How do you make sure that your scraping is ethical/legal/doesn't cause harm to the website (read TOS; rate limit; obtain permission; make sure the code is efficient, i.e. doesn't repeats calls unnecessarily) these ones seem good as well 🙂 submitted by /u/TheHollowJester [link] [comments]
Hey all, I love what Django has with django-fsm. I require something but without Django, as there is no user interaction with the workflow. All inputs and outputs are either rabbitmq or another api. This is to be a workflow management system. I am looking for state management backed up by database so we can not suffer if a k8s pod dies with all the states in memory. Some of our workflows could take weeks. Is it still best to make this in Django or is there a database backed state management module available? I see pytransitions but I would have to add database logging to it. submitted by /u/Extreme-Acid [link] [comments]
I just published a tutorial series on how to automate a Python script in Google Cloud using Cloud Functions and/or Cloud Run. Feedback would be great. Thanks! Automating Python with Google Cloud Automating Python with Google Cloud Functions Automating Python with Google Cloud Run submitted by /u/neb2357 [link] [comments]
This groundwork enables ecosystem players to consider deploying RAG solutions in real time without having to configure data retrieval systems. Link to Louis Brulé-Naudet's Hugging Face profile ```python import concurrent.futures import logging from datasets from tqdm import tqdm def dataset_loader( name:str, streaming:bool=True ) -> datasets.Dataset: """ Helper function to load a single dataset in parallel. Parameters ---------- name : str Name of the dataset to be loaded. streaming : bool, optional Determines if datasets are streamed. Default is True. Returns ------- dataset : datasets.Dataset Loaded dataset object. Raises ------ Exception If an error occurs during dataset loading. """ try: return datasets.load_dataset( name, split="train", streaming=streaming ) except Exception as exc: logging.error(f"Error loading dataset {name}: {exc}") return None def load_datasets( req:list, streaming:bool=True ) -> list: """ Downloads datasets specified in a list and creates a list of loaded datasets. Parameters ---------- req : list A list containing the names of datasets to be downloaded. streaming : bool, optional Determines if datasets are streamed. Default is True. Returns ------- datasets_list : list A list containing loaded datasets as per the requested names provided in 'req'. Raises ------ Exception If an error occurs during dataset loading or processing. Examples -------- >>> datasets = load_datasets(["dataset1", "dataset2"], streaming=False) """ datasets_list = [] with concurrent.futures.ThreadPoolExecutor() as executor: future_to_dataset = {executor.submit(dataset_loader, name): name for name in req} for future in tqdm(concurrent.futures.as_completed(future_to_dataset), total=len(req)): name = future_to_dataset[future] try: dataset = future.result() if dataset: datasets_list.append(dataset) except Exception as exc: logging.error(f"Error processing dataset {name}: {exc}") return datasets_list req = [ "louisbrulenaudet/code-artisanat", "louisbrulenaudet/code-action-sociale-familles", "louisbrulenaudet/code-assurances", "louisbrulenaudet/code-aviation-civile", "louisbrulenaudet/code-cinema-image-animee", "louisbrulenaudet/code-civil", "louisbrulenaudet/code-commande-publique", "louisbrulenaudet/code-commerce", "louisbrulenaudet/code-communes", "louisbrulenaudet/code-communes-nouvelle-caledonie", "louisbrulenaudet/code-consommation", "louisbrulenaudet/code-construction-habitation", "louisbrulenaudet/code-defense", "louisbrulenaudet/code-deontologie-architectes", "louisbrulenaudet/code-disciplinaire-penal-marine-marchande", "louisbrulenaudet/code-domaine-etat", "louisbrulenaudet/code-domaine-etat-collectivites-mayotte", "louisbrulenaudet/code-domaine-public-fluvial-navigation-interieure", "louisbrulenaudet/code-douanes", "louisbrulenaudet/code-douanes-mayotte", "louisbrulenaudet/code-education", "louisbrulenaudet/code-electoral", "louisbrulenaudet/code-energie", "louisbrulenaudet/code-entree-sejour-etrangers-droit-asile", "louisbrulenaudet/code-environnement", "louisbrulenaudet/code-expropriation-utilite-publique", "louisbrulenaudet/code-famille-aide-sociale", "louisbrulenaudet/code-forestier-nouveau", "louisbrulenaudet/code-fonction-publique", "louisbrulenaudet/code-propriete-personnes-publiques", "louisbrulenaudet/code-collectivites-territoriales", "louisbrulenaudet/code-impots", "louisbrulenaudet/code-impots-annexe-i", "louisbrulenaudet/code-impots-annexe-ii", "louisbrulenaudet/code-impots-annexe-iii", "louisbrulenaudet/code-impots-annexe-iv", "louisbrulenaudet/code-impositions-biens-services", "louisbrulenaudet/code-instruments-monetaires-medailles", "louisbrulenaudet/code-juridictions-financieres", "louisbrulenaudet/code-justice-administrative", "louisbrulenaudet/code-justice-militaire-nouveau", "louisbrulenaudet/code-justice-penale-mineurs", "louisbrulenaudet/code-legion-honneur-medaille-militaire-ordre-national-merite", "louisbrulenaudet/livre-procedures-fiscales", "louisbrulenaudet/code-minier", "louisbrulenaudet/code-minier-nouveau", "louisbrulenaudet/code-monetaire-financier", "louisbrulenaudet/code-mutualite", "louisbrulenaudet/code-organisation-judiciaire", "louisbrulenaudet/code-patrimoine", "louisbrulenaudet/code-penal", "louisbrulenaudet/code-penitentiaire", "louisbrulenaudet/code-pensions-civiles-militaires-retraite", "louisbrulenaudet/code-pensions-retraite-marins-francais-commerce-peche-plaisance", "louisbrulenaudet/code-pensions-militaires-invalidite-victimes-guerre", "louisbrulenaudet/code-ports-maritimes", "louisbrulenaudet/code-postes-communications-electroniques", "louisbrulenaudet/code-procedure-civile", "louisbrulenaudet/code-procedure-penale", "louisbrulenaudet/code-procedures-civiles-execution", "louisbrulenaudet/code-propriete-intellectuelle", "louisbrulenaudet/code-recherche", "louisbrulenaudet/code-relations-public-administration", "louisbrulenaudet/code-route", "louisbrulenaudet/code-rural-ancien", "louisbrulenaudet/code-rural-peche-maritime", "louisbrulenaudet/code-sante-publique", "louisbrulenaudet/code-securite-interieure", "louisbrulenaudet/code-securite-sociale", "louisbrulenaudet/code-service-national", "louisbrulenaudet/code-sport", "louisbrulenaudet/code-tourisme", "louisbrulenaudet/code-transports", "louisbrulenaudet/code-travail", "louisbrulenaudet/code-travail-maritime", "louisbrulenaudet/code-urbanisme", "louisbrulenaudet/code-voirie-routiere" ] dataset = load_datasets( req=req, streaming=True ) ``` submitted by /u/louisbrulenaudet [link] [comments]
Hello everyone! I am proud to introduce a Makefile Parser for Python that I think will be useful! the link is Thanatisia/makefile-parser-python As an introduction, I have been using python and been following this subreddit for quite awhile now, but this is my first post Recently, I've been planning a side-project involving the use of Makefiles in Python, and I required the use of a Makefile parser ala json or pyyaml - whereby you would import a Makefile into python as objects/dictionary/lists. Hence, I started searching for a Makefile Parser. The only parser I've found is PyMake(2) which is cool but it hasnt been updated since 2017 from what I recall and that it is more of a CLI utility, so with that in mind, I went about to make it I hope that this will be as useful as it is for me, currently I am using this in a side project and i'm able to format it such that its printing out a typical Makefile structure right off the bat, which is pretty nice. Additional information to the project What My Project Does This is a Makefile Parser, made in Python. The operational workflow is basically Start --> Import File into dictionary --> Manipulation and Usage --> Processing --> Output --> Export Target Audience (e.g., Is it meant for production, just a toy project, etc.) This is a library/package/module by design, much like json or pyyaml as mentioned but a smaller scale at the moment as its just started. I'm not sure if it applies for you but its a parser/importer Comparison (A brief comparison explaining how it differs from existing alternatives.) I'm not sure if there are any other Makefile Parsers other than Pymake2, but the idea is similar to the aforementioned ideas submitted by /u/Cybasura [link] [comments]
Weekly Thread: Professional Use, Jobs, and Education 🏢 Welcome to this week's discussion on Python in the professional world! This is your spot to talk about job hunting, career growth, and educational resources in Python. Please note, this thread is not for recruitment. How it Works: Career Talk: Discuss using Python in your job, or the job market for Python roles. Education Q&A: Ask or answer questions about Python courses, certifications, and educational resources. Workplace Chat: Share your experiences, challenges, or success stories about using Python professionally. Guidelines: This thread is not for recruitment. For job postings, please see r/PythonJobs or the recruitment thread in the sidebar. Keep discussions relevant to Python in the professional and educational context. Example Topics: Career Paths: What kinds of roles are out there for Python developers? Certifications: Are Python certifications worth it? Course Recommendations: Any good advanced Python courses to recommend? Workplace Tools: What Python libraries are indispensable in your professional work? Interview Tips: What types of Python questions are commonly asked in interviews? Let's help each other grow in our careers and education. Happy discussing! 🌟 submitted by /u/AutoModerator [link] [comments]
I recently ran into some challenges while trying to work with Apple's .numbers files on Linux. After a bit of experimentation, I figured out a workflow that simplifies the process. If you're planning to use .numbers files and need to load them into pandas, I've created a tutorial that covers the required dependencies and the steps to follow: https://nezhar.com/blog/load-apple-numbers-files-python-pandas-using-containers/. Has anyone else here worked with .numbers files in Python? I’d love to hear about your experiences or any tips you might have. submitted by /u/nez_har [link] [comments]
I have a repo in GitHub with Jupyter Notebooks for students. On each notebook I have a link where the student can open the notebook in Colab (by changing the URL from https://github.com/ to https://githubtocolab.com/). The problem is that some school boards have Colab blocked. Is there a similar technique I can use to open the notebooks in Kaggle? Are there other sites I should be considering? Thanks! submitted by /u/pbeens [link] [comments]
While Python doesn't explicitly support Type-Level Programming, I've had some success with it. I think this may be of interest to the Python community, so I'm sharing a POC (Proof Of Concept) I've written. This POC is a statically typed list that encodes its length in its type so that, for instance, when you concatenate two lists, the result has the correct length, all done at type-checking time. Read more on GitHub submitted by /u/Kiuhnm [link] [comments]
Weekly Thread: Beginner Questions 🐍 Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you. How it Works: Ask Anything: Feel free to ask any Python-related question. There are no bad questions here! Community Support: Get answers and advice from the community. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources. Guidelines: This thread is specifically for beginner questions. For more advanced queries, check out our Advanced Questions Thread. Recommended Resources: If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance. Example Questions: What is the difference between a list and a tuple? How do I read a CSV file in Python? What are Python decorators and how do I use them? How do I install a Python package using pip? What is a virtual environment and why should I use one? Let's help each other learn Python! 🌟 submitted by /u/AutoModerator [link] [comments]
i need some python library that can automatically generate a synchronization map between a list of text fragments and an audio file containing the narration of the text. aeneas does exactly that except it doesnt work on higher than 2.7 with windows. submitted by /u/jonnisaesipylsur [link] [comments]
Call for Proposals open through May 20 PyOhio is a fun, friendly, free general Python conference now in its Nth year (N is large). This year it will be Sat & Sun Jul 27-28, 2024 at The Westin Cleveland Downtown in Cleveland, OH (its first year outside Columbus)! Hope to see you there! submitted by /u/catherinedevlin [link] [comments]
I'm working on a simple PowerShell script called PyCompatManager that helps manage package installations for legacy Python versions. What My Project Does: PyCompatManager checks your current Python environment and finds the latest compatible release of a specified package on PyPI. This can be helpful when working with older Python versions that may not be compatible with the latest package releases. Target Audience: This script is primarily aimed at developers and system administrators who need to manage Python packages in environments with legacy Python versions Or developers who want to maintain legacy projects. It can be particularly useful for maintaining older applications or systems that cannot be easily upgraded to newer Python versions. Comparison: While tools like pip can be used to install packages, they may not always identify the latest compatible version for legacy Python environments. PyCompatManager focuses specifically on finding compatible releases, ensuring that installations work smoothly with your specific Python version. It can also install pip, setuptools, and wheel even when get-pip.py bootstrap fails. Goals My main goals for this project are: To provide a simple and effective way to manage legacy Python package compatibility. To gather feedback and contributions from the community to improve the script's functionality and usability. To potentially expand the script's capabilities to support additional package management tasks. I'd love to get your feedback and contributions! Please check out the script on GitHub I'm particularly interested in: Code reviews: Any feedback on the script's structure, logic, and efficiency is greatly appreciated. Feature suggestions: What additional functionalities would be useful for managing legacy Python packages? Contributions: Feel free to fork the repository and submit pull requests with improvements or new features. Thanks for your time! submitted by /u/foadsf [link] [comments]
Hi everyone, I developed a simple plugin for mkdocs-material. What My Project Does: In simple terms, this plugin allows users to mark pages as read and it shows a checkmark icon in navigation bar for the pages that was marked as read. The plugin adds a button under the page content and when users clicks, it stores read date in localStorage. It shows a checkmark icon in the navigation panels for the pages that marked as read. It also shows a "document updated" icon if the pages that marked as read got an update. Target Audience: Anyone who has a website built with Material for MkDocs (a.k.a. mkdocs-material). This plugin could be useful for the pages that user read by an order and wants to continue from where it left. I guess the Learn page of FastAPI documentation could be a great example to that. Comparison: No alternative plugin exist for Material for MkDocs afaik Project repo: github.com/berk-karaal/mkdocs-material-mark-as-read You can try this plugin on the documentation website. Let me know what you think about this plugin. Also please share if you have a feature request or an idea to improve this plugin. submitted by /u/br2krl [link] [comments]
Hello r/Python, What My Project Does: This script is designed to automate the management of Python virtual environments and dependencies, streamlining the setup process for Python projects. It facilitates a more efficient workflow for managing project-specific environments and package installations, especially in a professional development setting. Target Audience: This tool is particularly beneficial for developers and IT professionals looking for a systematic approach to environment management. It is designed to integrate into existing workflows, providing a reliable and consistent method for managing Python environments and dependencies. Features include: Automated creation of Python virtual environments. Batch installation of packages from a predefined array or a requirements.txt file. Can import, update, upgrade, and remove packages within the virtual environment. Functionalities for listing all installed packages for transparency and audit purposes. Benefits: Efficiency: Reduces manual setup and management of virtual environments. Consistency: Ensures uniform environments across development stages and projects. Flexibility: Supports custom package lists and requirements, adaptable to project-specific needs. Comparison: Fast setup , updates, and removal. Set custom paths to store your files Everyone is invited to download, implement, and provide feedback on this script to further refine its capabilities to meet professional standards and requirements (AKA just be really useful). For access and further details, please visit: GitHub submitted by /u/SAV_NC [link] [comments]
Today I Learned (TIL) You learn something new every day; what did you learn today? Submit interesting and specific facts about something that you just found out here.
Reddit Science This community is a place to share and discuss new scientific research. Read about the latest advances in astronomy, biology, medicine, physics, social science, and more. Find and submit new publications and popular science coverage of current research.