Python Libraries to Learn Before Tackling Data Analysis
To tackle data analysis effectively in Python, it's crucial to become familiar with several libraries that streamline the process of data manipulation, exploration, and visualization. Here's a breakdown of the essential libraries:
1. NumPy
- Purpose: Numerical computing.
- Why Learn It: NumPy provides support for large multi-dimensional arrays and matrices, along with a collection of mathematical functions to operate on these arrays efficiently.
- Key Features:
- Fast array processing.
- Mathematical operations on arrays (e.g., sum, mean, standard deviation).
- Linear algebra operations.
2. Pandas
- Purpose: Data manipulation and analysis.
- Why Learn It: Pandas offers data structures like DataFrames, making it easier to handle and analyze structured data.
- Key Features:
- Reading/writing data from CSV, Excel, SQL databases, and more.
- Handling missing data.
- Powerful group-by operations.
- Data filtering and transformation.
3. Matplotlib
- Purpose: Data visualization.
- Why Learn It: Matplotlib is one of the most widely used plotting libraries in Python, allowing for a wide range of static, animated, and interactive plots.
- Key Features:
- Line plots, bar charts, histograms, scatter plots.
- Customizable charts (labels, colors, legends).
- Integration with Pandas for quick plotting.
4. Seaborn
- Purpose: Statistical data visualization.
- Why Learn It: Built on top of Matplotlib, Seaborn simplifies the creation of attractive and informative statistical graphics.
- Key Features:
- High-level interface for drawing attractive statistical graphics.
- Easier to use for complex visualizations like heatmaps, pair plots, etc.
- Visualizations based on categorical data.
5. SciPy
- Purpose: Scientific and technical computing.
- Why Learn It: SciPy builds on NumPy and provides additional functionality for complex mathematical operations and scientific computing.
- Key Features:
- Optimized algorithms for numerical integration, optimization, and more.
- Statistics, signal processing, and linear algebra modules.
6. Scikit-learn
- Purpose: Machine learning and statistical modeling.
- Why Learn It: Scikit-learn provides simple and efficient tools for data mining, analysis, and machine learning.
- Key Features:
- Classification, regression, and clustering algorithms.
- Dimensionality reduction, model selection, and preprocessing utilities.
7. Statsmodels
- Purpose: Statistical analysis.
- Why Learn It: Statsmodels allows users to explore data, estimate statistical models, and perform tests.
- Key Features:
- Linear regression, logistic regression, time series analysis.
- Statistical tests and models for descriptive statistics.
8. Plotly
- Purpose: Interactive data visualization.
- Why Learn It: Plotly allows for the creation of interactive and web-based visualizations, making it ideal for dashboards and presentations.
- Key Features:
- Interactive plots like scatter, line, bar, and 3D plots.
- Easy integration with web frameworks.
- Dashboards and web applications with Dash.
9. TensorFlow/PyTorch (Optional)
- Purpose: Machine learning and deep learning.
- Why Learn It: If your data analysis involves machine learning, these libraries will help in building, training, and deploying deep learning models.
- Key Features:
- Tensor processing and automatic differentiation.
- Building neural networks.
10. Dask (Optional)
- Purpose: Parallel computing for data analysis.
- Why Learn It: Dask enables scalable data manipulation by parallelizing Pandas operations, making it ideal for big datasets.
- Key Features:
- Works with NumPy, Pandas, and Scikit-learn.
- Handles large data and parallel computations easily.
Focusing on NumPy, Pandas, Matplotlib, and Seaborn will set a strong foundation for basic data analysis.
Why Python Will Thrive: Future Trends and Applications
Python has already made a significant impact in the tech world, and its trajectory for the future is even more promising. From its simplicity and versatility to its widespread use in cutting-edge technologies, Python is expected to continue thriving in the coming years. Considering the kind support of Python Course in Chennai Whatever your level of experience or reason for switching from another programming language, learning Python gets much more fun.
Let's explore why Python will remain at the forefront of software development and what trends and applications will contribute to its ongoing dominance.
1. Artificial Intelligence and Machine Learning
Python is already the go-to language for AI and machine learning, and its role in these fields is set to expand further. With powerful libraries such as TensorFlow, PyTorch, and Scikit-learn, Python simplifies the development of machine learning models and artificial intelligence applications. As more industries integrate AI for automation, personalization, and predictive analytics, Python will remain a core language for developing intelligent systems.
2. Data Science and Big Data
Data science is one of the most significant areas where Python has excelled. Libraries like Pandas, NumPy, and Matplotlib make data manipulation and visualization simple and efficient. As companies and organizations continue to generate and analyze vast amounts of data, Python’s ability to process, clean, and visualize big data will only become more critical. Additionally, Python’s compatibility with big data platforms like Hadoop and Apache Spark ensures that it will remain a major player in data-driven decision-making.
3. Web Development
Python’s role in web development is growing thanks to frameworks like Django and Flask, which provide robust, scalable, and secure solutions for building web applications. With the increasing demand for interactive websites and APIs, Python is well-positioned to continue serving as a top language for backend development. Its integration with cloud computing platforms will also fuel its growth in building modern web applications that scale efficiently.
4. Automation and Scripting
Automation is another area where Python excels. Developers use Python to automate tasks ranging from system administration to testing and deployment. With the rise of DevOps practices and the growing demand for workflow automation, Python’s role in streamlining repetitive processes will continue to grow. Businesses across industries will rely on Python to boost productivity, reduce errors, and optimize performance. With the aid of Best Online Training & Placement Programs, which offer comprehensive training and job placement support to anyone looking to develop their talents, it’s easier to learn this tool and advance your career.
5. Cybersecurity and Ethical Hacking
With cyber threats becoming increasingly sophisticated, cybersecurity is a critical concern for businesses worldwide. Python is widely used for penetration testing, vulnerability scanning, and threat detection due to its simplicity and effectiveness. Libraries like Scapy and PyCrypto make Python an excellent choice for ethical hacking and security professionals. As the need for robust cybersecurity measures increases, Python’s role in safeguarding digital assets will continue to thrive.
6. Internet of Things (IoT)
Python’s compatibility with microcontrollers and embedded systems makes it a strong contender in the growing field of IoT. Frameworks like MicroPython and CircuitPython enable developers to build IoT applications efficiently, whether for home automation, smart cities, or industrial systems. As the number of connected devices continues to rise, Python will remain a dominant language for creating scalable and reliable IoT solutions.
7. Cloud Computing and Serverless Architectures
The rise of cloud computing and serverless architectures has created new opportunities for Python. Cloud platforms like AWS, Google Cloud, and Microsoft Azure all support Python, allowing developers to build scalable and cost-efficient applications. With its flexibility and integration capabilities, Python is perfectly suited for developing cloud-based applications, serverless functions, and microservices.
8. Gaming and Virtual Reality
Python has long been used in game development, with libraries such as Pygame offering simple tools to create 2D games. However, as gaming and virtual reality (VR) technologies evolve, Python’s role in developing immersive experiences will grow. The language’s ease of use and integration with game engines will make it a popular choice for building gaming platforms, VR applications, and simulations.
9. Expanding Job Market
As Python’s applications continue to grow, so does the demand for Python developers. From startups to tech giants like Google, Facebook, and Amazon, companies across industries are seeking professionals who are proficient in Python. The increasing adoption of Python in various fields, including data science, AI, cybersecurity, and cloud computing, ensures a thriving job market for Python developers in the future.
10. Constant Evolution and Community Support
Python’s open-source nature means that it’s constantly evolving with new libraries, frameworks, and features. Its vibrant community of developers contributes to its growth and ensures that Python stays relevant to emerging trends and technologies. Whether it’s a new tool for AI or a breakthrough in web development, Python’s community is always working to improve the language and make it more efficient for developers.
Conclusion
Python’s future is bright, with its presence continuing to grow in AI, data science, automation, web development, and beyond. As industries become increasingly data-driven, automated, and connected, Python’s simplicity, versatility, and strong community support make it an ideal choice for developers. Whether you are a beginner looking to start your coding journey or a seasoned professional exploring new career opportunities, learning Python offers long-term benefits in a rapidly evolving tech landscape.
Exploring Game-Changing Applications: Your Easy Steps to Learn Machine Learning:
Machine learning technology has truly transformed multiple industries and continues to hold enormous potential for future development. If you're considering incorporating machine learning into your business or are simply eager to learn more about this transformative field, seeking advice from experts or enrolling in specialized courses is a wise step. For instance, the ACTE Institute offers comprehensive machine learning training programs that equip you with the knowledge and skills necessary for success in this rapidly evolving industry. Recognizing the potential of machine learning can unlock numerous avenues for data analysis, automation, and informed decision-making.
Now, let me share my successful journey in machine learning, which I believe can benefit everyone. These 10 steps have proven to be incredibly effective in helping me become a proficient machine learning practitioner:
Step 1: Understand the Basics
Develop a strong grasp of fundamental mathematics, particularly linear algebra, calculus, and statistics.
Learn a programming language like Python, which is widely used in machine learning and provides a variety of useful libraries.
Step 2: Learn Machine Learning Concepts
Enroll in online courses from reputable platforms like Coursera, edX, and Udemy. Notably, the ACTE machine learning course is a stellar choice, offering comprehensive education, job placement, and certification.
Supplement your learning with authoritative books such as "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" by Aurélien Géron and "Pattern Recognition and Machine Learning" by Christopher Bishop.
Step 3: Hands-On Practice:
Dive into real-world projects using both simple and complex datasets. Practical experience is invaluable for gaining proficiency.
Participate in machine learning competitions on platforms like Kaggle to challenge yourself and learn from peers.
Step 4: Explore Advanced Topics
Delve into deep learning, a critical subset of machine learning that focuses on neural networks. Online resources like the Deep Learning Specialisation on Coursera are incredibly informative.
For those intrigued by language-related applications, explore Natural Language Processing (NLP) using resources like the "Natural Language Processing with Python" book by Steven Bird and Ewan Klein.
Step 5: Learn from the Community
Engage with online communities such as Reddit's r/Machine Learning and Stack Overflow. Participate in discussions, seek answers to queries, and absorb insights from others' experiences.
Follow machine learning blogs and podcasts to stay updated on the latest advancements, case studies, and best practices.
Step 6: Implement Advanced Projects
Challenge yourself with intricate projects that stretch your skills. This might involve tasks like image recognition, building recommendation systems, or even crafting your own AI-powered application.
Step 7: Stay updated
Stay current by reading research papers from renowned conferences like NeurIPS, ICML, and CVPR to stay on top of cutting-edge techniques.
Consider advanced online courses that delve into specialized topics such as reinforcement learning and generative adversarial networks (GANs).
Step 8: Build a Portfolio
Showcase your completed projects on GitHub to demonstrate your expertise to potential employers or collaborators.
Step 9: Network and Explore Career Opportunities
Attend conferences, workshops, and meetups to network with industry professionals and stay connected with the latest trends.
Explore job opportunities in data science and machine learning, leveraging your portfolio and projects to stand out during interviews.
In essence, mastering machine learning involves a step-by-step process encompassing learning core concepts, engaging in hands-on practice, and actively participating in the vibrant machine learning community. Starting from foundational mathematics and programming, progressing through online courses and projects, and eventually venturing into advanced topics like deep learning, this journey equips you with essential skills. Embracing the machine learning community and building a robust portfolio opens doors to promising opportunities in this dynamic and impactful field.
Our world is gradually evolving to become more technology-reliant. And one such technology expected to revolutionize the future is Machine learning.
With its advent; engineers program the computer to learn and provide data on its own with limited or no manual instructions.
Skill set for Machine Learning Engineer
Following skill set would be required for the best machine learning course
1.Programming language- Proficiency in many languages like Java programming for Map Reduce coding, R for visualization purpose, and Python for Machine learning.
2. Calculus and statistics- Most of the Machine learning algorithms are mathematical. Hence you should know your high school mathematics efficiently.
3. Signal Processing- You can expect exponential growth in your career; when you have the knowledge of signal processing.
4. Neural Networks- Due to their efficiency they have gained huge popularity in recent years.
5. Language processing- Machine learning deals with text data, voice data, and visual data. Hence the ability to program such data is mandatory.
Depending on their level of expertise ML Engineers may:
1. Study and transform the Data Sciences prototype.
2. Design machine learning system
3. Research and implement appropriate ML Algorithm and tools.
4. Develop Machine Learning Applications according to the rule of the industry
5. Look for Appropriate data sets and Data Representation techniques
6. Run machine learning Tests and experiments.
7. Execute Statistical Analysis and fine-tuning using test results
8. Train and Re-train systems when necessary.
9. Extend the existing ML libraries and framework.
Machine Learning Career Paths
MACHINE LEARNING ENGINEERS- Most coveted and Promising field of ML. They perform sophisticated programming to develop machines and systems. They learn and apply knowledge without specifications.
They; shape and develop efficient self-learning ML applications. And perform statistical analysis and fine-tuning using test results.
They also conduct experiments using programming languages like Java, R, Scala, and C++.
SKILLS REQUIRED: Must have a strong foundational knowledge of Mathematics, Statistics, and programming. Accomplished in software architecture, system design, data structures, data modeling, and ML algorithm.
DATA SCIENTIST-
Hailed as the “Sexiest Job of the 21st Century” by a Harvard Business review article. These are high-profile experts who leverage advanced technologies (like Big Data, AI, ML, Deep Learning, etc.) and analytical tools on a daily basis to collect, store, process, analyze, and interpret massive amounts of data. Their primary goal is to extract valuable insights from large datasets and convert them into business value.
SKILLS REQUIRED: like the MLE; they should acclaim Mathematics, Statistics, and programming skills.
And also have thorough experience in data mining and how to apply various statistical research techniques and use Big Data platforms (Hadoop, Pig, Hive, Spark, Flume, etc.).
NLP SCIENTIST
Natural Learning Process functions to coach machines the ability to understand natural human languages. NLP Scientists hold the core responsibility of designing and developing machines and applications. These machines can learn the patterns of speech of a human language and also translate spoken words into other languages.
SKILLS REQUIRED- They teach machines how to understand the nuances of human languages. Hence, they must be fluent in the syntax, spelling, and grammar of at least one language (the more, the better).
HUMAN-CENTRED MACHINE LEARNING DESIGNERS
This is an exclusive branch dedicated to designing ML algorithms centered around humans. These designers are responsible for creating intelligent systems that can “learn” the preferences and behavior patterns of individual humans through information processing and pattern recognition.
SKILLS REQUIRED- they must have an in-depth understanding of various ML concepts, algorithms, and how they function. And also have a good base in Mathematics and Statistics along with coding skills.
An excellent example is Amazon and Netflix's Recommendation Engine.
INDUSTRIES LEVERAGING MACHINE LEARNING ARE:
The Health care Industry
The Retail Industry
The Financial Service Industry
The Automotive Industry
Government Agencies
Transport Industries
Oil and Gas Industries.
AN AVERAGE INCOME- A machine learning engineer receives a clean slip of a roughly 7lkh per annum income in India.
CONCLUSION
While these are few careers in Machine Learning. Other profiles like Data Analyst, Data Architect, Cloud Architect, and Business Intelligence Developer also exist in this domain.
Every organization wants to capitalize on its data to gain insights, improve customer relations, increase the sale, or be competitive. ML has enough capability to help them achieve their goal.
Yet, Machine Learning's ability to automate, expect, and evolve is powerful, but that doesn't mean computers will take over the world. They still need human operators to provide context, to set parameters of operations, and to continue improving the algorithms.
Python is one of the most cherished programming dialects by designers, information researchers, programming engineers, and even programmers on account of its adaptability, adaptability, and item situated highlights.
Huge numbers of the web and portable applications we appreciate today is a direct result of Python's plentiful libraries, different systems, immense assortments of modules, and record expansions. Not just that, Python is extraordinary for building miniature task to large scale venture web benefits just as for supporting different sorts of programming dialects.
Even though it's an elevated level of language and can-do complex undertakings, Python is anything but difficult to learn and has a spotless linguistic structure. Consequently, it's recommendable for the two tenderfoots and experienced developers. Likewise, there are a lot of ways on the best way to learn Python; you can do self-study, enlist yourself in coding Bootcamp or school or watch online instructional exercises or take Python course London. The decision is yours to make!
Top 7 Uses of Python
With its close flawlessly excellence, Python is named as one of the top-picked programming dialects of most colleges and businesses. What makes it overwhelming is its wide applications to different fields and boundless conceivable outcomes to make astounding things out of scratch.
As per TechRepublic, Python engineer is one of the "10 Most In-Demand Tech Jobs of 2019". This is because of the ascent of AI and Machine Learning innovations in the worldwide market. As of February 23, 2019, the normal compensation for a Python designer is $123,201 every year in the United States, making it one of the most famous and worthwhile professions today.
In reality, there are numerous reasons why you ought to learn Python.
Furthermore, to comprehend further its degree and advantages, here are the Top 7 employments of Python.
Python can be utilized on the accompanying:
1. Various programming Paradigms
2. Web Testing
3. Data Extraction
4. Artificial Intelligence (AI) and Data Science Researches
5. Web Application and Internet Development
6. Database Easy Access, Interface Customization, and Quick System Integration
7. Cybersecurity
PyGame is a nice library for getting started with game development. It is a good way to learn programming in a "real programming language" while making games1. There is very "magic" going on in PyGame, and you can go from print("Hello World") to drawing a triangle on the screen and then to Pong or Tetris with a smooth learning curve.
PyGame is also an easy way to get your feet wet in game development if you know and use Python already, for example in machine learning or web development. That's how I came to learn PyGame.
2 Why Not?
PyGame is just a low-level library for creating windows, drawing on the screen, reading joystick input, and playing sounds. OpenFL, Löve2D, FNA and LibGDX fill similar niches in the Haxe, Lua, .Net and Java ecosystems, respectively. If you want a full-featured engine, use Unity3D, UE4, or Godot. If you don't want to learn to code, you will probably fare better with GameMaker or Stencyl2. If you want to tell an interactive story with little actual gameplay, maybe Twine or Ren'Py are more your thing.
3 Explicit Is Better Than Implicit
Maybe PyGame is not for everybody… However, if you do just want to make small 2D games, and don't want to rely on automagic functionality, PyGame could for you. That means you have to code your own game loop3, poll the keyboard or game pad, and update the screen yourself.
Your game loop will look something like this:
import pygame pygame.init() # ... # setup stuff here screen=pygame.display.set_mode(screen_size) clock=pygame.clock.Clock() running=True while running: clock.tick(30) events = pygame.event.get() keys = pygame.key.get_pressed() for e in events: if e.type==pygame.QUIT: running=False elif e.type==pygame.KEYDOWN and #... # handle jumping etcetera # ... # move sprites around, collision detection, etc # ... # draw pygame.display.update() pygame.quit()
The ease of just drawing pixels on the screen and the lack of hidden features sometimes makes it harder to get started making certain games genres. If you start from scratch in Unity3D, you already have a level and a camera and you can just create a sphere GameObject and check a box to enable the physics engine. If you draw a circle on the screen in PyGame, it's just pixels on the screen. These pixels don't know anything about cameras, raycasting, or physics, so if you want to move the view around, make your circle clickable, or have it bounce off walls, you have to connect the systems yourself. PyGame gives you tools to check for collisions, a Sprite class that is the closest equivalent to GameObject, and a way to find out where on the window the mouse was clicked.
If you want to use a pre-made physics engine like Box2D, or roll your own, you have to write some glue code, but PyGame will stay out of your way.
This can even be a tremendous advantage when you want to make a game like Tetris4, or like SimCity, where you don't want to use 3D collision detection and a physics engine anyway, and collision detection happens on a grid. A game like Braid, where you can rewind the entire game state, will also be much easier if you have full control over your game loop. You can do all these things in a big and opinionated engine, as Hearthstone (a card game made in Unity3D) has showed us, but if don't need these features, you sometimes have to code around them. If you code around features you don't need, or re-build your own systems on top of a commercial engine, you might end up writing more code than if you had just started from scratch5.
All of this goes double for tools like PuzzleScript, RPG Maker, GameMaker, Bitsy or Twine. These are all geared towards certain kinds of games, and while they make it really easy to make that kind of game, they make you jump through hoops if you want to build anything else.
4 Three Pain Points
Although I just told you that PyGame is a great library, it is not as popular among game developers as it should be. This is mostly due to three problems dating back to the Python 3 days: multi-threading, deployment, and HD screen resolutions.
Multi-threading: Due to the GIL, an unfortunate implementation detail in CPython6, you can't have two threads execute Python code at the same time. That means your multi-threaded Python programs will still run in parallel, but there will only ever be one thread running at any time. On a single-core machine, there is not that much to worry about, because you only have one core that can run code anyway, but on a multi-core machine, you can't really take advantage of the additional cores. Still, what you could do is have one thread handle input and draw to the screen, while another, long-running thread does complicated enemy AI calculations that take longer than one frame to complete. That won't make your code run any faster in aggregate, but it will can at least ensure that the tick-input-update-draw cycle of the game loop runs at a smooth frame rate while the AI takes a lot of time.
If you're using i/o operations from the Python standard library, long-running numpy code or clock.tick(FPS) in PyGame, these libraries do the right thing and tell the Python interpreter that they are entering a section of native code during which Python code can safely run in another thread. But in general, you could have a Python function that calls native code that doesn't return for a while, so the Python interpreter is executing the same line of Python and other threads can't run. If that bit of native code is waiting for input, your whole program will freeze until the input arrives, even if you have ten other threads waiting that could do useful things in the mean time.
There are two modules to help you with concurrency in Python 3: multiprocessing and asyncio. The first has been around since Python 2.6, and lets you execute Python code in another process. This is useful if you have long-running computations like enemy AI that you want to run on another core. The other, asyncio is new in Python 3.37. With asyncio, you can run multiple coroutines in the same thread. A scheduling object called an event loop will execute coroutines piecemeal whenever there is input available. Because asynchronous coroutines are basically Python functions with some extra syntactic sugar, you can have tens of thousands of them running at the same time, unlike processes or kernel threads, which need a comparatively large amount of RAM to create and some time to context-switch. The use case for asyncio is pretty much the opposite of the use case of multiprocessing: Lots of threads8 waiting for i/o instead of long-lived processes that crunch numbers.
Deployment: There used to be no simple and easy way to package Python 2 games for end users. Sure, you could ask them to install python, either from Python.org or from the package manager of their choice, and give them the source code. Or you could use py2exe or py2app and create portable versions of a python script, but you couldn't build Windows, Mac and Linux executables from the same configuration file, and only on the respective platform. There were multiple competing standards for packaging python modules as source distributions: distutils, setuptools, distribute, and distutils2.
In the mean time, a smart person invented cxfreeze, which would automatically package every single dependency, every resource file and library used by your program, into one big archive. Unfortunately, that approach tended to pull in a bunch of DLLs that are already present on most end users machines. You also need to have the script already running on the target platform to analyse the dependencies at runtime.
With Python 3, setuptools has won, and the others have been merged into it9. There is a new file format for packaging Python modules called wheels10, that lets users install modules with native code without a C compiler. PyInstaller11 lets you build self-contained, portable versions of scripts on Windows, Mac or Linux based on the specification, and pynsist12 lets you create a windows installer for a Python script from any platform, because it downloads windows-compatible wheels and includes them without setting up a cross-compiling environment. The Panda3D project is even working on a cross-platform binary packaging system for Python 3 games based on setuptools!
HD screens: The software-based rendering in PyGame gets slower proportionally to the number of pixels. If you draw a 16x16 character sprite onto the screen, that makes 256 pixels. If you blit a background picture into a 640x480 (VGA) window, that's 307200 pixels already. If you scale that window up to 1280x960, that's four times as many pixels, and 1080p full-HD resolution has 6.75 times as many pixels as VGA. Things that were quite fast at low resolutions become a real bottleneck on HD screens, and you really feel the burn when you run PyGame on devices with a slower CPU and a HD screen, like a phone or a Raspberry PI.
Even if you run your game logic at a low resolution, and just scale up the screen to HD with pygame.transform.scale in the final step, that still means moving lots of pixels around in software. Scaling your game up to full-screen on a HD display can easily take the majority of your processing time. And remember, we effectively have only one core!
If you could copy your 320x240 surface into a texture and scale up your pixel art with the GPU, you eliminate a major bottleneck. The GPU is surprisingly fast and energy-efficient in phones, tablets and on the raspberryPI.
Fortunately, instead of directly drawing pixels into your window, PyGame can give you an OpenGL context and let you use whatever OpenGL bindings you like. In the past, that would have been with PyOpenGL, or pyglet. With Python 3, we can use ModernGL. ModernGL is much simpler than the other two. It targets only modern versions of OpenGL. Unlike PyOpenGL and pyglet, there is no support for immediate mode or the old fixed-function pipeline. You have to use vertex buffer objects and custom shaders. It's fast, and it lets you do fancy post-processing effects in addition to scaling.
If you need more performance, or fancier effects, you can incrementally transition from drawing into PyGame Surfaces towards creating individual textures for your Sprites and rendering directly with OpenGL.
5 Other Cool Libraries
The missing level editor: Tiled13 is a great level editor. You can use PyTMX14 to read the maps files, and pyscroll15 to render them.
Discord Rich Presence: All the cool kids are on Discord16. With pypresence17, you can let your player's friends know what the game is called, which level they are on, and how many points they have scored. It definitely beats having "now playing Python3.exe" as your Discord status line.
Creating Videos: moviePy18 gives you a simple API to create, load and edit GIFs, WEBM and other video file formats.
6 More To Come
The problems I used to have in PyGame haven't disappeared, but in 2018 and with Python 3.6, they have become solvable. It is possible to started making games with PyGame, and to polish and deploy them on Steam and itch.io. There used to be a tool to deploy PyGame to Android phones and tablets, but that was based on Python 2.7 and some hacks to get everything to run. With the current effort to port PyGame to SDL2, we can soon deploy games written in PyGame to Windows, Mac, Linux, Android and all major consoles, which support SDL219
This post is longer than I thought already, so I'll take a break here. In the coming days I will write more about using the asyncio module to do i/o in PyGame without tanking your frame rate, using the moderngl module to scale up your pixel art, and trying out deploy-NG, a setuptools extension from Panda3D to distribute games to end users.
Footnotes:
1
I recommend Al Sweigart's tutorials at http://inventwithpython.com/, and http://inventwithpython.com/pygame/
2
Maybe you want to start designing games first instead of learning to program, so you start out with these tools, but I would recommend you learn at least a little bit of a real programming language as well, to get some perspective. If you learn C#, which is a real programming language, together with Unity or Godot, please look up which features are part of C# and which are unique to your engine.
3
To avoid common pitfalls with game loops, read these: http://www.koonsolo.com/news/dewitters-gameloop/ and https://gafferongames.com/post/fix_your_timestep/
4
I am speaking from experience here. We made a Tetris-like game in Unity, and it was not worth the trouble. You can see the result here, but it's nothing special: https://blubberquark.itch.io/tetrominions
5
Please don't write an engine from scratch! People like Jon Blow, Tommy Refenes, John Carmack, David Pittman or Notch wrote their own 3D engines for various reasons. That said, if you actually need a big 3D game engine for your game, please don't write your own on top of PyGame. You are not John Carmack (unless you are, in which case you should stop reading this because you know more about programming than I do). Use Panda3D or Godot if you want something that is open source and Python-based (Godot Python support is still in beta, but getting there). Or use one of the commercial engines listed above.
6
I'm simplifying massively here. Look up the Global Intepreter Lock if you want to know more. The GIL also allows PyGame to call SDL functions without worrying too much about whether SDL is thread-safe. It's not all bad.
7
Only with Python 3.5 it has become really usable, though, and they are still working out the kinks in the API.
8
I mean that as in lightweight threads, Erlang threads, green threads.
9
You can still find distutils in Python 3 for backward compatibility reasons.
10
http://pythonwheels.org
11
http://www.pyinstaller.org/
12
https://pynsist.readthedocs.io/en/latest/
13
https://www.mapeditor.org/
14
https://github.com/bitcraft/PyTMX
15
https://github.com/bitcraft/pyscroll
16
It manages to combine the good parts of Skype, Slack, and TeamSpeak, but it's not open source. https://discordapp.com/
17
https://github.com/qwertyquerty/pypresence
18
https://zulko.github.io/moviepy/
19
SDL2 already runs on the Nintendo Switch, for example. Unfortunately, the licensing terms of the Switch developer kit prohibit you from publicly sharing code that targets the Switch API in public. You have to contact Ryan Gordon and ask him for access, after signing the NDA. I am not a lawyer, this is not legal advice, but as far as I understand it, although the licensing terms of PyGame and SDL allow you to share the code, Nintendo could still sue you for breach of contract. That problem is not unique to PyGame though. A similar workaround exists in the Haxe world, where you have to contact Lars Doucet to get access to a version of OpenFL for Switch. There is also a homebrew version of Python available at https://github.com/nx-python/PyNX, but the NDA might prohibit you from using homebrew and reverse engineering.
Exploring Phantasms of the Living (1886) through Machine Learning: Presentiment, Crisis Apparitions and Thought Transference
NOTE: Click to open graphics for an expanded and clearer view of the findings they contain
Phantasms of the Living, published in 1886 by the Society for Psychical Research (SPR), was a landmark ESP study that presented the case for “telepathy" or thought transference from mind to mind. The study consisted of 702 cases spanning over 1400 pages that considered several varieties of telepathic experiences collectively referred to as “phantasms of the living”
The case collection examined non-sensory and internalized impressions, many of which were presentiment experiences involving dreams, clairvoyance, visions, feelings or an awareness in connection with the deaths of family members or friends. These experiences often coincided with the approximate time of death
Cases also considered sensory and externalized impressions, in particular apparitional representations of living persons, who were perceived to be in moments of crisis or danger. These situations appeared evidential of shock-induced forms of thought transference from a distressed agent to a percipient in the form of telepathic hallucinations
As a follow-on to the earlier wordcloud project, we wondered whether unsupervised machine learning could discover main topics within Phantasms of the Living. For the project, two varieties of generative topic models were used: Latent Dirichlet Allocation (LDA) and probabilistic Latent Semantic Analysis (pLSA)
Both models view documents as having a latent semantic structure of topics that can be inferred from co-occurrences of words in documents. The mathematics underlying both models are beyond the scope of this post, but on an intuitive level there are key differences between the two methods
pLSA views topics as probability distributions over words. Topics are seen as conditionally independent across the documents that produced them. Non-Negative Matrix Factorization (NMF) is a method for finding topic clusters that equates to pLSA
LDA by contrast views documents as probability distributions over topics and topics as probability distributions over words. All documents share the same collection of topics, but each document contains those topics in different proportions. The LDA algorithm samples words across topics until it arrives at topics and word selections that most likely generated the documents
The project used various packages and libraries for natural language processing within the Python programming platform to include: the Natural Language ToolKit (NLTK) for processing the data set; scikit-learn to prepare and fit the LDA and NMF models; pyLDAvis was used to display the results and t-Distributed Stochastic Neighbor Embedding (t-SNE) to map topic distances
The end-to-end project pipeline involved: data set processing; conversion of words and documents into matrix and vector space; fitting the LDA and NMF models; and then displaying the results
Processing. The book was decomposed into several documents from its constituent sections, chapters and volumes for the data set. Stopwords were removed such as common prepositions and conjunctions using the wordcloud application
Since telepathic experiences are spontaneous and can occur at any time or place, words conveying times and locations were removed as well as ordinal and cardinal types of numeric rankings
Nouns or titles representing persons were removed (e.g. man, woman, Mr., Mrs., etc.); however, interpersonal relationships were preserved (i.e. family, friends, acquaintances or strangers)
Conversion. Vector transformations converted the data set into a document-term matrix for mathematical processing
The rows of the matrix correspond to documents with columns corresponding to the frequency of a term. Count vectorizers count word frequencies. Term Frequency-Inverse Document Frequency (TF-IDF) vectorizers normalize (divide) word counts by their frequency in the documents
Both vectorizers converted words to lower case and removed non-word expressions. The vectorizers were also instructed to look for bigrams (or words that were often used together) such as "thought-transference" and "telepathic hallucination"
Model Fit/Display. The LDA and NMF models were fitted using ten topics. Words within topics were sorted and ranked with respect to their frequency in and relevance within a topic
The LDA model was fitted with using Count and TF-IDF vectorization and ran with a maximum of 10 iterations. LDA model results were displayed using pyLDAvis and t-SNE to map topic distances
The NMF model was fitted with TF-IDF vectorization only and ran with a maximum of 200 iterations. NMF model results were displayed via spreadsheet
Results. The topics produced from the models are unlabeled. However words within topics often can be woven into a coherent theme
The first two pyLDAvis graphs provide the top 30 words and bigrams in Topics 1 and 2 using Count vectorization
Words in Topic 1 include: “dreams”; “visions”; “impressions”; and “experiences” in connection with the “death”(s) of family members and friends. This can be considered a presentiment topic and it generated 67% of the content. This mirrors results from the prior wordcloud project
Words and bigrams in Topic 2 include: “thought-transference”, “hallucination(s)”, “phantasms”, “mind(s)”, “percipients”, “agent” and “telepathy.” This can be considered a telepathic hallucinations topic and it produced 27% of the content
The third pyLDAvis graph provides the top 30 words in Topic 1 using TF-IDF vectorization
Topic 1 combines all the aforementioned words into one topic. This can be considered a “presentiment and telepathic hallucinations ” topic and it accounts for 95% of the content, rendering all other topics practically insignificant in influence. The reason for this consolidation is that TF-IDF vectorization lowers the contribution weight of commonly used words
The spreadsheets compare LDA and NMF model runs using TF-IDF vectorizations with results limited to the top 10 words. Although topic weights and distances are not available, some topics appear more meaningful and cohesive, and are likely more impactful than others
There is overlap between topics 5 and 6 in the LDA model and together they form the presentiment and telepathic hallucinations topic. Topics 0 and 1 in the NMF model respectively appear to correspond to presentiment and crisis apparitions topics
The bigram “thought-transference” arises in both the LDA and NMF results and appears associated with the “Society” for “Psychical” Research and the late F.W.H. “Myers” who invented the term “telepathy”
This project had an extended preparation and production pipeline. The results indicate that unsupervised machine learning using LDA and NMF effectively and comprehensively summarized topical content in Phantasms of the Living. Moreover, key topics approximately corresponded to the types of internalized and externalized telepathic experiences described in the book
This project demonstrates the usefulness of topic generation models for finding meaningful patterns in masses of unlabeled or unstructured data. Moreover, visualization and graphing tools are essential for fully comprehending these patterns. Elsewhere in parapsychology LDA or NMF could also be applied to survey data, case collections, web or social media content of interest.
REFERENCES
Anaya, L. A. Comparing Latent Dirichlet Allocation and Latent Semantic Analysis as Classifiers. University of North Texas, 2011.
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022.
Christou, D. (2016). Feature extraction using Latent Dirichlet Allocation and Neural Networks: A case study on movie synopses. arXiv preprint arXiv:1604.01272.
Deerwester, S. (1988). Improving information retrieval with latent semantic indexing.
Gurney, E., Myers, F. W., & Podmore, F. (1886). Phantasms of the Living (2 vols.). London: Trübner. Reprinted at the Esalen Center.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Vanderplas, J. (2011). Scikit-learn: Machine learning in Python. Journal of machine learning research, 12(Oct), 2825-2830.
Sievert, C., & Shirley, K. (2014). LDAvis: A method for visualizing and interpreting topics. In Proceedings of the workshop on interactive language learning, visualization, and interfaces (pp. 63-70).
The Best Python Libraries For Machine Learning And AI.
Implementing ML and AI algorithms necessitates a well-structured and well-tested framework to enable developers to provide high-quality code solutions. A Python library or framework is a ready-to-use software for typical coding tasks.
Let's get to know the Best Python Libraries For Machine Learning:
TensorFlow is a complete Python machine learning toolkit for high-end numerical computations.
Keras Python is a popular open-source Python framework for building neural networks and machine learning projects.
Theano Python is a well-known scientific computing toolkit that allows you to design, optimize, and evaluate mathematical equations that deal with multidimensional arrays..
These libraries are incredibly useful while working on machine learning projects because they save time and give explicit functions on which to build.
