|
| https://realpython.com/ |
| Start Here | https://realpython.com/start-here/ |
|
Learn Python
| https://realpython.com/lessons/knn-python-summary/ |
| Python Tutorials →In-depth articles and video courses | https://realpython.com/search?kind=article&kind=course&order=newest |
| Learning Paths →Guided study plans for accelerated learning | https://realpython.com/learning-paths/ |
| Quizzes & Exercises →Check your learning progress | https://realpython.com/quizzes/ |
| Browse Topics →Focus on a specific area or skill level | https://realpython.com/tutorials/all/ |
| Community Chat →Learn with other Pythonistas | https://realpython.com/community/ |
| Office Hours →Live Q&A calls with Python experts | https://realpython.com/office-hours/ |
| Podcast →Hear what’s new in the world of Python | https://realpython.com/podcasts/rpp/ |
| Books →Round out your knowledge and learn offline | https://realpython.com/products/books/ |
| Reference →Concise definitions for common Python terms | https://realpython.com/ref/ |
| Code Mentor →BetaPersonalized code assistance & learning tools | https://realpython.com/mentor/ |
| Unlock All Content → | https://realpython.com/account/join/ |
|
More
| https://realpython.com/lessons/knn-python-summary/ |
| Learner Stories | https://realpython.com/learner-stories/ |
| Python Newsletter | https://realpython.com/newsletter/ |
| Python Job Board | https://www.pythonjobshq.com |
| Meet the Team | https://realpython.com/team/ |
| Become a Tutorial Writer | https://realpython.com/write-for-us/ |
| Become a Video Instructor | https://realpython.com/become-an-instructor/ |
| Search | https://realpython.com/search |
| https://realpython.com/search |
| Join | https://realpython.com/account/join/ |
| Sign‑In | https://realpython.com/account/login/?next=%2Flessons%2Fknn-python-summary%2F |
| Unlock This Lesson | https://realpython.com/account/join/?utm_source=rp_lesson&utm_content=knn-python |
| Unlock This Lesson | https://realpython.com/account/join/?utm_source=rp_lesson&utm_content=knn-python |
| https://realpython.com/courses/knn-python/#team |
| Using k-Nearest Neighbors (kNN) in Python | https://realpython.com/courses/knn-python/ |
| Kimberly Fessel | https://realpython.com/courses/knn-python/#team |
| Recommended Tutorial | https://realpython.com/knn-python/ |
| Course Slides (PDF) | https://realpython.com/courses/knn-python/downloads/knn-python-slides/ |
| Sample Code (IPYNB) | https://realpython.com/courses/knn-python/downloads/knn-python-code/ |
| Ask a Question | https://realpython.com/lessons/knn-python-summary/#discussion |
| https://realpython.com/feedback/survey/course/knn-python/liked/?from=lesson-title |
| https://realpython.com/feedback/survey/course/knn-python/disliked/?from=lesson-title |
| Contents | https://realpython.com/lessons/knn-python-summary/#description |
| Transcript | https://realpython.com/lessons/knn-python-summary/#transcript |
| Discussion (1) | https://realpython.com/lessons/knn-python-summary/#discussion |
| Machine Learning Learning Path | https://realpython.com/learning-paths/machine-learning-python/ |
| Unlock This Lesson | https://realpython.com/account/join/?utm_source=rp_lesson_preview&utm_content=knn-python |
| Sign-In | https://realpython.com/account/login/ |
| Unlock This Lesson | https://realpython.com/account/join/?utm_source=rp_lesson_preview&utm_content=knn-python |
| Sign-In | https://realpython.com/account/login/ |
| 00:00 | https://realpython.com/lessons/knn-python-summary/#t=0.57 |
| Let’s wrap up by summarizing what you’ve learned in this kNN course. | https://realpython.com/lessons/knn-python-summary/#t=0.57 |
| First, you got familiar with some of the properties of kNN. | https://realpython.com/lessons/knn-python-summary/#t=6.89 |
| k-nearest neighbors is a supervised machine learning algorithm. It’s nonlinear, | https://realpython.com/lessons/knn-python-summary/#t=11.04 |
| so it can pick up on complex patterns. kNN is also nonparametric. | https://realpython.com/lessons/knn-python-summary/#t=15.82 |
| 00:20 | https://realpython.com/lessons/knn-python-summary/#t=20.66 |
| It doesn’t assume any mathematical function between the input features and the | https://realpython.com/lessons/knn-python-summary/#t=20.66 |
| target output, so there are no parameters for it to learn. | https://realpython.com/lessons/knn-python-summary/#t=25.24 |
| You used kNN for a regression problem in this course, | https://realpython.com/lessons/knn-python-summary/#t=29.51 |
| but remember that it can also be used for classification problems to make | https://realpython.com/lessons/knn-python-summary/#t=32.91 |
| categorical predictions. | https://realpython.com/lessons/knn-python-summary/#t=36.69 |
| 00:40 | https://realpython.com/lessons/knn-python-summary/#t=40.38 |
| You also learned the main steps of the k-nearest neighbors algorithm. | https://realpython.com/lessons/knn-python-summary/#t=40.38 |
| Given a trained kNN model, | https://realpython.com/lessons/knn-python-summary/#t=44.89 |
| which is really just a collection of memorized training data points, | https://realpython.com/lessons/knn-python-summary/#t=46.81 |
| a prediction can be made for a new data point by finding that point’s nearest | https://realpython.com/lessons/knn-python-summary/#t=50.76 |
| neighbors and predicting based on those neighbors’ targets. | https://realpython.com/lessons/knn-python-summary/#t=54.92 |
| 00:58 | https://realpython.com/lessons/knn-python-summary/#t=58.5 |
| For regression problems, you just take the average of the neighbors’ targets, | https://realpython.com/lessons/knn-python-summary/#t=58.5 |
| and for classification, you use majority vote. | https://realpython.com/lessons/knn-python-summary/#t=62.86 |
| 01:08 | https://realpython.com/lessons/knn-python-summary/#t=68.24 |
| And you were able to code up kNN in Python in two ways. First | https://realpython.com/lessons/knn-python-summary/#t=68.24 |
| from scratch, where after a bit of data manipulation, | https://realpython.com/lessons/knn-python-summary/#t=72.89 |
| you calculated the distances between the features of a new data point and every | https://realpython.com/lessons/knn-python-summary/#t=76.43 |
| other observation. | https://realpython.com/lessons/knn-python-summary/#t=80.81 |
| 01:22 | https://realpython.com/lessons/knn-python-summary/#t=82.36 |
| Then you found the ID numbers of the points’ nearest neighbors with .argsort(), | https://realpython.com/lessons/knn-python-summary/#t=82.36 |
| and finally, | https://realpython.com/lessons/knn-python-summary/#t=87.35 |
| you made a prediction by collecting and averaging the neighbors target values. | https://realpython.com/lessons/knn-python-summary/#t=88.07 |
| 01:36 | https://realpython.com/lessons/knn-python-summary/#t=96.13 |
| Secondly, you used scikit-learn to build a kNN model. | https://realpython.com/lessons/knn-python-summary/#t=96.13 |
| You split your data into training and test sets using train_test_split(). | https://realpython.com/lessons/knn-python-summary/#t=99.55 |
| Next, | https://realpython.com/lessons/knn-python-summary/#t=104.44 |
| you instantiated a k-nearest neighbor model for your regression problem and then | https://realpython.com/lessons/knn-python-summary/#t=104.79 |
| fit that model to your training data. After that, | https://realpython.com/lessons/knn-python-summary/#t=109.35 |
| you made predictions for your test set using the .predict() method. | https://realpython.com/lessons/knn-python-summary/#t=112.39 |
| 01:58 | https://realpython.com/lessons/knn-python-summary/#t=118.2 |
| Now that you’ve learned all about kNN, | https://realpython.com/lessons/knn-python-summary/#t=118.2 |
| what would you say are its biggest benefits? Well, | https://realpython.com/lessons/knn-python-summary/#t=120.05 |
| it’s certainly an intuitive algorithm. | https://realpython.com/lessons/knn-python-summary/#t=123.41 |
| You can easily implement it yourself from scratch, | https://realpython.com/lessons/knn-python-summary/#t=125.6 |
| and you’ll be able to comfortably explain its main steps. | https://realpython.com/lessons/knn-python-summary/#t=128.44 |
| 02:12 | https://realpython.com/lessons/knn-python-summary/#t=132.69 |
| kNN is a highly flexible model that can pick up on complicated patterns due to | https://realpython.com/lessons/knn-python-summary/#t=132.69 |
| its nonlinear and nonparametric nature. | https://realpython.com/lessons/knn-python-summary/#t=137.29 |
| If you think about how the algorithm works, | https://realpython.com/lessons/knn-python-summary/#t=140.65 |
| kNN can actually adapt its predictions as new training observations are | https://realpython.com/lessons/knn-python-summary/#t=143.15 |
| collected. | https://realpython.com/lessons/knn-python-summary/#t=147.31 |
| 02:28 | https://realpython.com/lessons/knn-python-summary/#t=148.06 |
| This could be quite useful if you’re working on a problem that has a continuous | https://realpython.com/lessons/knn-python-summary/#t=148.06 |
| flow of new data. And finally, | https://realpython.com/lessons/knn-python-summary/#t=151.7 |
| kNN essentially has no training time. | https://realpython.com/lessons/knn-python-summary/#t=154.84 |
| Many machine learning algorithms need to perform numerous calculations before | https://realpython.com/lessons/knn-python-summary/#t=157.67 |
| they’re considered trained. The kNN only needs to store the training dataset, | https://realpython.com/lessons/knn-python-summary/#t=162.24 |
| which is virtually instantaneous. | https://realpython.com/lessons/knn-python-summary/#t=167.01 |
| 02:51 | https://realpython.com/lessons/knn-python-summary/#t=171.64 |
| Bbut on the downside, this virtue can also be considered a drawback. | https://realpython.com/lessons/knn-python-summary/#t=171.64 |
| k-nearest neighbors is a so-called lazy learner. | https://realpython.com/lessons/knn-python-summary/#t=176.14 |
| That means it does nearly all its work only when a prediction needs to be made. | https://realpython.com/lessons/knn-python-summary/#t=179.39 |
| 03:04 | https://realpython.com/lessons/knn-python-summary/#t=184.58 |
| Its training time is fast, but its prediction time can be quite slow. | https://realpython.com/lessons/knn-python-summary/#t=184.58 |
| In fact, the prediction time scales linearly with the size of its training set. | https://realpython.com/lessons/knn-python-summary/#t=189.54 |
| That can be a big drawback. | https://realpython.com/lessons/knn-python-summary/#t=193.91 |
| 03:15 | https://realpython.com/lessons/knn-python-summary/#t=195.73 |
| Machine learning models typically improve as their training data increases, | https://realpython.com/lessons/knn-python-summary/#t=195.73 |
| but with kNN, bigger training sets mean slower predictions. | https://realpython.com/lessons/knn-python-summary/#t=200.48 |
| k-nearest neighbors also requires a lot of memory since it needs to keep the full | https://realpython.com/lessons/knn-python-summary/#t=205.83 |
| training set to make a prediction. | https://realpython.com/lessons/knn-python-summary/#t=210.83 |
| 03:32 | https://realpython.com/lessons/knn-python-summary/#t=212.82 |
| This is another big drawback if you want to run your model on a small device, | https://realpython.com/lessons/knn-python-summary/#t=212.82 |
| like a phone, say. Finally, the physical measurements for the abalone problem. | https://realpython.com/lessons/knn-python-summary/#t=216.82 |
| 03:41 | https://realpython.com/lessons/knn-python-summary/#t=221.77 |
| In this course, were all roughly on the same scale, | https://realpython.com/lessons/knn-python-summary/#t=221.77 |
| but if you have some very small inputs and others on the scale of millions, | https://realpython.com/lessons/knn-python-summary/#t=224.93 |
| your features will need to be scaled for kNN to judge the distances between | https://realpython.com/lessons/knn-python-summary/#t=229.74 |
| points fairly. Otherwise, | https://realpython.com/lessons/knn-python-summary/#t=233.91 |
| the large scaled inputs will dominate the distance calculations. | https://realpython.com/lessons/knn-python-summary/#t=236.61 |
| 04:03 | https://realpython.com/lessons/knn-python-summary/#t=243.63 |
| If you want to learn more about other algorithms and data science techniques, | https://realpython.com/lessons/knn-python-summary/#t=243.63 |
| check out the Real Python Machine Learning With Python learning path. | https://realpython.com/lessons/knn-python-summary/#t=247.3 |
| It includes plenty of great tips for getting started, | https://realpython.com/lessons/knn-python-summary/#t=251.99 |
| including a course on splitting datasets with scikit-learn and train_test_split(), | https://realpython.com/lessons/knn-python-summary/#t=254.88 |
| as well as starting with linear aggression in Python. | https://realpython.com/lessons/knn-python-summary/#t=259.54 |
| 04:26 | https://realpython.com/lessons/knn-python-summary/#t=266.47 |
| This concludes Using k-Nearest Neighbors in Python. | https://realpython.com/lessons/knn-python-summary/#t=266.47 |
| Thanks for joining me, and enjoy building your own kNN models. | https://realpython.com/lessons/knn-python-summary/#t=269.46 |
| May 24, 2023 | https://realpython.com/lessons/knn-python-summary/#comment-10332f5f-efb3-489f-9ce4-0494f58c86d5 |
| Become a Member | https://realpython.com/account/join/ |
| https://realpython.com/lessons/knn-scikit-learn-score-and-update-k/ |
| Overview | https://realpython.com/courses/knn-python/ |
|
Using k-Nearest Neighbors (kNN) in Python (Overview) 01:41
| https://realpython.com/videos/knn-python-overview/ |
|
Distinguishing Features of kNN 05:50
| https://realpython.com/videos/knn-python-features/ |
|
Predicting the Age of Sea Snails 07:40
| https://realpython.com/lessons/knn-python-predict/ |
|
Understanding How kNN Works 08:59
| https://realpython.com/lessons/how-knn-works/ |
|
Coding kNN From Scratch: Data and Distances 05:36
| https://realpython.com/lessons/knn-data-and-distances/ |
|
Coding kNN From Scratch: Neighbors and Predictions 06:10
| https://realpython.com/lessons/knn-neighbors-and-predictions/ |
|
Using kNN in scikit-learn: Data, Fit, and Predict 09:16
| https://realpython.com/lessons/knn-scikit-learn-data-fit-predict/ |
|
Using kNN in scikit-learn: Score and Update k 07:15
| https://realpython.com/lessons/knn-scikit-learn-score-and-update-k/ |
|
Using k-Nearest Neighbors (kNN) in Python (Summary) 04:35
| https://realpython.com/lessons/knn-python-summary/ |
| Privacy Policy | https://realpython.com/privacy-policy/ |
Viewport: width=device-width, initial-scale=1, shrink-to-fit=no, viewport-fit=cover