It was developed by theGoogle Brain team forGoogle's internal use in research and production.[7][8][9] The initial version was released under theApache License 2.0 in 2015.[1][10] Google released an updated version, TensorFlow 2.0, in September 2019.[11]
TensorFlow can be used in a wide variety of programming languages, includingPython,JavaScript,C++, andJava,[12] facilitating its use in a range of applications in many sectors.
Starting in 2011, Google Brain built DistBelief as aproprietarymachine learning system based ondeep learningneural networks. Its use grew rapidly across diverseAlphabet companies in both research and commercial applications.[13][14] Google assigned multiple computer scientists, includingJeff Dean, to simplify andrefactor the codebase of DistBelief into a faster, more robust application-grade library, which became TensorFlow.[15] In 2009, the team, led byGeoffrey Hinton, had implemented generalizedbackpropagation and other improvements, which allowed generation ofneural networks with substantially higher accuracy, for instance a 25% reduction in errors inspeech recognition.[16]
Its flexible architecture allows for easy deployment of computation across a variety of platforms (CPUs, GPUs,TPUs), and from desktops to clusters of servers to mobile andedge devices.
TensorFlow computations are expressed asstatefuldataflowgraphs. The name TensorFlow derives from the operations that such neural networks perform on multidimensional data arrays, which are referred to astensors.[19] During theGoogle I/O Conference in June 2016, Jeff Dean stated that 1,500 repositories onGitHub mentioned TensorFlow, of which only 5 were from Google.[20]
In March 2018, Google announced TensorFlow.js version 1.0 for machine learning inJavaScript.[21]
In Jan 2019, Google announced TensorFlow 2.0.[22] It became officially available in September 2019.[11]
In May 2019, Google announced TensorFlow Graphics for deep learning in computer graphics.[23]
In May 2017, Google announced the second-generation, as well as the availability of the TPUs inGoogle Compute Engine.[25] The second-generation TPUs deliver up to 180teraflops of performance, and when organized into clusters of 64 TPUs, provide up to 11.5petaflops.[citation needed]
In May 2018, Google announced the third-generation TPUs delivering up to 420teraflops of performance and 128 GB highbandwidth memory (HBM). Cloud TPU v3 Pods offer 100+petaflops of performance and 32 TB HBM.[26]
In February 2018, Google announced that they were making TPUs available in beta on theGoogle Cloud Platform.[27]
In July 2018, the Edge TPU was announced. Edge TPU is Google's purpose-builtASIC chip designed to run TensorFlow Lite machine learning (ML) models on small client computing devices such as smartphones[28] known asedge computing.
In May 2017, Google announced a software stack specifically for mobile development, TensorFlow Lite.[29] In January 2019, the TensorFlow team released a developer preview of the mobile GPU inference engine with OpenGL ES 3.1 Compute Shaders on Android devices and Metal Compute Shaders on iOS devices.[30] In May 2019, Google announced that their TensorFlow Lite Micro (also known as TensorFlow Lite for Microcontrollers) andARM's uTensor would be merging.[31]
As TensorFlow's market share among research papers was declining to the advantage ofPyTorch,[32] the TensorFlow Team announced a release of a new major version of the library in September 2019. TensorFlow 2.0 introduced many changes, the most significant being TensorFlow eager, which changed the automatic differentiation scheme from the static computational graph to the "Define-by-Run" scheme originally made popular byChainer and laterPyTorch.[32] Other major changes included removal of old libraries, cross-compatibility between trained models on different versions of TensorFlow, and significant improvements to the performance on GPU.[33]
AutoDifferentiation is the process of automatically calculating the gradient vector of a model with respect to each of its parameters. With this feature, TensorFlow can automatically compute the gradients for the parameters in a model, which is useful to algorithms such asbackpropagation which require gradients to optimize performance.[34] To do so, the framework must keep track of the order of operations done to the input Tensors in a model, and then compute the gradients with respect to the appropriate parameters.[34]
TensorFlow includes an “eager execution” mode, which means that operations are evaluated immediately as opposed to being added to a computational graph which is executed later.[35] Code executed eagerly can be examined step-by step-through a debugger, since data is augmented at each line of code rather than later in a computational graph.[35] This execution paradigm is considered to be easier to debug because of its step by step transparency.[35]
In both eager and graph executions, TensorFlow provides an API for distributing computation across multiple devices with various distribution strategies.[36] Thisdistributed computing can often speed up the execution of training and evaluating of TensorFlow models and is a common practice in the field of AI.[36][37]
In order to assess the performance of machine learning models, TensorFlow gives API access to commonly used metrics. Examples include various accuracy metrics (binary, categorical, sparse categorical) along with other metrics such asPrecision, Recall, andIntersection-over-Union (IoU).[39]
TensorFlow offers a set of optimizers for training neural networks, includingADAM,ADAGRAD, andStochastic Gradient Descent (SGD).[41] When training a model, different optimizers offer different modes of parameter tuning, often affecting a model's convergence and performance.[42]
TensorFlow serves as a core platform and library for machine learning. TensorFlow's APIs useKeras to allow users to make their own machine-learning models.[33][43] In addition to building and training their model, TensorFlow can also help load the data to train the model, and deploy it using TensorFlow Serving.[44]
TensorFlow also has a library for machine learning in JavaScript. Using the providedJavaScript APIs, TensorFlow.js allows users to use either Tensorflow.js models or converted models from TensorFlow or TFLite, retrain the given models, and run on the web.[44][62]
TensorFlow Lite has APIs for mobile apps or embedded devices to generate and deploy TensorFlow models.[63] These models are compressed and optimized in order to be more efficient and have a higher performance on smaller capacity devices.[64]
TensorFlow Lite usesFlatBuffers as the data serialization format for network models, eschewing theProtocol Buffers format used by standard TensorFlow models.[64]
TensorFlow Extended (abbrev. TFX) provides numerous components to perform all the operations needed for end-to-end production.[65] Components include loading, validating, and transforming data, tuning, training, and evaluating the machine learning model, and pushing the model itself into production.[44][65]
Numpy is one of the most popularPython data libraries, and TensorFlow offers integration and compatibility with its data structures.[66] Numpy NDarrays, the library's native datatype, are automatically converted to TensorFlow Tensors in TF operations; the same is also true vice versa.[66] This allows for the two libraries to work in unison without requiring the user to write explicit data conversions. Moreover, the integration extends to memory optimization by having TF Tensors share the underlying memory representations of Numpy NDarrays whenever possible.[66]
TensorFlow also offers a variety oflibraries andextensions to advance and extend the models and methods used.[67] For example, TensorFlow Recommenders and TensorFlow Graphics arelibraries for their respective functional.[68] Other add-ons,libraries, andframeworks include TensorFlow Model Optimization, TensorFlow Probability, TensorFlow Quantum, and TensorFlow Decision Forests.[67][68]
Google also released Colaboratory, a TensorFlow Jupyter notebook environment that does not require any setup.[69] It runs on Google Cloud and allows users free access to GPUs and the ability to store and share notebooks onGoogle Drive.[70]
Google JAX is a machine learningframework for transforming numerical functions.[71][72][73] It is described as bringing together a modified version ofautograd (automatic obtaining of the gradient function through differentiation of a function) and TensorFlow'sXLA (Accelerated Linear Algebra). It is designed to follow the structure and workflow ofNumPy as closely as possible and works with TensorFlow as well as other frameworks such asPyTorch. The primary functions of JAX are:[71]
GE Healthcare used TensorFlow to increase the speed and accuracy ofMRIs in identifying specific body parts.[74] Google used TensorFlow to create DermAssist, a free mobile application that allows users to take pictures of their skin and identify potential health complications.[75]Sinovation Ventures used TensorFlow to identify and classify eye diseases fromoptical coherence tomography (OCT) scans.[75]
Twitter implemented TensorFlow to rank tweets by importance for a given user, and changed their platform to show tweets in order of this ranking.[76] Previously, tweets were simply shown in reverse chronological order.[76] The photo sharing appVSCO used TensorFlow to help suggest custom filters for photos.[75]
InSpace, a virtual learning platform, used TensorFlow to filter out toxic chat messages in classrooms.[78] Liulishuo, an online English learning platform, utilized TensorFlow to create an adaptive curriculum for each student.[79] TensorFlow was used to accurately assess a student's current abilities, and also helped decide the best future content to show based on those capabilities.[79]
The e-commerce platformCarousell used TensorFlow to provide personalized recommendations for customers.[75] The cosmetics company ModiFace used TensorFlow to create an augmented reality experience for customers to test various shades of make-up on their face.[80]
2016 comparison of original photo (left) and with TensorFlowneural style applied (right)
^Abadi, Martín; Barham, Paul; Chen, Jianmin; Chen, Zhifeng; Davis, Andy; Dean, Jeffrey; Devin, Matthieu; Ghemawat, Sanjay; Irving, Geoffrey; Isard, Michael; Kudlur, Manjunath; Levenberg, Josh; Monga, Rajat; Moore, Sherry; Murray, Derek G.; Steiner, Benoit; Tucker, Paul; Vasudevan, Vijay; Warden, Pete; Wicke, Martin; Yu, Yuan; Zheng, Xiaoqiang (2016).TensorFlow: A System for Large-Scale Machine Learning(PDF). Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16).arXiv:1605.08695.Archived(PDF) from the original on December 12, 2020. RetrievedOctober 26, 2020.
^TensorFlow: Open source machine learning. Google. 2015.Archived from the original on November 11, 2021. "It is machine learning software being used for various kinds of perceptual and language understanding tasks" – Jeffrey Dean, minute 0:47 / 2:17 from YouTube clip
^abCiaramella, Alberto; Ciaramella, Marco (July 2024).Introduction to Artificial Intelligence: from data analysis to generative AI. Intellisemantic Editions.ISBN9788894787603.
^Malmaud, Jon (August 12, 2019)."A Julia wrapper for TensorFlow".GitHub.Archived from the original on July 24, 2017. RetrievedAugust 14, 2019.operations like sin, * (matrix multiplication), .* (element-wise multiplication), etc [..]. Compare to Python, which requires learning specialized namespaced functions like tf.matmul.
