Given the challenges of maintaining installations of various frameworks in a single environment,users who would want to testivy with multiple frameworks at once can use our Docker images for a seamless experience.You can pull the images from:

docker pull unifyai/ivy:latest# CPUdocker pull unifyai/ivy:latest-gpu# GPU

You can also install Ivy from source if you want to take advantage ofthe latest changes, but we can't ensure everything will work asexpected 😅

git clone https://github.com/unifyai/ivy.gitcd ivypip install --user -e.

If you want to set up testing and various frameworks it's probably bestto check out theSetting Uppage, where OS-specific and IDE-specific instructions and videotutorials to do so are available!

I'm using PyTorch 

You can use Ivy to get PyTorch code from:

Any model

From TensorFlow

importivyimporttorchimporttensorflowastf# Get a pretrained keras modeleff_encoder=tf.keras.applications.efficientnet_v2.EfficientNetV2B0(include_top=False,weights="imagenet",input_shape=(224,224,3))# Transpile it into a torch.nn.Module with the corresponding parametersnoise=tf.random.normal(shape=(1,224,224,3))torch_eff_encoder=ivy.transpile(eff_encoder,source="tensorflow",to="torch",args=(noise,))# Build a classifier using the transpiled encoderclassClassifier(torch.nn.Module):def__init__(self,num_classes=20):super().__init__()self.encoder=torch_eff_encoderself.fc=torch.nn.Linear(1280,num_classes)defforward(self,x):x=self.encoder(x)returnself.fc(x)# Initialize a trainable, customizable, torch.nn.Moduleclassifier=Classifier()ret=classifier(torch.rand((1,244,244,3)))

From JAX

importivyimportjaximporttorch# Get a pretrained haiku model# https://github.com/unifyai/demos/blob/15c235f/scripts/deepmind_perceiver_io.pyfromdeepmind_perceiver_ioimportkey,perceiver_backbone# Transpile it into a torch.nn.Module with the corresponding parametersdummy_input=jax.random.uniform(key,shape=(1,3,224,224))params=perceiver_backbone.init(rng=key,images=dummy_input)ivy.set_backend("jax")backbone=ivy.transpile(perceiver_backbone,source="jax",to="torch",params_v=params,kwargs={"images":dummy_input})# Build a classifier using the transpiled backboneclassPerceiverIOClassifier(torch.nn.Module):def__init__(self,num_classes=20):super().__init__()self.backbone=backboneself.max_pool=torch.nn.MaxPool2d((512,1))self.flatten=torch.nn.Flatten()self.fc=torch.nn.Linear(1024,num_classes)defforward(self,x):x=self.backbone(images=x)x=self.flatten(self.max_pool(x))returnself.fc(x)# Initialize a trainable, customizable, torch.nn.Moduleclassifier=PerceiverIOClassifier()ret=classifier(torch.rand((1,3,224,224)))

Any library

From Tensorflow

importivyimporttorchimportosos.environ["SM_FRAMEWORK"]="tf.keras"importsegmentation_modelsassm# transpile sm from tensorflow to torchtorch_sm=ivy.transpile(sm,source="tensorflow",to="torch")# get some image-like arraysoutput=torch.rand((1,3,512,512))target=torch.rand((1,3,512,512))# and use the transpiled version of any function from the library!out=torch_sm.metrics.iou_score(output,target)

From JAX

importivyimportraximporttorch# transpile rax from jax to torchtorch_rax=ivy.transpile(rax,source="jax",to="torch")# get some arraysscores=torch.tensor([2.2,1.3,5.4])labels=torch.tensor([1.0,0.0,0.0])# and use the transpiled version of any function from the library!out=torch_rax.poly1_softmax_loss(scores,labels)

From NumPy

importivyimporttorchimportmadmom# transpile madmon from numpy to torchtorch_madmom=ivy.transpile(madmom,source="numpy",to="torch")# get some arraysfreqs=torch.arange(20)*10# and use the transpiled version of any function from the library!out=torch_madmom.audio.filters.hz2midi(freqs)

Any function

From Tensorflow

importivyimporttensorflowastfimporttorchdefloss(predictions,targets):returntf.sqrt(tf.reduce_mean(tf.square(predictions-targets)))# transpile any function from tf to torchtorch_loss=ivy.transpile(loss,source="tensorflow",to="torch")# get some arraysp=torch.tensor([3.0,2.0,1.0])t=torch.tensor([0.0,0.0,0.0])# and use the transpiled version!out=torch_loss(p,t)

From JAX

importivyimportjax.numpyasjnpimporttorchdefloss(predictions,targets):returnjnp.sqrt(jnp.mean((predictions-targets)**2))# transpile any function from jax to torchtorch_loss=ivy.transpile(loss,source="jax",to="torch")# get some arraysp=torch.tensor([3.0,2.0,1.0])t=torch.tensor([0.0,0.0,0.0])# and use the transpiled version!out=torch_loss(p,t)

From NumPy

importivyimportnumpyasnpimporttorchdefloss(predictions,targets):returnnp.sqrt(np.mean((predictions-targets)**2))# transpile any function from numpy to torchtorch_loss=ivy.transpile(loss,source="numpy",to="torch")# get some arraysp=torch.tensor([3.0,2.0,1.0])t=torch.tensor([0.0,0.0,0.0])# and use the transpiled version!out=torch_loss(p,t)

I'm using TensorFlow 

You can use Ivy to get TensorFlow code from:

Any model

From PyTorch

importivyimporttorchimporttimmimporttensorflowastf# Get a pretrained pytorch modelmlp_encoder=timm.create_model("mixer_b16_224",pretrained=True,num_classes=0)# Transpile it into a keras.Model with the corresponding parametersnoise=torch.randn(1,3,224,224)mlp_encoder=ivy.transpile(mlp_encoder,to="tensorflow",args=(noise,))# Build a classifier using the transpiled encoderclassClassifier(tf.keras.Model):def__init__(self):super().__init__()self.encoder=mlp_encoderself.output_dense=tf.keras.layers.Dense(units=1000,activation="softmax")defcall(self,x):x=self.encoder(x)returnself.output_dense(x)# Transform the classifier and use it as a standard keras.Modelx=tf.random.normal(shape=(1,3,224,224))model=Classifier()ret=model(x)

From JAX

importivyimportjaximporttensorflowastf# Get a pretrained haiku model# https://unify.ai/demos/scripts/deepmind_perceiver_io.pyfromdeepmind_perceiver_ioimportkey,perceiver_backbone# Transpile it into a tf.keras.Model with the corresponding parametersdummy_input=jax.random.uniform(key,shape=(1,3,224,224))params=perceiver_backbone.init(rng=key,images=dummy_input)backbone=ivy.transpile(perceiver_backbone,to="tensorflow",params_v=params,args=(dummy_input,))# Build a classifier using the transpiled backboneclassPerceiverIOClassifier(tf.keras.Model):def__init__(self,num_classes=20):super().__init__()self.backbone=backboneself.max_pool=tf.keras.layers.MaxPooling1D(pool_size=512)self.flatten=tf.keras.layers.Flatten()self.fc=tf.keras.layers.Dense(num_classes)defcall(self,x):x=self.backbone(x)x=self.flatten(self.max_pool(x))returnself.fc(x)# Initialize a trainable, customizable, tf.keras.Modelx=tf.random.normal(shape=(1,3,224,224))classifier=PerceiverIOClassifier()ret=classifier(x)

Any library

From PyTorch

importivyimportkorniaimportrequestsimportnumpyasnpimporttensorflowastffromPILimportImage# transpile kornia from torch to tensorflowtf_kornia=ivy.transpile(kornia,source="torch",to="tensorflow")# get an imageurl="http://images.cocodataset.org/train2017/000000000034.jpg"raw_img=Image.open(requests.get(url,stream=True).raw)# convert it to the format expected by korniaimg=np.array(raw_img)img=tf.transpose(tf.constant(img), (2,0,1))img=tf.expand_dims(img,0)/255# and use the transpiled version of any function from the library!out=tf_kornia.enhance.sharpness(img,5)

From JAX

importivyimportraximporttensorflowastf# transpile rax from jax to tensorflowtf_rax=ivy.transpile(rax,source="jax",to="tensorflow")# get some arraysscores=tf.constant([2.2,1.3,5.4])labels=tf.constant([1.0,0.0,0.0])# and use the transpiled version of any function from the library!out=tf_rax.poly1_softmax_loss(scores,labels)

From NumPy

importivyimportmadmomimporttensorflowastf# transpile madmom from numpy to tensorflowtf_madmom=ivy.transpile(madmom,source="numpy",to="tensorflow")# get some arraysfreqs=tf.range(20)*10# and use the transpiled version of any function from the library!out=tf_madmom.audio.filters.hz2midi(freqs)

Any function

From PyTorch

importivyimporttorchimporttensorflowastfdefloss(predictions,targets):returntorch.sqrt(torch.mean((predictions-targets)**2))# transpile any function from torch to tensorflowtf_loss=ivy.transpile(loss,source="torch",to="tensorflow")# get some arraysp=tf.constant([3.0,2.0,1.0])t=tf.constant([0.0,0.0,0.0])# and use the transpiled version!out=tf_loss(p,t)

From JAX

importivyimportjax.numpyasjnpimporttensorflowastfdefloss(predictions,targets):returnjnp.sqrt(jnp.mean((predictions-targets)**2))# transpile any function from jax to tensorflowtf_loss=ivy.transpile(loss,source="jax",to="tensorflow")# get some arraysp=tf.constant([3.0,2.0,1.0])t=tf.constant([0.0,0.0,0.0])# and use the transpiled version!out=tf_loss(p,t)

From NumPy

importivyimportnumpyasnpimporttensorflowastfdefloss(predictions,targets):returnnp.sqrt(np.mean((predictions-targets)**2))# transpile any function from numpy to tensorflowtf_loss=ivy.transpile(loss,source="numpy",to="tensorflow")# get some arraysp=tf.constant([3.0,2.0,1.0])t=tf.constant([0.0,0.0,0.0])# and use the transpiled version!out=tf_loss(p,t)

I'm using Jax 

You can use Ivy to get JAX code from:

Any model

From PyTorch

importivyimporttimmimporttorchimportjaximporthaikuashk# Get a pretrained pytorch modelmlp_encoder=timm.create_model("mixer_b16_224",pretrained=True,num_classes=0)# Transpile it into a hk.Module with the corresponding parametersnoise=torch.randn(1,3,224,224)mlp_encoder=ivy.transpile(mlp_encoder,source="torch",to="haiku",args=(noise,))# Build a classifier using the transpiled encoderclassClassifier(hk.Module):def__init__(self,num_classes=1000):super().__init__()self.encoder=mlp_encoder()self.fc=hk.Linear(output_size=num_classes,with_bias=True)def__call__(self,x):x=self.encoder(x)x=self.fc(x)returnxdef_forward_classifier(x):module=Classifier()returnmodule(x)# Transform the classifier and use it as a standard hk.Modulerng_key=jax.random.PRNGKey(42)x=jax.random.uniform(key=rng_key,shape=(1,3,224,224),dtype=jax.numpy.float32)forward_classifier=hk.transform(_forward_classifier)params=forward_classifier.init(rng=rng_key,x=x)ret=forward_classifier.apply(params,None,x)

From TensorFlow

importivyimportjaximporthaikuashkimporttensorflowastfjax.config.update("jax_enable_x64",True)# Get a pretrained keras modeleff_encoder=tf.keras.applications.efficientnet_v2.EfficientNetV2B0(include_top=False,weights="imagenet",input_shape=(224,224,3))# Transpile it into a hk.Module with the corresponding parametersnoise=tf.random.normal(shape=(1,224,224,3))hk_eff_encoder=ivy.transpile(eff_encoder,source="tensorflow",to="haiku",args=(noise,))# Build a classifier using the transpiled encoderclassClassifier(hk.Module):def__init__(self,num_classes=1000):super().__init__()self.encoder=hk_eff_encoder()self.fc=hk.Linear(output_size=num_classes,with_bias=True)def__call__(self,x):x=self.encoder(x)x=self.fc(x)returnxdef_forward_classifier(x):module=Classifier()returnmodule(x)# Transform the classifier and use it as a standard hk.Modulerng_key=jax.random.PRNGKey(42)dummy_x=jax.random.uniform(key=rng_key,shape=(1,224,224,3))forward_classifier=hk.transform(_forward_classifier)params=forward_classifier.init(rng=rng_key,x=dummy_x)ret=forward_classifier.apply(params,None,dummy_x)

Any library

From PyTorch

importivyimportkorniaimportrequestsimportjax.numpyasjnpfromPILimportImagejax.config.update("jax_enable_x64",True)# transpile kornia from torch to jaxjax_kornia=ivy.transpile(kornia,source="torch",to="jax")# get an imageurl="http://images.cocodataset.org/train2017/000000000034.jpg"raw_img=Image.open(requests.get(url,stream=True).raw)# convert it to the format expected by korniaimg=jnp.transpose(jnp.array(raw_img), (2,0,1))img=jnp.expand_dims(img,0)/255# and use the transpiled version of any function from the library!out=jax_kornia.enhance.sharpness(img,5)

From TensorFlow

importivyimportjaximportosos.environ["SM_FRAMEWORK"]="tf.keras"importsegmentation_modelsassm# transpile sm from tensorflow to jaxjax_sm=ivy.transpile(sm,source="tensorflow",to="jax")# get some image-like arrayskey=jax.random.PRNGKey(23)key1,key2=jax.random.split(key)output=jax.random.uniform(key1, (1,3,512,512))target=jax.random.uniform(key2, (1,3,512,512))# and use the transpiled version of any function from the library!out=jax_sm.metrics.iou_score(output,target)

From NumPy

importivyimportmadmomimportjax.numpyasjnp# transpile madmon from numpy to jaxjax_madmom=ivy.transpile(madmom,source="numpy",to="jax")# get some arraysfreqs=jnp.arange(20)*10# and use the transpiled version of any function from the library!out=jax_madmom.audio.filters.hz2midi(freqs)

Any function

From PyTorch

importivyimporttorchimportjax.numpyasjnpdefloss(predictions,targets):returntorch.sqrt(torch.mean((predictions-targets)**2))# transpile any function from torch to jaxjax_loss=ivy.transpile(loss,source="torch",to="jax")# get some arraysp=jnp.array([3.0,2.0,1.0])t=jnp.array([0.0,0.0,0.0])# and use the transpiled version!out=jax_loss(p,t)

From TensorFlow

importivyimporttensorflowastfimportjax.numpyasjnpdefloss(predictions,targets):returntf.sqrt(tf.reduce_mean(tf.square(predictions-targets)))# transpile any function from tf to jaxjax_loss=ivy.transpile(loss,source="tensorflow",to="jax")# get some arraysp=jnp.array([3.0,2.0,1.0])t=jnp.array([0.0,0.0,0.0])# and use the transpiled version!out=jax_loss(p,t)

From NumPy

importivyimportnumpyasnpimportjaximportjax.numpyasjnpjax.config.update('jax_enable_x64',True)defloss(predictions,targets):returnnp.sqrt(np.mean((predictions-targets)**2))# transpile any function from numpy to jaxjax_loss=ivy.transpile(loss,source="numpy",to="jax")# get some arraysp=jnp.array([3.0,2.0,1.0])t=jnp.array([0.0,0.0,0.0])# and use the transpiled version!out=jax_loss(p,t)

I'm using NumPy 

You can use Ivy to get NumPy code from:

Any library

From PyTorch

importivyimportkorniaimportrequestsimportnumpyasnpfromPILimportImage# transpile kornia from torch to npnp_kornia=ivy.transpile(kornia,source="torch",to="numpy")# get an imageurl="http://images.cocodataset.org/train2017/000000000034.jpg"raw_img=Image.open(requests.get(url,stream=True).raw)# convert it to the format expected by korniaimg=np.transpose(np.array(raw_img), (2,0,1))img=np.expand_dims(img,0)/255# and use the transpiled version of any function from the library!out=np_kornia.enhance.sharpness(img,5)

From TensorFlow

importivyimportnumpyasnpimportosos.environ["SM_FRAMEWORK"]="tf.keras"importsegmentation_modelsassm# transpile sm from tensorflow to numpynp_sm=ivy.transpile(sm,source="tensorflow",to="numpy")# get some image-like arraysoutput=np.random.rand(1,3,512,512).astype(dtype=np.float32)target=np.random.rand(1,3,512,512).astype(dtype=np.float32)# and use the transpiled version of any function from the library!out=np_sm.metrics.iou_score(output,target)

From Jax

importivyimportraximportnumpyasnp# transpile rax from jax to numpynp_rax=ivy.transpile(rax,source="jax",to="numpy")# get some arraysscores=np.array([2.2,1.3,5.4])labels=np.array([1.0,0.0,0.0])# and use the transpiled version of any function from the library!out=np_rax.poly1_softmax_loss(scores,labels)

Any function

From PyTorch

importivyimporttorchimportnumpyasnpdefloss(predictions,targets):returntorch.sqrt(torch.mean((predictions-targets)**2))# transpile any function from torch to numpynp_loss=ivy.transpile(loss,source="torch",to="numpy")# get some arraysp=np.array([3.0,2.0,1.0])t=np.array([0.0,0.0,0.0])# and use the transpiled version!out=np_loss(p,t)

From TensorFlow

importivyimporttensorflowastfimportnumpyasnpdefloss(predictions,targets):returntf.sqrt(tf.reduce_mean(tf.square(predictions-targets)))# transpile any function from tf to numpynp_loss=ivy.transpile(loss,source="tensorflow",to="numpy")# get some arraysp=np.array([3.0,2.0,1.0])t=np.array([0.0,0.0,0.0])# and use the transpiled version!out=np_loss(p,t)

From JAX

importivyimportjax.numpyasjnpimportnumpyasnpdefloss(predictions,targets):returnjnp.sqrt(jnp.mean((predictions-targets)**2))# transpile any function from jax to numpynp_loss=ivy.transpile(loss,source="jax",to="numpy")# get some arraysp=np.array([3.0,2.0,1.0])t=np.array([0.0,0.0,0.0])# and use the transpiled version!out=np_loss(p,t)

I'm using Ivy 

importivy# A simple image classification modelclassIvyNet(ivy.Module):def__init__(self,h_w=(32,32),input_channels=3,output_channels=512,num_classes=2,data_format="NCHW",device="cpu",    ):self.h_w=h_wself.input_channels=input_channelsself.output_channels=output_channelsself.num_classes=num_classesself.data_format=data_formatsuper().__init__(device=device)def_build(self,*args,**kwargs):self.extractor=ivy.Sequential(ivy.Conv2D(self.input_channels,6, [5,5],1,"SAME",data_format=self.data_format),ivy.GELU(),ivy.Conv2D(6,16, [5,5],1,"SAME",data_format=self.data_format),ivy.GELU(),ivy.Conv2D(16,self.output_channels, [5,5],1,"SAME",data_format=self.data_format),ivy.GELU(),        )self.classifier=ivy.Sequential(# Since the padding is "SAME", this would be image_height x image_width x output_channelsivy.Linear(self.h_w[0]*self.h_w[1]*self.output_channels,512),ivy.GELU(),ivy.Linear(512,self.num_classes),        )def_forward(self,x):x=self.extractor(x)# flatten all dims except batch dimx=ivy.flatten(x,start_dim=1,end_dim=-1)logits=self.classifier(x)probs=ivy.softmax(logits)returnlogits,probs

After building your model in Ivy, you can set your favourite frameworkas the backend to use its operations under the hood!

ivy.set_backend("torch")model=IvyNet()x=torch.randn(1,3,32,32)logits,probs=model(x)

ivy.set_backend("tensorflow")model=IvyNet()x=tf.random.uniform(shape=(1,3,32,32))logits,probs=model(x)

ivy.set_backend("jax")model=IvyNet()x=jax.random.uniform(key,shape=(1,3,32,32))logits,probs=model(x)

ivy.set_backend("numpy")model=IvyNet()x=np.random.uniform(size=(1,3,32,32))logits,probs=model(x)

Last but not least, we can also build the training pipeline in pure ivy⬇️

# helper function for loading the dataset in batchesdefgenerate_batches(images,classes,dataset_size,batch_size=32):ifbatch_size>dataset_size:raiseivy.utils.exceptions.IvyError("Use a smaller batch size")foridxinrange(0,dataset_size,batch_size):yieldimages[idx :min(idx+batch_size,dataset_size)],classes[idx :min(idx+batch_size,dataset_size)        ]# helper function to get the number of current predictionsdefnum_correct(preds,labels):return (preds.argmax()==labels).sum().to_numpy().item()# define a loss functiondefloss_fn(params):v,model,x,y=params_,probs=model(x,v=v)returnivy.cross_entropy(y,probs),probs

# train the model on gpu if it's availabledevice="gpu:0"ifivy.gpu_is_available()else"cpu"# training hyperparamsoptimizer=ivy.Adam(1e-4)batch_size=4num_epochs=20num_classes=10model=IvyNet(h_w=(28,28),input_channels=1,output_channels=120,num_classes=num_classes,device=device,)images=ivy.random_uniform(shape=(16,1,28,28))classes=ivy.randint(0,num_classes-1,shape=(16,))# training loopdeftrain(images,classes,epochs,model,device,num_classes=10,batch_size=32):# training metricsepoch_loss=0.0metrics= []dataset_size=len(images)forepochinrange(epochs):train_correct=0train_loop=tqdm(generate_batches(images,classes,len(images),batch_size=batch_size),total=dataset_size//batch_size,position=0,leave=True,        )forxbatch,ybatchintrain_loop:xbatch,ybatch=xbatch.to_device(device),ybatch.to_device(device)# Since the cross entropy function expects the target classes to be in one-hot encoded formatybatch_encoded=ivy.one_hot(ybatch,num_classes)# update model paramsloss_probs,grads=ivy.execute_with_gradients(loss_fn,                (model.v,model,xbatch,ybatch_encoded),            )model.v=optimizer.step(model.v,grads["0"])batch_loss=ivy.to_numpy(loss_probs[0]).mean().item()# batch mean lossepoch_loss+=batch_loss*xbatch.shape[0]train_correct+=num_correct(loss_probs[1],ybatch)train_loop.set_description(f"Epoch [{epoch+1:2d}/{epochs}]")train_loop.set_postfix(running_loss=batch_loss,accuracy_percentage=(train_correct/dataset_size)*100,            )epoch_loss=epoch_loss/dataset_sizetraining_accuracy=train_correct/dataset_sizemetrics.append([epoch,epoch_loss,training_accuracy])train_loop.write(f"\nAverage training loss:{epoch_loss:.6f}, Train Correct:{train_correct}",end="\n",        )# assuming the dataset(images and classes) are already prepared in a foldertrain(images,classes,num_epochs,model,device,num_classes=num_classes,batch_size=batch_size,)

When should I use Ivy as a transpiler?

If you want to use building blocks published in other frameworks (neuralnetworks, layers, array computing libraries, training pipelines...),you want to integrate code developed in various frameworks, or maybestraight up move code from one framework to another, the transpiler isdefinitely the tool 🔧 for the job! As the output of transpilation isnative code in the target framework, you can use the converted code justas if it was code originally developed in that framework, applyingframework-specific optimizations or tools, instantly exposing yourproject to all of the unique perks of a different framework.

Ivy's transpiler allows you to use code from any other framework (orfrom any other version of the same framework!) in your own code, by justadding one line of code. Under the hood, Ivy traces a computationalgraph and leverages the frontends and backends to link one framework toanother.

This way, Ivy makes all ML-related projects available for you,independently of the framework you want to use to research, develop, ordeploy systems. Feel free to head over to the docs for the full APIreference, but the functions you'd most likely want to use are:

# Traces an efficient fully-functional graph from a function, removing all wrapping and redundant codeivy.trace_graph()# Converts framework-specific code to a different frameworkivy.transpile()# Converts framework-specific code to Ivyivy.unify()

These functions can be used eagerly or lazily. If you pass the necessaryarguments for function tracing, the graph tracing/transpilation step willhappen instantly (eagerly). Otherwise, the graph tracing/transpilationwill happen only when the returned function is first invoked.

importivyimportjaxivy.set_backend("jax")# Simple JAX function to transpiledeftest_fn(x):returnjax.numpy.sum(x)x1=ivy.array([1.,2.])

# Arguments are available -> transpilation happens eagerlyeager_graph=ivy.transpile(test_fn,source="jax",to="torch",args=(x1,))# eager_graph is now torch code and runs efficientlyret=eager_graph(x1)

# Arguments are not available -> transpilation happens lazilylazy_graph=ivy.transpile(test_fn,source="jax",to="torch")# The transpiled graph is initialized, transpilation will happen hereret=lazy_graph(x1)# lazy_graph is now torch code and runs efficientlyret=lazy_graph(x1)

If you want to learn more, you can find more information in theIvy asa transpiler section of thedocs!

When should I use Ivy as a framework?

As Ivy supports multiple backends, writing code in Ivy breaks you freefrom framework limitations. If you want to publish highly flexible codefor everyone to use, independently of the framework they are using, oryou plan to develop ML-related tools and want them to be interoperablewith not only the already existing frameworks, but also with futureframeworks, then Ivy is for you!

The Ivy framework is built on top of various essential components,mainly theBackendHandler,which manages what framework is being used behind the scenes and theBackend FunctionalAPIs,which provide framework-specific implementations of the Ivy functions.Likewise, classes such asivy.Container orivy.Array are alsoavailable, facilitating the use of structured data and array-likeobjects (learn more about themhere!).

All of the functionalities in Ivy are exposed through theIvy functional API and theIvy stateful API. All functions in theFunctionalAPIareFramework Agnostic Functions, which means that we can use themlike this:

importivyimportjax.numpyasjnpimporttensorflowastfimportnumpyasnpimporttorchdefmse_loss(y,target):returnivy.mean((y-target)**2)jax_mse=mse_loss(jnp.ones((5,)),jnp.ones((5,)))tf_mse=mse_loss(tf.ones((5,)),tf.ones((5,)))np_mse=mse_loss(np.ones((5,)),np.ones((5,)))torch_mse=mse_loss(torch.ones((5,)),torch.ones((5,)))

In the example above we show how Ivy's functions are compatible withtensors from different frameworks. This is the same for ALL Ivyfunctions. They can accept tensors from any framework and return thecorrect result.

TheIvy StatefulAPI,on the other hand, allows you to define trainable modules and layers,which you can use alone or as a part of any other framework code!

importivyclassRegressor(ivy.Module):def__init__(self,input_dim,output_dim):self.input_dim=input_dimself.output_dim=output_dimsuper().__init__()def_build(self,*args,**kwargs):self.linear0=ivy.Linear(self.input_dim,128)self.linear1=ivy.Linear(128,self.output_dim)def_forward(self,x):x=self.linear0(x)x=ivy.functional.relu(x)x=self.linear1(x)returnx

If we put it all together, we'll have something like this. This exampleuses PyTorch as the backend, but this can easily be changed to yourfavorite frameworks, such as TensorFlow, or JAX.

importivyclassRegressor(ivy.Module):def__init__(self,input_dim,output_dim):self.input_dim=input_dimself.output_dim=output_dimsuper().__init__()def_build(self,*args,**kwargs):self.linear0=ivy.Linear(self.input_dim,128)self.linear1=ivy.Linear(128,self.output_dim)def_forward(self,x):x=self.linear0(x)x=ivy.functional.relu(x)x=self.linear1(x)returnxivy.set_backend('torch')# set backend to PyTorch (or any other backend!)model=Regressor(input_dim=1,output_dim=1)optimizer=ivy.Adam(0.3)n_training_examples=2000noise=ivy.random.random_normal(shape=(n_training_examples,1),mean=0,std=0.1)x=ivy.linspace(-6,3,n_training_examples).reshape((n_training_examples,1))y=0.2*x**2+0.5*x+0.1+noisedefloss_fn(v,x,target):pred=model(x,v=v)returnivy.mean((pred-target)**2)forepochinrange(40):# forward passpred=model(x)# compute loss and gradientsloss,grads=ivy.execute_with_gradients(lambdaparams:loss_fn(*params), (model.v,x,y))# update parametersmodel.v=optimizer.step(model.v,grads)# print current lossprint(f'Epoch:{epoch+1:2d} --- Loss:{ivy.to_numpy(loss).item():.5f}')print('Finished training!')

The model's output can be visualized as follows:

As always, you can find more information aboutIvy as a framework inthedocs!