Neural Receiver for OFDM SIMO Systems
In this notebook, you will learn how to train a neural receiver that implements OFDM detection. The considered setup is shown in the figure below. As one can see, the neural receiver substitutes channel estimation, equalization, and demapping. It takes as input the post-DFT (discrete Fourier transform) received samples, which form the received resource grid, and computes log-likelihood ratios (LLRs) on the transmitted coded bits. These LLRs are then fed to the outer decoder to reconstruct thetransmitted information bits.
Two baselines are considered for benchmarking, which are shown in the figure above. Both baselines use linear minimum mean square error (LMMSE) equalization and demapping assuming additive white Gaussian noise (AWGN). They differ by how channel estimation is performed:
Pefect CSI: Perfect channel state information (CSI) knowledge is assumed.
LS estimation: Uses the transmitted pilots to perform least squares (LS) estimation of the channel with nearest-neighbor interpolation.
All the considered end-to-end systems use an LDPC outer code from the 5G NR specification, QPSK modulation, and a 3GPP CDL channel model simulated in the frequency domain.
GPU Configuration and Imports
[1]:
importosifos.getenv("CUDA_VISIBLE_DEVICES")isNone:gpu_num=0# Use "" to use the CPUos.environ["CUDA_VISIBLE_DEVICES"]=f"{gpu_num}"os.environ['TF_CPP_MIN_LOG_LEVEL']='3'# Import Sionnatry:importsionna.phyexceptImportErrorase:importsysif'google.colab'insys.modules:# Install Sionna in Google Colabprint("Installing Sionna and restarting the runtime. Please run the cell again.")os.system("pip install sionna")os.kill(os.getpid(),5)else:raisee# Configure the notebook to use only a single GPU and allocate only as much memory as needed# For more details, see https://www.tensorflow.org/guide/gpuimporttensorflowastfgpus=tf.config.list_physical_devices('GPU')ifgpus:try:tf.config.experimental.set_memory_growth(gpus[0],True)exceptRuntimeErrorase:print(e)# Avoid warnings from TensorFlowtf.get_logger().setLevel('ERROR')sionna.phy.config.seed=42# Set seed for reproducible random number generation
[2]:
%matplotlib inlineimportmatplotlib.pyplotaspltimportnumpyasnpimportpicklefromtensorflow.kerasimportModelfromtensorflow.keras.layersimportLayer,Conv2D,LayerNormalizationfromtensorflow.nnimportrelufromsionna.phyimportBlockfromsionna.phy.channel.tr38901importAntenna,AntennaArray,CDLfromsionna.phy.channelimportOFDMChannelfromsionna.phy.mimoimportStreamManagementfromsionna.phy.ofdmimportResourceGrid,ResourceGridMapper,LSChannelEstimator, \LMMSEEqualizer,RemoveNulledSubcarriers,ResourceGridDemapperfromsionna.phy.utilsimportebnodb2no,insert_dims,log10,expand_to_rankfromsionna.phy.fec.ldpcimportLDPC5GEncoder,LDPC5GDecoderfromsionna.phy.mappingimportMapper,Demapper,BinarySourcefromsionna.phy.utilsimportsim_ber
Simulation Parameters
[3]:
############################################## Channel configurationcarrier_frequency=3.5e9# Hzdelay_spread=100e-9# scdl_model="C"# CDL model to usespeed=10.0# Speed for evaluation and training [m/s]# SNR range for evaluation and training [dB]ebno_db_min=-5.0ebno_db_max=10.0############################################## OFDM waveform configurationsubcarrier_spacing=30e3# Hzfft_size=128# Number of subcarriers forming the resource grid, including the null-subcarrier and the guard bandsnum_ofdm_symbols=14# Number of OFDM symbols forming the resource griddc_null=True# Null the DC subcarriernum_guard_carriers=[5,6]# Number of guard carriers on each sidepilot_pattern="kronecker"# Pilot patternpilot_ofdm_symbol_indices=[2,11]# Index of OFDM symbols carrying pilotscyclic_prefix_length=0# Simulation in frequency domain. This is useless############################################## Modulation and coding configurationnum_bits_per_symbol=2# QPSKcoderate=0.5# Coderate for LDPC code############################################## Neural receiver configurationnum_conv_channels=128# Number of convolutional channels for the convolutional layers forming the neural receiver############################################## Training configurationnum_training_iterations=30000# Number of training iterationstraining_batch_size=128# Training batch sizemodel_weights_path="neural_receiver_weights"# Location to save the neural receiver weights once training is done############################################## Evaluation configurationresults_filename="neural_receiver_results"# Location to save the results
TheStreamManagement class is used to configure the receiver-transmitter association and the number of streams per transmitter. A SIMO system is considered, with a single transmitter equipped with a single non-polarized antenna. Therefore, there is only a single stream, and the receiver-transmitter association matrix is\([1]\). The receiver is equipped with an antenna array.
[4]:
stream_manager=StreamManagement(np.array([[1]]),# Receiver-transmitter association matrix1)# One stream per transmitter
TheResourceGrid class is used to configure the OFDM resource grid. It is initialized with the parameters defined above.
[5]:
resource_grid=ResourceGrid(num_ofdm_symbols=num_ofdm_symbols,fft_size=fft_size,subcarrier_spacing=subcarrier_spacing,num_tx=1,num_streams_per_tx=1,cyclic_prefix_length=cyclic_prefix_length,dc_null=dc_null,pilot_pattern=pilot_pattern,pilot_ofdm_symbol_indices=pilot_ofdm_symbol_indices,num_guard_carriers=num_guard_carriers)
Outer coding is performed such that all the databits carried by the resource grid with sizefft_sizexnum_ofdm_symbols form a single codeword.
[6]:
# Codeword length. It is calculated from the total number of databits carried by the resource grid, and the number of bits transmitted per resource elementn=int(resource_grid.num_data_symbols*num_bits_per_symbol)# Number of information bits per codewordk=int(n*coderate)
The SIMO link is setup by considering an uplink transmission with one user terminal (UT) equipped with a single non-polarized antenna, and a base station (BS) equipped with an antenna array. One can try other configurations for the BS antenna array.
[7]:
ut_antenna=Antenna(polarization="single",polarization_type="V",antenna_pattern="38.901",carrier_frequency=carrier_frequency)bs_array=AntennaArray(num_rows=1,num_cols=1,polarization="dual",polarization_type="VH",antenna_pattern="38.901",carrier_frequency=carrier_frequency)
Neural Receiver
The next cell defines the Keras layers that implement the neural receiver. As in [1] and [2], a neural receiver using residual convolutional layers is implemented. Convolutional layers are leveraged to efficienly process the 2D resource grid, that is fed as an input to the neural receiver. Residual (skip) connections are used to avoid gradient vanishing [3].
For convinience, a Keras layer that implements aresidual block is first defined. The Keras layer that implements the neural receiver is built by stacking such blocks. The following figure shows the architecture of the neural receiver.
[8]:
classResidualBlock(Layer):r""" This Keras layer implements a convolutional residual block made of two convolutional layers with ReLU activation, layer normalization, and a skip connection. The number of convolutional channels of the input must match the number of kernel of the convolutional layers ``num_conv_channel`` for the skip connection to work. Input ------ : [batch size, num time samples, num subcarriers, num_conv_channel], tf.float Input of the layer Output ------- : [batch size, num time samples, num subcarriers, num_conv_channel], tf.float Output of the layer """defbuild(self,input_shape):# Layer normalization is done over the last three dimensions: time, frequency, conv 'channels'self._layer_norm_1=LayerNormalization(axis=(-1,-2,-3))self._conv_1=Conv2D(filters=num_conv_channels,kernel_size=[3,3],padding='same',activation=None)# Layer normalization is done over the last three dimensions: time, frequency, conv 'channels'self._layer_norm_2=LayerNormalization(axis=(-1,-2,-3))self._conv_2=Conv2D(filters=num_conv_channels,kernel_size=[3,3],padding='same',activation=None)defcall(self,inputs):z=self._layer_norm_1(inputs)z=relu(z)z=self._conv_1(z)z=self._layer_norm_2(z)z=relu(z)z=self._conv_2(z)# [batch size, num time samples, num subcarriers, num_channels]# Skip connectionz=z+inputsreturnzclassNeuralReceiver(Layer):r""" Keras layer implementing a residual convolutional neural receiver. This neural receiver is fed with the post-DFT received samples, forming a resource grid of size num_of_symbols x fft_size, and computes LLRs on the transmitted coded bits. These LLRs can then be fed to an outer decoder to reconstruct the information bits. As the neural receiver is fed with the entire resource grid, including the guard bands and pilots, it also computes LLRs for these resource elements. They must be discarded to only keep the LLRs corresponding to the data-carrying resource elements. Input ------ y : [batch size, num rx antenna, num ofdm symbols, num subcarriers], tf.complex Received post-DFT samples. no : [batch size], tf.float32 Noise variance. At training, a different noise variance value is sampled for each batch example. Output ------- : [batch size, num ofdm symbols, num subcarriers, num_bits_per_symbol] LLRs on the transmitted bits. LLRs computed for resource elements not carrying data (pilots, guard bands...) must be discarded. """defbuild(self,input_shape):# Input convolutionself._input_conv=Conv2D(filters=num_conv_channels,kernel_size=[3,3],padding='same',activation=None)# Residual blocksself._res_block_1=ResidualBlock()self._res_block_2=ResidualBlock()self._res_block_3=ResidualBlock()self._res_block_4=ResidualBlock()# Output convself._output_conv=Conv2D(filters=num_bits_per_symbol,kernel_size=[3,3],padding='same',activation=None)defcall(self,y,no):# Feeding the noise power in log10 scale helps with the performanceno=log10(no)# Stacking the real and imaginary components of the different antennas along the 'channel' dimensiony=tf.transpose(y,[0,2,3,1])# Putting antenna dimension lastno=insert_dims(no,3,1)no=tf.tile(no,[1,y.shape[1],y.shape[2],1])# z : [batch size, num ofdm symbols, num subcarriers, 2*num rx antenna + 1]z=tf.concat([tf.math.real(y),tf.math.imag(y),no],axis=-1)# Input convz=self._input_conv(z)# Residual blocksz=self._res_block_1(z)z=self._res_block_2(z)z=self._res_block_3(z)z=self._res_block_4(z)# Output convz=self._output_conv(z)returnz
End-to-end System
The following cell defines the end-to-end system.
Training is done on the bit-metric decoding (BMD) rate which is computed from the transmitted bits and LLRs:
\begin{equation}R = 1 - \frac{1}{SNMK} \sum_{s = 0}^{S-1} \sum_{n = 0}^{N-1} \sum_{m = 0}^{M-1} \sum_{k = 0}^{K-1} \texttt{BCE} \left( B_{s,n,m,k}, \texttt{LLR}_{s,n,m,k} \right)\end{equation}
where
\(S\) is the batch size
\(N\) the number of subcarriers
\(M\) the number of OFDM symbols
\(K\) the number of bits per symbol
\(B_{s,n,m,k}\) the\(k^{th}\) coded bit transmitted on the resource element\((n,m)\) and for the\(s^{th}\) batch example
\(\texttt{LLR}_{s,n,m,k}\) the LLR (logit) computed by the neural receiver corresponding to the\(k^{th}\) coded bit transmitted on the resource element\((n,m)\) and for the\(s^{th}\) batch example
\(\texttt{BCE} \left( \cdot, \cdot \right)\) the binary cross-entropy in log base 2
Because no outer code is required at training, the outer encoder and decoder are not used at training to reduce computational complexity.
The BMD rate is known to be an achievable information rate for BICM systems, which motivates its used as objective function [4].
[9]:
## Transmitterbinary_source=BinarySource()mapper=Mapper("qam",num_bits_per_symbol)rg_mapper=ResourceGridMapper(resource_grid)## Channelcdl=CDL(cdl_model,delay_spread,carrier_frequency,ut_antenna,bs_array,"uplink",min_speed=speed)channel=OFDMChannel(cdl,resource_grid,normalize_channel=True,return_channel=True)## Receiverneural_receiver=NeuralReceiver()rg_demapper=ResourceGridDemapper(resource_grid,stream_manager)# Used to extract data-carrying resource elements
The following cell performs one forward step through the end-to-end system:
[10]:
batch_size=64ebno_db=tf.fill([batch_size],5.0)no=ebnodb2no(ebno_db,num_bits_per_symbol,coderate)## Transmitter# Generate codewordsc=binary_source([batch_size,1,1,n])print("c shape: ",c.shape)# Map bits to QAM symbolsx=mapper(c)print("x shape: ",x.shape)# Map the QAM symbols to a resource gridx_rg=rg_mapper(x)print("x_rg shape: ",x_rg.shape)######################################## Channel# A batch of new channel realizations is sampled and applied at every inferenceno_=expand_to_rank(no,tf.rank(x_rg))y,_=channel(x_rg,no_)print("y shape: ",y.shape)######################################## Receiver# The neural receiver computes LLRs from the frequency domain received symbols and N0y=tf.squeeze(y,axis=1)llr=neural_receiver(y,no)print("llr shape: ",llr.shape)# Reshape the input to fit what the resource grid demapper is expectedllr=insert_dims(llr,2,1)# Extract data-carrying resource elements. The other LLRs are discardedllr=rg_demapper(llr)llr=tf.reshape(llr,[batch_size,1,1,n])print("Post RG-demapper LLRs: ",llr.shape)
c shape: (64, 1, 1, 2784)x shape: (64, 1, 1, 1392)x_rg shape: (64, 1, 1, 14, 128)y shape: (64, 1, 2, 14, 128)llr shape: (64, 14, 128, 2)Post RG-demapper LLRs: (64, 1, 1, 2784)
The BMD rate is computed from the LLRs and transmitted bits as follows:
[11]:
bce=tf.nn.sigmoid_cross_entropy_with_logits(c,llr)bce=tf.reduce_mean(bce)rate=tf.constant(1.0,tf.float32)-bce/tf.math.log(2.)print(f"Rate:{rate:.2E} bit")
Rate: -7.58E-01 bit
The rate is very poor (negative values means 0 bit) as the neural receiver is not trained.
End-to-end System as a Sionna Block
The following Sionna block implements the three considered end-to-end systems (perfect CSI baseline, LS estimation baseline, and neural receiver).
When instantiating the end-to-end model, the parametersystem is used to specify the system to setup, and the parametertraining is used to specified if the system is instantiated to be trained or to be evaluated. Thetraining parameter is only relevant when the neural receiver is used.
At each call of this model:
A batch of codewords is randomly sampled, modulated, and mapped to resource grids to form the channel inputs
A batch of channel realizations is randomly sampled and applied to the channel inputs
The receiver is executed on the post-DFT received samples to compute LLRs on the coded bits. Which receiver is executed (baseline with perfect CSI knowledge, baseline with LS estimation, or neural receiver) depends on the specified
systemparameter.If not training, the outer decoder is applied to reconstruct the information bits
If training, the BMD rate is estimated over the batch from the LLRs and the transmitted bits
[12]:
classE2ESystem(Block):r""" Sionna Block that implements the end-to-end system As the three considered end-to-end systems (perfect CSI baseline, LS estimation baseline, and neural receiver) share most of the link components (transmitter, channel model, outer code...), they are implemented using the same end-to-end model. When instantiating the Sionna block, the parameter ``system`` is used to specify the system to setup, and the parameter ``training`` is used to specified if the system is instantiated to be trained or to be evaluated. The ``training`` parameter is only relevant when the neural At each call of this model: * A batch of codewords is randomly sampled, modulated, and mapped to resource grids to form the channel inputs * A batch of channel realizations is randomly sampled and applied to the channel inputs * The receiver is executed on the post-DFT received samples to compute LLRs on the coded bits. Which receiver is executed (baseline with perfect CSI knowledge, baseline with LS estimation, or neural receiver) depends on the specified ``system`` parameter. * If not training, the outer decoder is applied to reconstruct the information bits * If training, the BMD rate is estimated over the batch from the LLRs and the transmitted bits Parameters ----------- system : str Specify the receiver to use. Should be one of 'baseline-perfect-csi', 'baseline-ls-estimation' or 'neural-receiver' training : bool Set to `True` if the system is instantiated to be trained. Set to `False` otherwise. Defaults to `False`. If the system is instantiated to be trained, the outer encoder and decoder are not instantiated as they are not required for training. This significantly reduces the computational complexity of training. If training, the bit-metric decoding (BMD) rate is computed from the transmitted bits and the LLRs. The BMD rate is known to be an achievable information rate for BICM systems, and therefore training of the neural receiver aims at maximizing this rate. Input ------ batch_size : int Batch size no : scalar or [batch_size], tf.float Noise variance. At training, a different noise variance should be sampled for each batch example. Output ------- If ``training`` is set to `True`, then the output is a single scalar, which is an estimation of the BMD rate computed over the batch. It should be used as objective for training. If ``training`` is set to `False`, the transmitted information bits and their reconstruction on the receiver side are returned to compute the block/bit error rate. """def__init__(self,system,training=False):super().__init__()self._system=systemself._training=training######################################## Transmitterself._binary_source=BinarySource()# To reduce the computational complexity of training, the outer code is not used when training,# as it is not requiredifnottraining:self._encoder=LDPC5GEncoder(k,n)self._mapper=Mapper("qam",num_bits_per_symbol)self._rg_mapper=ResourceGridMapper(resource_grid)######################################## Channel# A 3GPP CDL channel model is usedcdl=CDL(cdl_model,delay_spread,carrier_frequency,ut_antenna,bs_array,"uplink",min_speed=speed)self._channel=OFDMChannel(cdl,resource_grid,normalize_channel=True,return_channel=True)######################################## Receiver# Three options for the receiver depending on the value of `system`if"baseline"insystem:ifsystem=='baseline-perfect-csi':# Perfect CSIself._removed_null_subc=RemoveNulledSubcarriers(resource_grid)elifsystem=='baseline-ls-estimation':# LS estimationself._ls_est=LSChannelEstimator(resource_grid,interpolation_type="nn")# Components required by both baselinesself._lmmse_equ=LMMSEEqualizer(resource_grid,stream_manager,)self._demapper=Demapper("app","qam",num_bits_per_symbol)elifsystem=="neural-receiver":# Neural receiverself._neural_receiver=NeuralReceiver()self._rg_demapper=ResourceGridDemapper(resource_grid,stream_manager)# Used to extract data-carrying resource elements# To reduce the computational complexity of training, the outer code is not used when training,# as it is not requiredifnottraining:self._decoder=LDPC5GDecoder(self._encoder,hard_out=True)@tf.functiondefcall(self,batch_size,ebno_db):# If `ebno_db` is a scalar, a tensor with shape [batch size] is created as it is what is expected by some layersiflen(ebno_db.shape)==0:ebno_db=tf.fill([batch_size],ebno_db)######################################## Transmitterno=ebnodb2no(ebno_db,num_bits_per_symbol,coderate)# Outer coding is only performed if not trainingifself._training:c=self._binary_source([batch_size,1,1,n])else:b=self._binary_source([batch_size,1,1,k])c=self._encoder(b)# Modulationx=self._mapper(c)x_rg=self._rg_mapper(x)######################################## Channel# A batch of new channel realizations is sampled and applied at every inferenceno_=expand_to_rank(no,tf.rank(x_rg))y,h=self._channel(x_rg,no_)######################################## Receiver# Three options for the receiver depending on the value of ``system``if"baseline"inself._system:ifself._system=='baseline-perfect-csi':h_hat=self._removed_null_subc(h)# Extract non-null subcarrierserr_var=0.0# No channel estimation error when perfect CSI knowledge is assumedelifself._system=='baseline-ls-estimation':h_hat,err_var=self._ls_est(y,no)# LS channel estimation with nearest-neighborx_hat,no_eff=self._lmmse_equ(y,h_hat,err_var,no)# LMMSE equalizationno_eff_=expand_to_rank(no_eff,tf.rank(x_hat))llr=self._demapper(x_hat,no_eff_)# Demappingelifself._system=="neural-receiver":# The neural receiver computes LLRs from the frequency domain received symbols and N0y=tf.squeeze(y,axis=1)llr=self._neural_receiver(y,no)llr=insert_dims(llr,2,1)# Reshape the input to fit what the resource grid demapper is expectedllr=self._rg_demapper(llr)# Extract data-carrying resource elements. The other LLrs are discardedllr=tf.reshape(llr,[batch_size,1,1,n])# Reshape the LLRs to fit what the outer decoder is expected# Outer coding is not needed if the information rate is returnedifself._training:# Compute and return BMD rate (in bit), which is known to be an achievable# information rate for BICM systems.# Training aims at maximizing the BMD ratebce=tf.nn.sigmoid_cross_entropy_with_logits(c,llr)bce=tf.reduce_mean(bce)rate=tf.constant(1.0,tf.float32)-bce/tf.math.log(2.)returnrateelse:# Outer decodingb_hat=self._decoder(llr)returnb,b_hat# Ground truth and reconstructed information bits returned for BER/BLER computation
Evaluation of the Baselines
We evaluate the BERs achieved by the baselines in the next cell.
Note: Evaluation of the two systems can take a while. Therefore, we provide pre-computed results at the end of this notebook.
[13]:
# Range of SNRs over which the systems are evaluatedebno_dbs=np.arange(ebno_db_min,# Min SNR for evaluationebno_db_max,# Max SNR for evaluation0.5)# Step
[14]:
# Dictionary storing the evaluation resultsBLER={}model=E2ESystem('baseline-perfect-csi')_,bler=sim_ber(model,ebno_dbs,batch_size=128,num_target_block_errors=100,max_mc_iter=100)BLER['baseline-perfect-csi']=bler.numpy()model=E2ESystem('baseline-ls-estimation')_,bler=sim_ber(model,ebno_dbs,batch_size=128,num_target_block_errors=100,max_mc_iter=100)BLER['baseline-ls-estimation']=bler.numpy()
EbNo [dB] | BER | BLER | bit errors | num bits | block errors | num blocks | runtime [s] | status--------------------------------------------------------------------------------------------------------------------------------------- -5.0 | 2.5209e-01 | 1.0000e+00 | 44916 | 178176 | 128 | 128 | 9.4 |reached target block errors -4.5 | 2.3712e-01 | 1.0000e+00 | 42249 | 178176 | 128 | 128 | 0.1 |reached target block errors -4.0 | 2.1769e-01 | 1.0000e+00 | 38787 | 178176 | 128 | 128 | 0.1 |reached target block errors -3.5 | 1.9162e-01 | 1.0000e+00 | 34142 | 178176 | 128 | 128 | 0.1 |reached target block errors -3.0 | 1.5846e-01 | 1.0000e+00 | 28233 | 178176 | 128 | 128 | 0.1 |reached target block errors -2.5 | 1.0702e-01 | 9.9219e-01 | 19068 | 178176 | 127 | 128 | 0.1 |reached target block errors -2.0 | 1.6891e-02 | 5.1172e-01 | 6019 | 356352 | 131 | 256 | 0.2 |reached target block errors -1.5 | 8.1623e-04 | 2.6562e-02 | 4363 | 5345280 | 102 | 3840 | 2.4 |reached target block errors -1.0 | 1.3958e-04 | 2.1875e-03 | 2487 | 17817600 | 28 | 12800 | 8.0 |reached max iterations -0.5 | 1.2628e-05 | 1.5625e-04 | 225 | 17817600 | 2 | 12800 | 8.0 |reached max iterations 0.0 | 1.5266e-05 | 1.5625e-04 | 272 | 17817600 | 2 | 12800 | 8.0 |reached max iterations 0.5 | 1.4592e-06 | 7.8125e-05 | 26 | 17817600 | 1 | 12800 | 8.0 |reached max iterations 1.0 | 0.0000e+00 | 0.0000e+00 | 0 | 17817600 | 0 | 12800 | 8.0 |reached max iterationsSimulation stopped as no error occurred @ EbNo = 1.0 dB.EbNo [dB] | BER | BLER | bit errors | num bits | block errors | num blocks | runtime [s] | status--------------------------------------------------------------------------------------------------------------------------------------- -5.0 | 3.9043e-01 | 1.0000e+00 | 69565 | 178176 | 128 | 128 | 3.0 |reached target block errors -4.5 | 3.7739e-01 | 1.0000e+00 | 67241 | 178176 | 128 | 128 | 0.1 |reached target block errors -4.0 | 3.6709e-01 | 1.0000e+00 | 65406 | 178176 | 128 | 128 | 0.1 |reached target block errors -3.5 | 3.5535e-01 | 1.0000e+00 | 63315 | 178176 | 128 | 128 | 0.1 |reached target block errors -3.0 | 3.4154e-01 | 1.0000e+00 | 60855 | 178176 | 128 | 128 | 0.1 |reached target block errors -2.5 | 3.2643e-01 | 1.0000e+00 | 58162 | 178176 | 128 | 128 | 0.1 |reached target block errors -2.0 | 3.1102e-01 | 1.0000e+00 | 55416 | 178176 | 128 | 128 | 0.1 |reached target block errors -1.5 | 2.9498e-01 | 1.0000e+00 | 52558 | 178176 | 128 | 128 | 0.1 |reached target block errors -1.0 | 2.7463e-01 | 1.0000e+00 | 48932 | 178176 | 128 | 128 | 0.1 |reached target block errors -0.5 | 2.5716e-01 | 1.0000e+00 | 45820 | 178176 | 128 | 128 | 0.1 |reached target block errors 0.0 | 2.3105e-01 | 1.0000e+00 | 41167 | 178176 | 128 | 128 | 0.1 |reached target block errors 0.5 | 2.0566e-01 | 1.0000e+00 | 36643 | 178176 | 128 | 128 | 0.1 |reached target block errors 1.0 | 1.7111e-01 | 1.0000e+00 | 30487 | 178176 | 128 | 128 | 0.1 |reached target block errors 1.5 | 9.3728e-02 | 9.2969e-01 | 16700 | 178176 | 119 | 128 | 0.1 |reached target block errors 2.0 | 1.1097e-02 | 2.4805e-01 | 7909 | 712704 | 127 | 512 | 0.3 |reached target block errors 2.5 | 1.0694e-03 | 1.3706e-02 | 10861 | 10156032 | 100 | 7296 | 4.6 |reached target block errors 3.0 | 2.0177e-04 | 2.0313e-03 | 3595 | 17817600 | 26 | 12800 | 8.0 |reached max iterations 3.5 | 6.2130e-05 | 5.4688e-04 | 1107 | 17817600 | 7 | 12800 | 8.1 |reached max iterations 4.0 | 2.6715e-05 | 2.3437e-04 | 476 | 17817600 | 3 | 12800 | 8.1 |reached max iterations 4.5 | 1.5154e-05 | 7.8125e-05 | 270 | 17817600 | 1 | 12800 | 8.1 |reached max iterations 5.0 | 0.0000e+00 | 0.0000e+00 | 0 | 17817600 | 0 | 12800 | 8.1 |reached max iterationsSimulation stopped as no error occurred @ EbNo = 5.0 dB.
Training the Neural Receiver
In the next cell, one forward pass is performed within agradient tape, which enables the computation of gradient and therefore the optimization of the neural network through stochastic gradient descent (SGD).
Note: For an introduction to the implementation of differentiable communication systems and their optimization through SGD and backpropagation with Sionna, please refer tothe Part 2 of the Sionna tutorial for Beginners.
[15]:
# The end-to-end system equipped with the neural receiver is instantiated for training.# When called, it therefore returns the estimated BMD ratemodel=E2ESystem('neural-receiver',training=True)# Sampling a batch of SNRsebno_db=tf.random.uniform(shape=[],minval=ebno_db_min,maxval=ebno_db_max)# Forward passwithtf.GradientTape()astape:rate=model(training_batch_size,ebno_db)# Tensorflow optimizers only know how to minimize loss function.# Therefore, a loss function is defined as the additive inverse of the BMD rateloss=-rate
Next, one can perform one step of stochastic gradient descent (SGD). The Adam optimizer is used
[16]:
optimizer=tf.keras.optimizers.Adam()# Computing and applying gradientsweights=tape.watched_variables()grads=tape.gradient(loss,weights)optimizer.apply_gradients(zip(grads,weights))
[16]:
<Variable path=adam/iteration, shape=(), dtype=int64, value=1>
Training consists in looping over SGD steps. The next cell implements a training loop.
At each iteration:
A batch of SNRs\(E_b/N_0\) is sampled
A forward pass through the end-to-end system is performed within a gradient tape
The gradients are computed using the gradient tape, and applied using the Adam optimizer
The achieved BMD rate is periodically shown
After training, the weights of the models are saved in a file
Note: Training can take a while. Therefore,we have made pre-trained weights available. Do not execute the next cell if you don’t want to train the model from scratch.
[17]:
training=False# Change to True to train your own modeliftraining:model=E2ESystem('neural-receiver',training=True)optimizer=tf.keras.optimizers.Adam()foriinrange(num_training_iterations):# Sampling a batch of SNRsebno_db=tf.random.uniform(shape=[],minval=ebno_db_min,maxval=ebno_db_max)# Forward passwithtf.GradientTape()astape:rate=model(training_batch_size,ebno_db)# Tensorflow optimizers only know how to minimize loss function.# Therefore, a loss function is defined as the additive inverse of the BMD rateloss=-rate# Computing and applying gradientsweights=tape.watched_variables()grads=tape.gradient(loss,weights)optimizer.apply_gradients(zip(grads,weights))# Periodically printing the progressifi%100==0:print('Iteration{}/{} Rate:{:.4f} bit'.format(i,num_training_iterations,rate.numpy()),end='\r')# Save the weights in a fileweights=model._neural_receiver.weightswithopen(model_weights_path,'wb')asf:pickle.dump(weights,f)
Evaluation of the Neural Receiver
The next cell evaluates the neural receiver.
Note: Evaluation of the system can take a while and requires having the trained weights of the neural receiver. Therefore, we provide pre-computed results at the end of this notebook.
[18]:
model=E2ESystem('neural-receiver')# Run one inference to build the layers and loading the weightsmodel(1,tf.constant(10.0,tf.float32))withopen(model_weights_path,'rb')asf:weights=pickle.load(f)fori,winenumerate(weights):model._neural_receiver.weights[i].assign(w)# Evaluations_,bler=sim_ber(model,ebno_dbs,batch_size=128,num_target_block_errors=100,max_mc_iter=100)BLER['neural-receiver']=bler.numpy()
EbNo [dB] | BER | BLER | bit errors | num bits | block errors | num blocks | runtime [s] | status--------------------------------------------------------------------------------------------------------------------------------------- -5.0 | 2.6003e-01 | 1.0000e+00 | 46331 | 178176 | 128 | 128 | 2.7 |reached target block errors -4.5 | 2.4025e-01 | 1.0000e+00 | 42806 | 178176 | 128 | 128 | 0.1 |reached target block errors -4.0 | 2.2764e-01 | 1.0000e+00 | 40560 | 178176 | 128 | 128 | 0.1 |reached target block errors -3.5 | 2.0312e-01 | 1.0000e+00 | 36191 | 178176 | 128 | 128 | 0.1 |reached target block errors -3.0 | 1.7575e-01 | 1.0000e+00 | 31314 | 178176 | 128 | 128 | 0.1 |reached target block errors -2.5 | 1.4274e-01 | 1.0000e+00 | 25433 | 178176 | 128 | 128 | 0.1 |reached target block errors -2.0 | 4.2873e-02 | 8.3594e-01 | 7639 | 178176 | 107 | 128 | 0.1 |reached target block errors -1.5 | 3.7594e-03 | 1.3672e-01 | 4019 | 1069056 | 105 | 768 | 0.6 |reached target block errors -1.0 | 7.4014e-04 | 9.7656e-03 | 10550 | 14254080 | 100 | 10240 | 8.5 |reached target block errors -0.5 | 3.1817e-04 | 2.5781e-03 | 5669 | 17817600 | 33 | 12800 | 10.7 |reached max iterations 0.0 | 1.0080e-04 | 9.3750e-04 | 1796 | 17817600 | 12 | 12800 | 10.7 |reached max iterations 0.5 | 1.1393e-04 | 9.3750e-04 | 2030 | 17817600 | 12 | 12800 | 10.7 |reached max iterations 1.0 | 5.6349e-05 | 3.9063e-04 | 1004 | 17817600 | 5 | 12800 | 10.7 |reached max iterations 1.5 | 1.5490e-05 | 2.3437e-04 | 276 | 17817600 | 3 | 12800 | 10.7 |reached max iterations 2.0 | 2.1888e-05 | 1.5625e-04 | 390 | 17817600 | 2 | 12800 | 10.7 |reached max iterations 2.5 | 1.4256e-05 | 7.8125e-05 | 254 | 17817600 | 1 | 12800 | 10.7 |reached max iterations 3.0 | 3.1149e-05 | 1.5625e-04 | 555 | 17817600 | 2 | 12800 | 10.7 |reached max iterations 3.5 | 8.9799e-07 | 7.8125e-05 | 16 | 17817600 | 1 | 12800 | 10.7 |reached max iterations 4.0 | 0.0000e+00 | 0.0000e+00 | 0 | 17817600 | 0 | 12800 | 10.7 |reached max iterationsSimulation stopped as no error occurred @ EbNo = 4.0 dB.
Finally, we plots the BLERs
[19]:
plt.figure(figsize=(10,6))# Baseline - Perfect CSIplt.semilogy(ebno_dbs,BLER['baseline-perfect-csi'],'o-',c=f'C0',label=f'Baseline - Perfect CSI')# Baseline - LS Estimationplt.semilogy(ebno_dbs,BLER['baseline-ls-estimation'],'x--',c=f'C1',label=f'Baseline - LS Estimation')# Neural receiverplt.semilogy(ebno_dbs,BLER['neural-receiver'],'s-.',c=f'C2',label=f'Neural receiver')#plt.xlabel(r"$E_b/N_0$ (dB)")plt.ylabel("BLER")plt.grid(which="both")plt.ylim((1e-4,1.0))plt.legend()plt.tight_layout()
Pre-computed Results
[20]:
pre_computed_results="{'baseline-perfect-csi': [1.0, 1.0, 1.0, 1.0, 1.0, 0.9916930379746836, 0.5367080479452054, 0.0285078125, 0.0017890625, 0.0006171875, 0.0002265625, 9.375e-05, 2.34375e-05, 7.8125e-06, 1.5625e-05, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'baseline-ls-estimation': [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.9998022151898734, 0.9199448529411764, 0.25374190938511326, 0.0110234375, 0.002078125, 0.0008359375, 0.0004375, 0.000171875, 9.375e-05, 4.6875e-05, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'neural-receiver': [1.0, 1.0, 1.0, 1.0, 1.0, 0.9984177215189873, 0.7505952380952381, 0.10016025641025642, 0.00740625, 0.0021640625, 0.000984375, 0.0003671875, 0.000203125, 0.0001484375, 3.125e-05, 2.34375e-05, 7.8125e-06, 7.8125e-06, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}"BLER=eval(pre_computed_results)
References
[1] M. Honkala, D. Korpi and J. M. J. Huttunen, “DeepRx: Fully Convolutional Deep Learning Receiver,” in IEEE Transactions on Wireless Communications, vol. 20, no. 6, pp. 3925-3940, June 2021, doi: 10.1109/TWC.2021.3054520.
[2] F. Ait Aoudia and J. Hoydis, “End-to-end Learning for OFDM: From Neural Receivers to Pilotless Communication,” in IEEE Transactions on Wireless Communications, doi: 10.1109/TWC.2021.3101364.
[3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, “Deep Residual Learning for Image Recognition”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778
[4] G. Böcherer, “Achievable Rates for Probabilistic Shaping”, arXiv:1707.01134, 2017.