1//===-- RISCVISelLowering.h - RISC-V DAG Lowering Interface -----*- C++ -*-===// 3// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. 4// See https://llvm.org/LICENSE.txt for license information. 5// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception 7//===----------------------------------------------------------------------===// 9// This file defines the interfaces that RISC-V uses to lower LLVM code into a 12//===----------------------------------------------------------------------===// 14#ifndef LLVM_LIB_TARGET_RISCV_RISCVISELLOWERING_H 15#define LLVM_LIB_TARGET_RISCV_RISCVISELLOWERING_H 27structRISCVRegisterInfo;
38 /// Select with condition operator - This selects between a true value and 39 /// a false value (ops #3 and #4) based on the boolean result of comparing 40 /// the lhs and rhs (ops #0 and #1) of a conditional expression with the 41 /// condition code in op #2, a XLenVT constant from the ISD::CondCode enum. 42 /// The lhs and rhs are XLenVT integers. The true and false values can be 43 /// integer or floating point. 47 /// Turn a pair of `i<xlen>`s into an even-odd register pair (`untyped`). 48 /// - Output: `untyped` even-odd register pair 49 /// - Input 0: `i<xlen>` low-order bits, for even register. 50 /// - Input 1: `i<xlen>` high-order bits, for odd register. 53 /// Turn an even-odd register pair (`untyped`) into a pair of `i<xlen>`s. 54 /// - Output 0: `i<xlen>` low-order bits, from even register. 55 /// - Output 1: `i<xlen>` high-order bits, from odd register. 56 /// - Input: `untyped` even-odd register pair 59 /// Turns a pair of `i32`s into an `f64`. Needed for rv32d/ilp32. 61 /// - Input 0: low-order bits (31-0) (as `i32`), for even register. 62 /// - Input 1: high-order bits (63-32) (as `i32`), for odd register. 65 /// Turns a `f64` into a pair of `i32`s. Needed for rv32d/ilp32. 66 /// - Output 0: low-order bits (31-0) (as `i32`), from even register. 67 /// - Output 1: high-order bits (63-32) (as `i32`), from odd register. 71// Add the Lo 12 bits from an address. Selected to ADDI. 73// Get the Hi 20 bits from an address. Selected to LUI. 76// Represents an AUIPC+ADDI pair. Selected to PseudoLLA. 79// Selected as PseudoAddTPRel. Used to emit a TP-relative relocation. 82// Multiply high for signedxunsigned. 85// Represents (ADD (SHL a, b), c) with the arguments appearing in the order 86// a, b, c. 'b' must be a constant. Maps to sh1add/sh2add/sh3add with zba 87// or addsl with XTheadBa. 90// RV64I shifts, directly matching the semantics of the named RISC-V 95// 32-bit operations from RV64M that can't be simply matched with a pattern 96// at instruction selection time. These have undefined behavior for division 97// by 0 or overflow (divw) like their target independent counterparts. 101// RV64IB rotates, directly matching the semantics of the named RISC-V 105// RV64IZbb bit counting instructions directly matching the semantics of the 106// named RISC-V instructions. 110// RV64IZbb absolute value for i32. Expanded to (max (negw X), X) during isel. 113// FPR<->GPR transfer operations when the FPR is smaller than XLEN, needed as 114// XLEN is the only legal integer width. 116// FMV_H_X matches the semantics of the FMV.H.X. 117// FMV_X_ANYEXTH is similar to FMV.X.H but has an any-extended result. 118// FMV_X_SIGNEXTH is similar to FMV.X.H and has a sign-extended result. 119// FMV_W_X_RV64 matches the semantics of the FMV.W.X. 120// FMV_X_ANYEXTW_RV64 is similar to FMV.X.W but has an any-extended result. 122// This is a more convenient semantic for producing dagcombines that remove 123// unnecessary GPR->FPR->GPR moves. 129// FP to XLen int conversions. Corresponds to fcvt.l(u).s/d/h on RV64 and 130// fcvt.w(u).s/d/h on RV32. Unlike FP_TO_S/UINT these saturate out of 131// range inputs. These are used for FP_TO_S/UINT_SAT lowering. Rounding mode 132// is passed as a TargetConstant operand using the RISCVFPRndMode enum. 135// FP to 32 bit int conversions for RV64. These are used to keep track of the 136// result being sign extended to 64 bit. These saturate out of range inputs. 137// Used for FP_TO_S/UINT and FP_TO_S/UINT_SAT lowering. Rounding mode 138// is passed as a TargetConstant operand using the RISCVFPRndMode enum. 142// Rounds an FP value to its corresponding integer in the same FP format. 143// First operand is the value to round, the second operand is the largest 144// integer that can be represented exactly in the FP format. This will be 145// expanded into multiple instructions and basic blocks with a custom 152// Floating point fmax and fmin matching the RISC-V instruction semantics. 155// Zfa fli instruction for constant materialization. 158// A read of the 64-bit counter CSR on a 32-bit target (returns (Lo, Hi)). 159// It takes a chain operand and another two target constant operands (the 160// CSR numbers of the low and high parts of the counter). 163// brev8, orc.b, zip, and unzip from Zbb and Zbkb. All operands are i32 or 170// Scalar cryptography 181// VMV_V_V_VL matches the semantics of vmv.v.v but includes an extra operand 182// for the VL value to be used for the operation. The first operand is 185// VMV_V_X_VL matches the semantics of vmv.v.x but includes an extra operand 186// for the VL value to be used for the operation. The first operand is 189// VFMV_V_F_VL matches the semantics of vfmv.v.f but includes an extra operand 190// for the VL value to be used for the operation. The first operand is 193// VMV_X_S matches the semantics of vmv.x.s. The result is always XLenVT sign 194// extended from the vector element size. 196// VMV_S_X_VL matches the semantics of vmv.s.x. It carries a VL operand. 198// VFMV_S_F_VL matches the semantics of vfmv.s.f. It carries a VL operand. 200// Splats an 64-bit value that has been split into two i32 parts. This is 201// expanded late to two scalar stores and a stride 0 vector load. 202// The first operand is passthru operand. 204// Truncates a RVV integer vector by one power-of-two. Carries both an extra 205// mask and VL operand. 207// Truncates a RVV integer vector by one power-of-two. If the value doesn't 208// fit in the destination type, the result is saturated. These correspond to 209// vnclip and vnclipu with a shift of 0. Carries both an extra mask and VL 213// Matches the semantics of vslideup/vslidedown. The first operand is the 214// pass-thru operand, the second is the source vector, the third is the XLenVT 215// index (either constant or non-constant), the fourth is the mask, the fifth 216// is the VL and the sixth is the policy. 219// Matches the semantics of vslide1up/slide1down. The first operand is 220// passthru operand, the second is source vector, third is the XLenVT scalar 221// value. The fourth and fifth operands are the mask and VL operands. 224// Matches the semantics of vfslide1up/vfslide1down. The first operand is 225// passthru operand, the second is source vector, third is a scalar value 226// whose type matches the element type of the vectors. The fourth and fifth 227// operands are the mask and VL operands. 230// Matches the semantics of the vid.v instruction, with a mask and VL 233// Matches the semantics of the vfcnvt.rod function (Convert double-width 234// float to single-width float, rounding towards odd). Takes a double-width 235// float vector and produces a single-width float vector. Also has a mask and 238// These nodes match the semantics of the corresponding RVV vector reduction 239// instructions. They produce a vector result which is the reduction 240// performed over the second vector operand plus the first element of the 241// third vector operand. The first operand is the pass-thru operand. The 242// second operand is an unconstrained vector type, and the result, first, and 243// third operand's types are expected to be the corresponding full-width 244// LMUL=1 type for the second operand: 245// nxv8i8 = vecreduce_add nxv8i8, nxv32i8, nxv8i8 246// nxv2i32 = vecreduce_add nxv2i32, nxv8i32, nxv2i32 247// The different in types does introduce extra vsetvli instructions but 248// similarly it reduces the number of registers consumed per reduction. 249// Also has a mask and VL operand. 263// Vector binary ops with a passthru as a third operand, a mask as a fourth 264// operand, and VL as a fifth operand. 296// Averaging adds of signed integers. 298// Averaging adds of unsigned integers. 300// Rounding averaging adds of signed integers. 302// Rounding averaging adds of unsigned integers. 314// Vector unary ops with a mask as a second operand and VL as a third operand. 332// Vector FMA ops with a mask as a fourth operand and VL as a fifth operand. 338// Vector widening FMA ops with a mask as a fourth operand and VL as a fifth 345// Widening instructions with a passthru value a third operand, a mask as a 346// fourth operand, and VL as a fifth operand. 366// Widening ternary operations with a mask as the fourth operand and VL as the 372// Vector compare producing a mask. Fourth operand is input mask. Fifth 376// General vmerge node with mask, true, false, passthru, and vl operands. 377// Tail agnostic vselect can be implemented by setting passthru to undef. 380// Mask binary operators. 385// Set mask vector to all zeros or ones. 389// Matches the semantics of vrgather.vx and vrgather.vv with extra operands 390// for passthru and VL. Operands are (src, index, mask, passthru, vl). 395// Vector sign/zero extend with additional mask & VL operands. 399// vcpop.m with additional mask and VL operands. 402// vfirst.m with additional mask and VL operands. 409// Reads value of CSR. 410// The first operand is a chain pointer. The second specifies address of the 411// required CSR. Two results are produced, the read value and the new chain 414// Write value to CSR. 415// The first operand is a chain pointer, the second specifies address of the 416// required CSR and the third is the value to write. The result is the new 419// Read and write value of CSR. 420// The first operand is a chain pointer, the second specifies address of the 421// required CSR and the third is the value to write. Two results are produced, 422// the value read before the modification and the new chain pointer. 425// Branchless select operations, matching the semantics of the instructions 426// defined in Zicond or XVentanaCondOps. 430// Software guarded BRIND node. Operand 0 is the chain operand and 431// operand 1 is the target address. 433// Software guarded calls for large code model 464// To avoid stack clash, allocation is performed by block and each block is 468// RISC-V vector tuple type version of INSERT_SUBVECTOR/EXTRACT_SUBVECTOR. 472// FP to 32 bit int conversions for RV64. These are used to keep track of the 473// result being sign extended to 64 bit. These saturate out of range inputs. 508}
// namespace RISCVISD 521unsigned Intrinsic)
const override;
540unsigned OldShiftOpcode,
unsigned NewShiftOpcode,
546bool ForCodeSize)
const override;
548unsignedIndex)
const override;
556 /// Return the register type for a given MVT, ensuring vectors are treated 557 /// as a series of gpr sized integers. 559EVT VT)
const override;
561 /// Return the number of registers for a given MVT, for inline assembly 564 std::optional<MVT> RegisterVT = std::nullopt)
const override;
566 /// Return the number of registers for a given MVT, ensuring vectors are 567 /// treated as a series of gpr sized integers. 570EVT VT)
const override;
575unsigned &NumIntermediates,
576MVT &RegisterVT)
const override;
579EVT VT)
const override;
581 /// Return true if the given shuffle mask can be codegen'd directly, or if it 582 /// should be stack expanded. 586// If the pair to store is a mixture of float and int values, we will 587// save two bitwise instructions and one float-to-int instruction and 588// increase one store instruction. There is potentially a more 589// significant benefit because it avoids the float->int domain switch 590// for input value. So It is more likely a win. 594// If the pair only contains int values, we will save two bitwise 595// instructions and increase one store instruction (costing one more 596// store buffer). Since the benefit is more blurred we leave such a pair 597// out until we get testcase to prove it is a win. 603unsigned DefinedValues)
const override;
607 /// Return the cost of LMUL for linear operations. 615// Provide custom lowering hooks for some operations. 623constAPInt &DemandedElts,
624 TargetLoweringOpt &TLO)
const override;
628constAPInt &DemandedElts,
630unsignedDepth)
const override;
632constAPInt &DemandedElts,
634unsignedDepth)
const override;
637constAPInt &DemandedElts,
640unsignedDepth)
const override;
644// This method returns the name of a target specific DAG node. 662 std::pair<unsigned, const TargetRegisterClass *>
667 std::vector<SDValue> &Ops,
678EVT VT)
const override;
681bool MathUsed)
const override{
682if (VT == MVT::i8 || VT == MVT::i16)
689unsigned AddrSpace)
const override{
690// If we can replace 4 or more scalar stores, there will be a reduction 691// in instructions even after we add a vector constant load. 706// Note that one specific case requires fence insertion for an 707// AtomicCmpXchgInst but is handled via the RISCVZacasABIFix pass rather 708// than this hook due to limitations in the interface here. 717EVT VT)
const override;
726unsigned KeptBits)
const override;
730unsigned ExpansionFactor)
const override{
740 /// If a physical register, this returns the register that receives the 741 /// exception address on entry to an EH pad. 745 /// If a physical register, this returns the register that receives the 746 /// exception typeid on entry to a landing pad. 753 /// Returns the register with the specified architectural or ABI name. This 754 /// method is necessary to lower the llvm.read_register.* and 755 /// llvm.write_register.* intrinsics. Allocatable registers must be reserved 756 /// with the clang -ffixed-xX flag for access to be allowed. 760// Lower incoming arguments, copy physregs into vregs 778Type *Ty)
const override;
787SDValue ConstNode)
const override;
803 /// Returns true if the target allows unaligned memory accesses of the 808unsigned *
Fast =
nullptr)
const override;
815unsigned NumParts,
MVT PartVT, std::optional<CallingConv::ID>
CC)
820unsigned NumParts,
MVT PartVT,
EVT ValueVT,
821 std::optional<CallingConv::ID>
CC)
const override;
823// Return the value of VLMax for the given vector type (i.e. SEW and LMUL) 830// VLMAX = (VectorBits / EltSize) * LMUL 831// where LMUL = MinSize / RISCV::RVVBitsPerBlock 832// The following equations have been reordered to prevent loss of precision 833// when calculating fractional LMUL. 837// Return inclusive (low, high) bounds on the value of VLMAX for the 838// given scalable container type given known bounds on VLEN. 839static std::pair<unsigned, unsigned>
845static std::pair<unsigned, unsigned>
847unsigned InsertExtractIdx,
877// Scaled addressing not supported on indexed load/stores 881 /// If the target has a standard location for the stack protector cookie, 882 /// returns the address of that location. Otherwise, returns nullptr. 885 /// Returns whether or not generating a interleaved load/store intrinsic for 886 /// this type will be legal. 888Align Alignment,
unsigned AddrSpace,
891 /// Return true if a stride load store of the given result type and 892 /// alignment is legal. 902unsigned Factor)
const override;
905unsigned Factor)
const override;
922 /// True if stack clash protection is enabled for this functions. 936bool IsRet, CallLoweringInfo *CLI,
939template <
class NodeTy>
941bool IsExternWeak =
false)
const;
963 int64_t ExtTrueVal)
const;
1012unsigned ExtendOpc)
const;
1031bool isEligibleForTailCallOptimization(
1035 /// Generate error diagnostics if any register used by CC has been marked 1037void validateCCReservedRegs(
1041bool useRVVForFixedLengthVectorVT(
MVT VT)
const;
1043MVT getVPExplicitVectorLengthTy()
const override;
1045bool shouldExpandGetVectorLength(
EVT TripCountVT,
unsigned VF,
1046bool IsScalable)
const override;
1048 /// RVV code generation for fixed length vectors does not lower all 1049 /// BUILD_VECTORs. This makes BUILD_VECTOR legalisation a source of stores to 1050 /// merge. However, merging them creates a BUILD_VECTOR that is just as 1051 /// illegal as the original, thus leading to an infinite legalisation loop. 1052 /// NOTE: Once BUILD_VECTOR can be custom lowered for all legal vector types, 1053 /// this override can be removed. 1054bool mergeStoresAfterLegalization(
EVT VT)
const override;
1056 /// Disable normalizing 1057 /// select(N0&N1, X, Y) => select(N0, select(N1, X, Y), Y) and 1058 /// select(N0|N1, X, Y) => select(N0, select(N1, X, Y, Y)) 1059 /// RISC-V doesn't have flags so it's better to perform the and/or in a GPR. 1060bool shouldNormalizeToSelectSequence(
LLVMContext &,
EVT)
const override{
1064 /// For available scheduling models FDIV + two independent FMULs are much 1065 /// faster than two FDIVs. 1066unsigned combineRepeatedFPDivisors()
const override;
1068 SDValue BuildSDIVPow2(SDNode *
N,
const APInt &Divisor, SelectionDAG &DAG,
1069 SmallVectorImpl<SDNode *> &Created)
const override;
1071bool shouldFoldSelectWithSingleBitTest(EVT VT,
1072const APInt &AndMask)
const override;
1074unsigned getMinimumJumpTableEntries()
const override;
1076 SDValue emitFlushICache(SelectionDAG &DAG, SDValue InChain, SDValue Start,
1077 SDValue
End, SDValue Flags, SDLoc
DL)
const;
1079 std::pair<const TargetRegisterClass *, uint8_t>
1080 findRepresentativeClass(
const TargetRegisterInfo *
TRI, MVT VT)
const override;
1083namespaceRISCVVIntrinsicsTable {
1090// 0xF is not valid. See NoScalarOperand in IntrinsicsRISCV.td. 1094// 0x1F is not valid. See NoVLOperand in IntrinsicsRISCV.td. 1099using namespaceRISCV;
1101#define GET_RISCVVIntrinsicsTable_DECL 1102#include "RISCVGenSearchableTables.inc" 1103#undef GET_RISCVVIntrinsicsTable_DECL 1105}
// end namespace RISCVVIntrinsicsTable 1107}
// end namespace llvm MachineBasicBlock MachineBasicBlock::iterator DebugLoc DL
MachineBasicBlock MachineBasicBlock::iterator MBBI
Function Alias Analysis Results
Analysis containing CSE Info
static GCMetadataPrinterRegistry::Add< ErlangGCPrinter > X("erlang", "erlang-compatible garbage collector")
const HexagonInstrInfo * TII
unsigned const TargetRegisterInfo * TRI
static GCMetadataPrinterRegistry::Add< OcamlGCMetadataPrinter > Y("ocaml", "ocaml 3.10-compatible collector")
const SmallVectorImpl< MachineOperand > & Cond
This file describes how to lower LLVM code to machine code.
Class for arbitrary precision integers.
ArrayRef - Represent a constant reference to an array (0 or more elements consecutively in memory),...
An instruction that atomically checks whether a specified value is in a memory location,...
an instruction that atomically reads a memory location, combines it with another value,...
CCState - This class holds information needed while lowering arguments and return values.
This class represents a function call, abstracting a target machine's calling convention.
This is the shared class of boolean and integer constants.
This is an important base class in LLVM.
This class represents an Operation in the Expression.
A parsed version of the target data layout string in and methods for querying it.
bool hasMinSize() const
Optimize this function for minimum size (-Oz).
Common base class shared among various IRBuilders.
This is an important class for using LLVM in a threaded context.
An instruction for reading from memory.
This class is used to represent ISD::LOAD nodes.
Context object for machine code objects.
Base class for the full range of assembler expressions which are needed for parsing.
Instructions::iterator instr_iterator
Function & getFunction()
Return the LLVM function that this machine code represents.
Representation of each machine instruction.
Flags
Flags values. These may be or'd together.
This is an abstract virtual class for memory operations.
static std::pair< unsigned, unsigned > computeVLMAXBounds(MVT ContainerVT, const RISCVSubtarget &Subtarget)
static std::pair< unsigned, unsigned > decomposeSubvectorInsertExtractToSubRegs(MVT VecVT, MVT SubVecVT, unsigned InsertExtractIdx, const RISCVRegisterInfo *TRI)
InstructionCost getVRGatherVVCost(MVT VT) const
Return the cost of a vrgather.vv instruction for the type VT.
bool getIndexedAddressParts(SDNode *Op, SDValue &Base, SDValue &Offset, ISD::MemIndexedMode &AM, SelectionDAG &DAG) const
static unsigned getSubregIndexByMVT(MVT VT, unsigned Index)
Value * getIRStackGuard(IRBuilderBase &IRB) const override
If the target has a standard location for the stack protector cookie, returns the address of that loc...
bool shouldConvertFpToSat(unsigned Op, EVT FPVT, EVT VT) const override
Should we generate fp_to_si_sat and fp_to_ui_sat from type FPVT to type VT from min(max(fptoi)) satur...
bool storeOfVectorConstantIsCheap(bool IsZero, EVT MemVT, unsigned NumElem, unsigned AddrSpace) const override
Return true if it is expected to be cheaper to do a store of vector constant with the given size and ...
InlineAsm::ConstraintCode getInlineAsmMemConstraint(StringRef ConstraintCode) const override
SDValue LowerReturn(SDValue Chain, CallingConv::ID CallConv, bool IsVarArg, const SmallVectorImpl< ISD::OutputArg > &Outs, const SmallVectorImpl< SDValue > &OutVals, const SDLoc &DL, SelectionDAG &DAG) const override
This hook must be implemented to lower outgoing return values, described by the Outs array,...
unsigned getMaxSupportedInterleaveFactor() const override
Get the maximum supported factor for interleaved memory accesses.
bool shouldFoldSelectWithIdentityConstant(unsigned BinOpcode, EVT VT) const override
Return true if pulling a binary operation into a select with an identity constant is profitable.
bool mayBeEmittedAsTailCall(const CallInst *CI) const override
Return true if the target may be able emit the call instruction as a tail call.
MachineBasicBlock * EmitInstrWithCustomInserter(MachineInstr &MI, MachineBasicBlock *BB) const override
This method should be implemented by targets that mark instructions with the 'usesCustomInserter' fla...
Instruction * emitLeadingFence(IRBuilderBase &Builder, Instruction *Inst, AtomicOrdering Ord) const override
Inserts in the IR a target-specific intrinsic specifying a fence.
ISD::NodeType getExtendForAtomicOps() const override
Returns how the platform's atomic operations are extended (ZERO_EXTEND, SIGN_EXTEND,...
bool isTruncateFree(Type *SrcTy, Type *DstTy) const override
Return true if it's free to truncate a value of type FromTy to type ToTy.
bool preferZeroCompareBranch() const override
Return true if the heuristic to prefer icmp eq zero should be used in code gen prepare.
bool lowerInterleaveIntrinsicToStore(StoreInst *SI, ArrayRef< Value * > InterleaveValues) const override
Lower an interleave intrinsic to a target specific store intrinsic.
bool shouldRemoveExtendFromGSIndex(SDValue Extend, EVT DataVT) const override
Value * emitMaskedAtomicRMWIntrinsic(IRBuilderBase &Builder, AtomicRMWInst *AI, Value *AlignedAddr, Value *Incr, Value *Mask, Value *ShiftAmt, AtomicOrdering Ord) const override
Perform a masked atomicrmw using a target-specific intrinsic.
EVT getOptimalMemOpType(const MemOp &Op, const AttributeList &FuncAttributes) const override
Returns the target specific optimal type for load and store operations as a result of memset,...
bool allowsMisalignedMemoryAccesses(EVT VT, unsigned AddrSpace=0, Align Alignment=Align(1), MachineMemOperand::Flags Flags=MachineMemOperand::MONone, unsigned *Fast=nullptr) const override
Returns true if the target allows unaligned memory accesses of the specified type.
const Constant * getTargetConstantFromLoad(LoadSDNode *LD) const override
This method returns the constant pool value that will be loaded by LD.
const RISCVSubtarget & getSubtarget() const
TargetLowering::ShiftLegalizationStrategy preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N, unsigned ExpansionFactor) const override
SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override
This method will be invoked for all target nodes and for any target-independent nodes that the target...
bool isOffsetFoldingLegal(const GlobalAddressSDNode *GA) const override
Return true if folding a constant offset with the given GlobalAddress is legal.
void computeKnownBitsForTargetNode(const SDValue Op, KnownBits &Known, const APInt &DemandedElts, const SelectionDAG &DAG, unsigned Depth) const override
Determine which of the bits specified in Mask are known to be either zero or one and return them in t...
bool preferScalarizeSplat(SDNode *N) const override
const char * getTargetNodeName(unsigned Opcode) const override
This method returns the name of a target specific DAG node.
bool shouldExtendTypeInLibCall(EVT Type) const override
Returns true if arguments should be extended in lib calls.
bool isLegalAddImmediate(int64_t Imm) const override
Return true if the specified immediate is legal add immediate, that is the target has add instruction...
bool shouldSignExtendTypeInLibCall(Type *Ty, bool IsSigned) const override
Returns true if arguments should be sign-extended in lib calls.
const MCExpr * LowerCustomJumpTableEntry(const MachineJumpTableInfo *MJTI, const MachineBasicBlock *MBB, unsigned uid, MCContext &Ctx) const override
InstructionCost getVRGatherVICost(MVT VT) const
Return the cost of a vrgather.vi (or vx) instruction for the type VT.
bool shouldConvertConstantLoadToIntImm(const APInt &Imm, Type *Ty) const override
Return true if it is beneficial to convert a load of a constant to just the constant itself.
bool targetShrinkDemandedConstant(SDValue Op, const APInt &DemandedBits, const APInt &DemandedElts, TargetLoweringOpt &TLO) const override
bool shouldExpandBuildVectorWithShuffles(EVT VT, unsigned DefinedValues) const override
MVT getRegisterTypeForCallingConv(LLVMContext &Context, CallingConv::ID CC, EVT VT) const override
Return the register type for a given MVT, ensuring vectors are treated as a series of gpr sized integ...
bool decomposeMulByConstant(LLVMContext &Context, EVT VT, SDValue C) const override
Return true if it is profitable to transform an integer multiplication-by-constant into simpler opera...
bool CanLowerReturn(CallingConv::ID CallConv, MachineFunction &MF, bool IsVarArg, const SmallVectorImpl< ISD::OutputArg > &Outs, LLVMContext &Context, const Type *RetTy) const override
This hook should be implemented to check whether the return values described by the Outs array can fi...
bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM, Type *Ty, unsigned AS, Instruction *I=nullptr) const override
Return true if the addressing mode represented by AM is legal for this target, for a load/store of th...
bool hasAndNotCompare(SDValue Y) const override
Return true if the target should transform: (X & Y) == Y —> (~X & Y) == 0 (X & Y) !...
bool shouldScalarizeBinop(SDValue VecOp) const override
Try to convert an extract element of a vector binary operation into an extract element followed by a ...
bool isDesirableToCommuteWithShift(const SDNode *N, CombineLevel Level) const override
Return true if it is profitable to move this shift by a constant amount through its operand,...
bool areTwoSDNodeTargetMMOFlagsMergeable(const MemSDNode &NodeX, const MemSDNode &NodeY) const override
Return true if it is valid to merge the TargetMMOFlags in two SDNodes.
bool hasBitTest(SDValue X, SDValue Y) const override
Return true if the target has a bit-test instruction: (X & (1 << Y)) ==/!= 0 This knowledge can be us...
static unsigned computeVLMAX(unsigned VectorBits, unsigned EltSize, unsigned MinSize)
bool shouldExpandCttzElements(EVT VT) const override
Return true if the @llvm.experimental.cttz.elts intrinsic should be expanded using generic code in Se...
bool isCheapToSpeculateCtlz(Type *Ty) const override
Return true if it is cheap to speculate a call to intrinsic ctlz.
Value * emitMaskedAtomicCmpXchgIntrinsic(IRBuilderBase &Builder, AtomicCmpXchgInst *CI, Value *AlignedAddr, Value *CmpVal, Value *NewVal, Value *Mask, AtomicOrdering Ord) const override
Perform a masked cmpxchg using a target-specific intrinsic.
bool isFPImmLegal(const APFloat &Imm, EVT VT, bool ForCodeSize) const override
Returns true if the target can instruction select the specified FP immediate natively.
bool convertSelectOfConstantsToMath(EVT VT) const override
Return true if a select of constants (select Cond, C1, C2) should be transformed into simple math ops...
InstructionCost getLMULCost(MVT VT) const
Return the cost of LMUL for linear operations.
bool supportKCFIBundles() const override
Return true if the target supports kcfi operand bundles.
unsigned getJumpTableEncoding() const override
Return the entry encoding for a jump table in the current function.
bool isMulAddWithConstProfitable(SDValue AddNode, SDValue ConstNode) const override
Return true if it may be profitable to transform (mul (add x, c1), c2) -> (add (mul x,...
InstructionCost getVSlideVICost(MVT VT) const
Return the cost of a vslidedown.vi or vslideup.vi instruction for the type VT.
bool fallBackToDAGISel(const Instruction &Inst) const override
EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context, EVT VT) const override
Return the ValueType of the result of SETCC operations.
bool lowerInterleavedLoad(LoadInst *LI, ArrayRef< ShuffleVectorInst * > Shuffles, ArrayRef< unsigned > Indices, unsigned Factor) const override
Lower an interleaved load into a vlsegN intrinsic.
bool isCtpopFast(EVT VT) const override
Return true if ctpop instruction is fast.
unsigned ComputeNumSignBitsForTargetNode(SDValue Op, const APInt &DemandedElts, const SelectionDAG &DAG, unsigned Depth) const override
This method can be implemented by targets that want to expose additional information about sign bits ...
MVT getContainerForFixedLengthVector(MVT VT) const
static unsigned getRegClassIDForVecVT(MVT VT)
Register getExceptionPointerRegister(const Constant *PersonalityFn) const override
If a physical register, this returns the register that receives the exception address on entry to an ...
TargetLowering::AtomicExpansionKind shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const override
Returns how the IR-level AtomicExpand pass should expand the given AtomicRMW, if at all.
bool isExtractSubvectorCheap(EVT ResVT, EVT SrcVT, unsigned Index) const override
Return true if EXTRACT_SUBVECTOR is cheap for extracting this result type from this source type with ...
std::pair< unsigned, const TargetRegisterClass * > getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI, StringRef Constraint, MVT VT) const override
Given a physical register constraint (e.g.
MachineBasicBlock * emitDynamicProbedAlloc(MachineInstr &MI, MachineBasicBlock *MBB) const
MachineMemOperand::Flags getTargetMMOFlags(const Instruction &I) const override
This callback is used to inspect load/store instructions and add target-specific MachineMemOperand fl...
SDValue computeVLMax(MVT VecVT, const SDLoc &DL, SelectionDAG &DAG) const
bool signExtendConstant(const ConstantInt *CI) const override
Return true if this constant should be sign extended when promoting to a larger type.
bool shouldTransformSignedTruncationCheck(EVT XVT, unsigned KeptBits) const override
Should we tranform the IR-optimal check for whether given truncation down into KeptBits would be trun...
bool shouldProduceAndByConstByHoistingConstFromShiftsLHSOfAnd(SDValue X, ConstantSDNode *XC, ConstantSDNode *CC, SDValue Y, unsigned OldShiftOpcode, unsigned NewShiftOpcode, SelectionDAG &DAG) const override
Given the pattern (X & (C l>>/<< Y)) ==/!= 0 return true if it should be transformed into: ((X <</l>>...
bool hasInlineStackProbe(const MachineFunction &MF) const override
True if stack clash protection is enabled for this functions.
Register getRegisterByName(const char *RegName, LLT VT, const MachineFunction &MF) const override
Returns the register with the specified architectural or ABI name.
InstructionCost getVSlideVXCost(MVT VT) const
Return the cost of a vslidedown.vx or vslideup.vx instruction for the type VT.
SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override
This callback is invoked for operations that are unsupported by the target, which are registered to u...
static unsigned getRegClassIDForLMUL(RISCVII::VLMUL LMul)
bool isUsedByReturnOnly(SDNode *N, SDValue &Chain) const override
Return true if result of the specified node is used by a return node only.
bool softPromoteHalfType() const override
bool isFMAFasterThanFMulAndFAdd(const MachineFunction &MF, EVT VT) const override
Return true if an FMA operation is faster than a pair of fmul and fadd instructions.
TargetLowering::AtomicExpansionKind shouldExpandAtomicCmpXchgInIR(AtomicCmpXchgInst *CI) const override
Returns how the given atomic cmpxchg should be expanded by the IR-level AtomicExpand pass.
Register getExceptionSelectorRegister(const Constant *PersonalityFn) const override
If a physical register, this returns the register that receives the exception typeid on entry to a la...
bool convertSetCCLogicToBitwiseLogic(EVT VT) const override
Use bitwise logic to make pairs of compares more efficient.
unsigned getCustomCtpopCost(EVT VT, ISD::CondCode Cond) const override
Return the maximum number of "x & (x - 1)" operations that can be done instead of deferring to a cust...
void AdjustInstrPostInstrSelection(MachineInstr &MI, SDNode *Node) const override
This method should be implemented by targets that mark instructions with the 'hasPostISelHook' flag.
bool isShuffleMaskLegal(ArrayRef< int > M, EVT VT) const override
Return true if the given shuffle mask can be codegen'd directly, or if it should be stack expanded.
bool isCheapToSpeculateCttz(Type *Ty) const override
Return true if it is cheap to speculate a call to intrinsic cttz.
bool isLegalICmpImmediate(int64_t Imm) const override
Return true if the specified immediate is legal icmp immediate, that is the target has icmp instructi...
ISD::NodeType getExtendForAtomicCmpSwapArg() const override
Returns how the platform's atomic compare and swap expects its comparison value to be extended (ZERO_...
bool lowerInterleavedStore(StoreInst *SI, ShuffleVectorInst *SVI, unsigned Factor) const override
Lower an interleaved store into a vssegN intrinsic.
bool isLegalScaleForGatherScatter(uint64_t Scale, uint64_t ElemSize) const override
SDValue LowerFormalArguments(SDValue Chain, CallingConv::ID CallConv, bool IsVarArg, const SmallVectorImpl< ISD::InputArg > &Ins, const SDLoc &DL, SelectionDAG &DAG, SmallVectorImpl< SDValue > &InVals) const override
This hook must be implemented to lower the incoming (formal) arguments, described by the Ins array,...
void ReplaceNodeResults(SDNode *N, SmallVectorImpl< SDValue > &Results, SelectionDAG &DAG) const override
This callback is invoked when a node result type is illegal for the target, and the operation was reg...
bool getTgtMemIntrinsic(IntrinsicInfo &Info, const CallInst &I, MachineFunction &MF, unsigned Intrinsic) const override
Given an intrinsic, checks if on the target the intrinsic will need to map to a MemIntrinsicNode (tou...
unsigned getVectorTypeBreakdownForCallingConv(LLVMContext &Context, CallingConv::ID CC, EVT VT, EVT &IntermediateVT, unsigned &NumIntermediates, MVT &RegisterVT) const override
Certain targets such as MIPS require that some types such as vectors are always broken down into scal...
bool isLegalElementTypeForRVV(EVT ScalarTy) const
bool isVScaleKnownToBeAPowerOfTwo() const override
Return true only if vscale must be a power of two.
static RISCVII::VLMUL getLMUL(MVT VT)
int getLegalZfaFPImm(const APFloat &Imm, EVT VT) const
void LowerAsmOperandForConstraint(SDValue Op, StringRef Constraint, std::vector< SDValue > &Ops, SelectionDAG &DAG) const override
Lower the specified operand into the Ops vector.
bool splitValueIntoRegisterParts(SelectionDAG &DAG, const SDLoc &DL, SDValue Val, SDValue *Parts, unsigned NumParts, MVT PartVT, std::optional< CallingConv::ID > CC) const override
Target-specific splitting of values into parts that fit a register storing a legal type.
Instruction * emitTrailingFence(IRBuilderBase &Builder, Instruction *Inst, AtomicOrdering Ord) const override
unsigned getNumRegistersForCallingConv(LLVMContext &Context, CallingConv::ID CC, EVT VT) const override
Return the number of registers for a given MVT, ensuring vectors are treated as a series of gpr sized...
ConstraintType getConstraintType(StringRef Constraint) const override
getConstraintType - Given a constraint letter, return the type of constraint it is for this target.
bool shouldConsiderGEPOffsetSplit() const override
MachineInstr * EmitKCFICheck(MachineBasicBlock &MBB, MachineBasicBlock::instr_iterator &MBBI, const TargetInstrInfo *TII) const override
bool isLegalInterleavedAccessType(VectorType *VTy, unsigned Factor, Align Alignment, unsigned AddrSpace, const DataLayout &) const
Returns whether or not generating a interleaved load/store intrinsic for this type will be legal.
bool canCreateUndefOrPoisonForTargetNode(SDValue Op, const APInt &DemandedElts, const SelectionDAG &DAG, bool PoisonOnly, bool ConsiderFlags, unsigned Depth) const override
Return true if Op can create undef or poison from non-undef & non-poison operands.
bool isIntDivCheap(EVT VT, AttributeList Attr) const override
Return true if integer divide is usually cheaper than a sequence of several shifts,...
SDValue expandIndirectJTBranch(const SDLoc &dl, SDValue Value, SDValue Addr, int JTI, SelectionDAG &DAG) const override
Expands target specific indirect branch for the case of JumpTable expansion.
bool lowerDeinterleaveIntrinsicToLoad(LoadInst *LI, ArrayRef< Value * > DeinterleaveValues) const override
Lower a deinterleave intrinsic to a target specific load intrinsic.
unsigned getNumRegisters(LLVMContext &Context, EVT VT, std::optional< MVT > RegisterVT=std::nullopt) const override
Return the number of registers for a given MVT, for inline assembly.
bool isMultiStoresCheaperThanBitsMerge(EVT LTy, EVT HTy) const override
Return true if it is cheaper to split the store of a merged int val from a pair of smaller values int...
bool getPostIndexedAddressParts(SDNode *N, SDNode *Op, SDValue &Base, SDValue &Offset, ISD::MemIndexedMode &AM, SelectionDAG &DAG) const override
Returns true by value, base pointer and offset pointer and addressing mode by reference if this node ...
bool shouldFormOverflowOp(unsigned Opcode, EVT VT, bool MathUsed) const override
Try to convert math with an overflow comparison into the corresponding DAG node operation.
bool getPreIndexedAddressParts(SDNode *N, SDValue &Base, SDValue &Offset, ISD::MemIndexedMode &AM, SelectionDAG &DAG) const override
Returns true by value, base pointer and offset pointer and addressing mode by reference if the node's...
SDValue joinRegisterPartsIntoValue(SelectionDAG &DAG, const SDLoc &DL, const SDValue *Parts, unsigned NumParts, MVT PartVT, EVT ValueVT, std::optional< CallingConv::ID > CC) const override
Target-specific combining of register parts into its original value.
bool isMaskAndCmp0FoldingBeneficial(const Instruction &AndI) const override
Return if the target supports combining a chain like:
bool isSExtCheaperThanZExt(EVT SrcVT, EVT DstVT) const override
Return true if sign-extension from FromTy to ToTy is cheaper than zero-extension.
bool isLegalStridedLoadStore(EVT DataType, Align Alignment) const
Return true if a stride load store of the given result type and alignment is legal.
SDValue LowerCall(TargetLowering::CallLoweringInfo &CLI, SmallVectorImpl< SDValue > &InVals) const override
This hook must be implemented to lower calls into the specified DAG.
bool isZExtFree(SDValue Val, EVT VT2) const override
Return true if zero-extending the specific node Val to type VT2 is free (either because it's implicit...
unsigned getStackProbeSize(const MachineFunction &MF, Align StackAlign) const
bool shouldInsertFencesForAtomic(const Instruction *I) const override
Whether AtomicExpandPass should automatically insert fences and reduce ordering for this atomic.
Wrapper class representing virtual and physical registers.
Wrapper class for IR location info (IR ordering and DebugLoc) to be passed into SDNode creation funct...
Represents one node in the SelectionDAG.
Unlike LLVM values, Selection DAG nodes may return multiple values as the result of a computation.
This is used to represent a portion of an LLVM function in a low-level Data Dependence DAG representa...
MachineFunction & getMachineFunction() const
This instruction constructs a fixed permutation of two input vectors.
This class consists of common code factored out of the SmallVector class to reduce code duplication b...
This is a 'vector' (really, a variable-sized array), optimized for the case when the array is small.
An instruction for storing to memory.
StringRef - Represent a constant reference to a string, i.e.
TargetInstrInfo - Interface to description of machine instruction set.
virtual bool shouldFormOverflowOp(unsigned Opcode, EVT VT, bool MathUsed) const
Try to convert math with an overflow comparison into the corresponding DAG node operation.
ShiftLegalizationStrategy
Return the preferred strategy to legalize tihs SHIFT instruction, with ExpansionFactor being the recu...
virtual ShiftLegalizationStrategy preferredShiftLegalizationStrategy(SelectionDAG &DAG, SDNode *N, unsigned ExpansionFactor) const
AtomicExpansionKind
Enum that specifies what an atomic load/AtomicRMWInst is expanded to, if at all.
This class defines information used to lower LLVM code to legal SelectionDAG operators that the targe...
Primary interface to the complete machine description for the target machine.
TargetRegisterInfo base class - We assume that the target defines a static array of TargetRegisterDes...
The instances of the Type class are immutable: once they are created, they are never changed.
LLVM Value Representation.
Base class of all SIMD vector types.
@ Fast
Attempts to make calls as fast as possible (e.g.
@ C
The default llvm calling convention, compatible with C.
NodeType
ISD::NodeType enum - This enum defines the target-independent operators for a SelectionDAG.
@ BUILTIN_OP_END
BUILTIN_OP_END - This must be the last enum value in this list.
@ SIGN_EXTEND
Conversion operators.
MemIndexedMode
MemIndexedMode enum - This enum defines the load / store indexed addressing modes.
CondCode
ISD::CondCode enum - These are ordered carefully to make the bitfields below work out,...
@ SplitF64
Turns a f64 into a pair of i32s.
@ SELECT_CC
Select with condition operator - This selects between a true value and a false value (ops #3 and #4) ...
@ STRICT_VFCVT_RTZ_XU_F_VL
@ TRUNCATE_VECTOR_VL_USAT
@ BuildPairF64
Turns a pair of i32s into an f64.
@ BuildGPRPair
Turn a pair of i<xlen>s into an even-odd register pair (untyped).
@ STRICT_VFROUND_NOEXCEPT_VL
@ SPLAT_VECTOR_SPLIT_I64_VL
@ SplitGPRPair
Turn an even-odd register pair (untyped) into a pair of i<xlen>s.
@ TRUNCATE_VECTOR_VL_SSAT
@ STRICT_VFCVT_RTZ_X_F_VL
static constexpr unsigned RVVBitsPerBlock
This is an optimization pass for GlobalISel generic memory operations.
AtomicOrdering
Atomic ordering for LLVM's memory model.
bool RISCVCCAssignFn(unsigned ValNo, MVT ValVT, MVT LocVT, CCValAssign::LocInfo LocInfo, ISD::ArgFlagsTy ArgFlags, CCState &State, bool IsFixed, bool IsRet, Type *OrigTy)
RISCVCCAssignFn - This target-specific function extends the default CCValAssign with additional infor...
This struct is a compact representation of a valid (non-zero power of two) alignment.
bool isFloatingPoint() const
Return true if this is a FP or a vector FP type.
bool isScalarInteger() const
Return true if this is an integer, but not a vector.
bool isInteger() const
Return true if this is an integer or a vector integer type.
bool hasVLOperand() const
bool hasScalarOperand() const
This structure contains all information that is necessary for lowering calls.
