Collectives™ on Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives 0votes
0answers
30views
Azure Data Flow - parse transformation of XML element with both attribute and value
Given XML with element that has both attribute and value, e.g.<root><elem attr="world">hello</elem></root>how can I parse both the attribute and the value in ...
- 181
0votes
0answers
37views
Azure Synapse spark pool throwing hostname not trusted error
I'm trying to read from Azure kusto source using below code in Azure synapse notebook.df = spark.read \.format("com.microsoft.kusto.spark.synapse.datasource") \.option("kustoCluster&...
- 221
0votes
1answer
220views
REST API with Post Method as source in ADF Data flow
I am Trying to read data from an API source. Which works fine in Postman. Since I am not allowed to post images directly sharing them as link.Postman_outputBut the same fails when I try to read the ...
0votes
1answer
164views
Dataflow in Synapse pipeline is not reading CSV file correctly
I have a CSV file and it looks like below: (Column Delimiter : +| and row delimiter : &|\n)This file has two problems :1st - Both row and column delimiter is 2 characters, and this does not ...
- 1,304
0votes
1answer
237views
Synapse notebook script runs fine but stops on time-out and stucks queued in pipelines
I have a python script in a notebook that performs a parquet file schema correction. It is working fine, runs in less than 10 seconds, depending on the number of files to process. Not exactly fine, ...
- 1,175
0votes
1answer
154views
Dynamic Mapping in ADF Copy Activity Not Retrieving OData Formatted Values
Setup:Source: API dataset (OData API link).Additional Headers: Prefer = odata.include-annotations="*" (to retrieve lookup values).**API Response Example: Manual Mapping (Works Fine)...
0votes
0answers
41views
How connect and process on-prem HDFS data from Azure Synapse
I want to write data from an Azure Synapse notebook to an on-prem HDFS file system. While Azure Synapse provides Copy Activity for reading data, there doesn't seem to be a direct way to write data ...
0votes
1answer
401views
Azure Synapse time out - Token expire
Backstory: I have 12 zip files in Gen2 storage, each around 300 mb.I am running a notebook within a pipeline job.It goes smoothly until extracting the 6th zip file using pd.read_csv(compression ='...
- 21
0votes
1answer
173views
Azure Data Flow flatten and parse key/value column for complex JSON
I'm trying to transform a key/value data into column using Azure synapse Data Flow. Basically this:"scoreIndeces": {"66": 0.2,"67": 0.3,"68": 0.4,"70&...
0votes
1answer
213views
Get Max date value from append variable array - azure synapse pipeline
In my Synapse pipeline, I am looping through a set of filenames. Inside the loop, I use a "Get Metadata" activity to retrieve the last modified date of each file and append these dates to an ...
0votes
1answer
168views
Synapse how to copy pipelines between branches?
We have 3 synapse workspaces (dev, test, maint) connects to it's own branch in the same Azure DevOPs repository. We also have feature branches based off the dev branch. All our development is done ...
0votes
1answer
384views
How to run an Azure Machine Learning pipeline from Synapse?
I have a working v2 pipeline in Azure Machine Learning that I want to run from Synapse. However, I cannot select my pipelines from the dropdown menu.I have a working linked service and a non-self-...
- 273
0votes
1answer
198views
Azure Synapse pipeline with dataflows failing randomly
I am having issues with a series of pipelines that build our data platform Spark databases hosted in Azure Synapse.The pipelines host dataflows which have 'recreate table' enabled. The dataflows ...
- 334
0votes
1answer
137views
How to work with string properties in full fidelity analytical store schema in CDC pipelines?
I have a CosmosDB account using MongoDB API and am trying to work with the change data capture output of this.The problem I am seeing specifically is that full fidelity string properties appear to ...
- 457k
0votes
1answer
379views
How to parametrize the KeyVault secret field in a Linked Service, using functions and LS parameters?
EDIT: Pipeline executes fine and the behavior is observed only when testing connection...I am trying to parametrize the Secret Name value in a linked service. When using a LS parameter, this works ...
- 86
- The Overflow Blog
- Featured on Meta
Hot Network Questions
- Find the value of y in this geometric figure
- A subset of a rectangle that 'blocks' every curve that goes from right to left must connect upper and lower sides
- Permanence of the basic forces
- Manga involving the MC playing a computer game connected to a medieval fantasy world
- About \def parameter list
- Why is the verb "to sic" conjugated with a double c rather than the more typical ck?
- If photons are the carriers of the electromagnetic force, why isn't electricity made of photons?
- Cutoff times for Seoul City Air Terminal check-in
- Visualizing Different Coordinate Representations of Complex Numbers
- Old movie about the child Jesus, who is seeing his own future as he walks through Jerusalem
- Did around 50% of sailors die from scurvy on long voyages?
- Bridge Edge Loops not working between 2 clearly defined Edge Loops
- What is the meaning/utility of __[] AKA BlankSequence[][]?
- What are the events in Parliament Square and Sieges Allee that Woolf refers to in A Room Of One's Own?
- How to find parent ways of nodes
- Testing binary data for statistical independence
- Why did I freeze to death even though I had cold resistance?
- Proverb for when one is already in a hurry but the condition wastes even more time
- What is the point of All In?
- How to flip or mirror keyframes horizontally on the X-axis in Graph Editor?
- What are the correct lyrics to the song "Lola", by The Kinks?
- Why do different implementations produce different DER signatures for the same private key and hash?
- Do customizable blank-slate protagonists limit the narrative potential of action RPGs?
- constexpr self-referencing struct