Just like with any other troubleshooting situation, it often is very beneficial to think from the perspective of what is actually happening in the service under the hood. I proceeded to correct the storage account link in our git and voilà, everything started working. Somehow the configuration for a storage account linked service had changed to an invalid state, and when a dataset dependent on that linked service was tried to be created by the ARM provider, the failure happened. I verified this exact reference resource in the ADF portal with the "test connection" button, and it indeed was broken. Looking into the error message itself, we find something quite interesting:Īnd there we finally had the culprit. As our deployment ended up in a failure, ADF did not reach the step it tries to delete the logs in. It seems that ADF creates the deployment, and you can see it in the history view while it runs, but if it succeeds it also gets deleted directly after. I had to think about this for a while, but then I figured out where to start: ARM template deployments should always leave a deployment history log! So what do you know, we had some clearly identifiable logs in the resource group. How can the states be desynchronized if there is no initial state on the factory? As soon as we clicked publish, some resources were deployed inside the ADF, but we ended up with the same error message. Not necessarily a problem considering all of your logic should already be in the Git repo instead of tied to the factory at all. The solution for usĪfter following the official instructions (which already is almost the nuclear option) without luck, we decided to just delete our data factory instance completely. And judging from multiple github issues found by googling, many others are facing similar results. Well, unfortunately this did not fix the issue for us. After you create a PR towards your collaboration branch and publishing again, in theory you should have the desynched state included in your master configuration. There is a decent idea behind this because when you are first enabling the integration, you get the option to import the current setup of the ADF instance as a new branch in your repository. In the official documentation, the troubleshooting steps for the git integration are pretty much just "remove and redo". So the collaboration branch should be the source of truth, but chances are that someone could have for example deployed some old ARM version of the contents, causing the desynchronization of the branch and the ADF config. Read more about that in my previous post here. It also generates a publish branch in the git, which contains those templates. The data factory actually generates ARM templates for the contents automatically when you hit publish from the git view. To make the changes live you will need to make a publish from the "collaboration branch". The git integration view does NOT show the current status of your ADF, but instead shows the status of the git.The primary mode in Author view "Data Factory" can no longer publish changes to the instance.You get a secondary "mode" in the Author view for the integration.How this actually works in the background is that when you set it up, three major things happen in your data factory instance: What does the message actually say?Īs is evident by the error message, we are using the Microsoft-recommended git-integration with Azure Data Factory v2. This is likely due to publishing outside of Git mode." during our publish in dev. During this change, we ran into an error message "The publish branch is out of sync with the collaboration branch. We recently changed some of our application resource structures and thus created a new data factory resource.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |