More Videos

  • Getting to Know Swift Package Manager

    The Swift Package Manager makes it possible to easily develop and distribute source code in the Swift ecosystem. Learn about its goals, design, unique features, and the opportunities it has for continued evolution.

    Resources

    Related Videos

    WWDC19

  • Hi, welcome to Getting toKnow Swift Package Manager.I'm Rick Ballard.With me today is Boris Buegling.And we're very excited to tellyou all about Swift's PackageManager, or as we call itsometimes SwiftPM.We're going to focus today onthe open source project.And not on Apple's otherdeveloper tools.But we've got plenty to tell youabout here today.

    The Swift Package Manager makesit easier to develop anddistribute source code in theSwift ecosystem.Today we're going to talk aboutits goals, some of its design,and a little bit about where wecan go from here.

    We'll start by telling you alittle bit about why we decidedto create a new Package Manageras part of the Swift project.We'll show you little bit abouthow to use it.And then dive into its designand features.

    We'll tell you a little bitabout where we can go in thefuture and close out bydescribing SwiftPM's open sourceprocess and how you can getinvolved if you're interested.

    I'm sure most of you arefamiliar with package managersthey're a great way to share andreuse code.But why did we decide to createa new one for Swift?First of all, Swift is across-platform language.So we wanted a greatcross-platform tool for buildingyour Swift code.This makes it easy to configureyour code in a consistent wayand run it on all of Swift'ssupported platforms.SwiftPM includes its owncomplete build system, allowingyou to configure your software,build it, test it, and even runit from one tool.

    We also wanted to make it aseasy as possible for you toshare your Swift libraries withanyone wherever they are byproviding a canonical packagemanager in the Swift project, wehope to define a common standardfor the way that you candistribute your libraries.This makes it easy to grow theSwift ecosystem and make Swiftbetter for everyone.Many of you may have great ideasfor features that you'd like toadd.But we'd like to be carefulabout what is added to the corelibraries so we can maintain acareful and curated API.A great package manager makes iteasy to distribute these ideasas packages instead of having toput them into the core library.The best ideas can organicallygain traction with the communityover time and becomeincreasingly standardized.

    Finally, by building a packagemanager alongside Swift, we'reable to take advantage ofSwift's power and philosophy.SwiftPM is itself written inSwift and it's even a Swiftpackage.Beyond that, we have theopportunity to work closely withthe Swift language and corelibrary projects to build greatpackage manager features thatwill help your Swift code sing.

    SwiftPM is part of the Swiftopen source project, and has itsown presence on Swift.org, andon GitHub.The Swift Package Managersection of Swift.org is a greatplace to go to get started.

    When you're ready to try it outyou can find it included inevery Swift toolchain, alsodownloadable from Swift.org.And of course, it's included inevery release of Xcode alongsidethe Swift tools.

    So, to start telling you alittle bit about how to use it,I'd like to invite BorisBuegling up to show you thebasics.Thanks, Rick.Let's take a look at how to useSwiftPM.

    SwiftPM consists of four commandline tools, and at the top levelSwift Command.Swift Build, to build yourpackage.Swift Run to run its executableproducts.Swift Test to run tests.And Swift Package to run variousnon-build operations on thepackage.Packages are stored in gitrepositories.And diversions are representedby git tags.

    Next, I'm going to show you ademo of how easy it is to createyour own first Swift Package.

    We start in terminal, and wecreate a new directory, calledhelloworld.This will also be the name ofour package.Switch to that directory and wewill run Swift Package init withthe type executable.With this, SwiftPM creates abasic package and structure forus.Let's open finder to look at ita little bit more closely.

    We have the Package.swiftmanifest file, which describesthe structure of the package.We get a basic README.You have the Sources directorywith a subfolder for ourhelloworld target.And the main.swift file for ourexecutable.You also get a test directory,where we could later put someunit tests.Let's switch back to terminal.And we will type swift run tobuild and run this package.This compiles the package, linksexecutable, and we seehelloworld is the output.Next, I'm going to switch toanother terminal window, whereI've prepared a more complexpackage. We will use this in thefollowing to discuss the basicconcepts of SwiftPM.But, first, let's also just runit to see what it does.So, you can see, it outputsrandomly generated playing cardsto the terminal.

    Now, we can switch back to theslides to talk about SwiftPM'sbasic concepts.

    A package consists of threemajor parts; dependencies,targets and products.And we'll look into each ofthese in more detail in thefollowing.

    Dependencies are those Swiftpackages that you can use whendeveloping your features.Each dependency provides one ormore products such as librariesthat your package can use.Let's take a look at howdependencies look in the packageof Swift manifest file.Each dependency has a sourcelocation and it is versioned.

    Targets are the basic buildingblocks of packages.

    A target describes how to builda set of source files intoeither a module or a test suite.

    Targets can depend on othertargets of the same package andon products exported from otherpackages, declared asdependencies.

    Products are executable tolibraries and products areassembled from the buildartifacts of one or more target.

    Packages provide libraries forother packages by definingproducts.By default, you do not have todeclare the type of libraryexplicitly, but SwiftPM willchoose it for you based on itsuse.If needed, you can explicitlydeclare a library a static ordynamic.Let's take a look how ourtargets are configured in themanifest.In our example, we have threetargets.

    The first is called libdealer,and it contains theimplementation of our mainfunctionality.And it has one dependency, thedeck of playing cards productwhich comes from the dependencywe declared earlier.

    Second target, dealer depends onthat to provide the command linetool that we just run earlier.And finally, we also have a testtarget that depends on the twoother targets and this is wherewe can unit test ourfunctionality.

    In our example package, we havealso configured two products.

    The first is a library productcorresponding to the libdealertarget.And this provides ourimplementation as a library forexternal consumption.Second, we have an executabletarget depending on the dealertarget which provides anexecutable for command line use.

    To complete this section, I willshow you how we can use apackage to add a new feature tothe example.

    This, we switch to a newterminal window and we open upthe package.swift manifest file.We want to add a new dependency.In this case, this is actuallySwiftPM itself.As Rick told you, it is itself,its own Swift Package.It doesn't provide a stable API,though, that is why we'redepending on an exact versionnumber.

    We also want to depend on one ofits products in the libdealertarget.It is called utility.And among other things, it has aclass called terminalcontroller, which provides uswith the possibility to colorthe terminal output.Note however, that this is noofficial Apple API, we're justusing it for the demo.Let's switch back to theterminal.

    I already changed the code tomake use of the new dependencybefore this demo.So, we can just run it to seethe result.And as you can see, we have thesame output, but now it's alittle bit more fun with somecolors.

    I want to show you one last demowhich is how SwiftPM can runtests.For this, we're using the SwiftNeo package.A networking library that Appleopen source earlier in thespring.We will run Swift Test with aparallel option.

    This allows us to run tests inparallel. So, you get your test resultsfaster.And we also pass the filteroption.

    This allows you to run a subsetof tests to you can iterate on asingle future.

    This will now again compile ourpackage and run the tests injust a few seconds.

    And as you can see, we get anice progress bar and the testsfinish really fast, because wewere running them in parallel.

    Let's switch back to the slidesagain.

    Next, I'm going to talk to youabout the design of the SwiftPackage Manager.

    SwiftPM follows Swift'sphilosophy.It is fast, safe, andexpressive.It is safe due to its isolatedbuild environment and the factthe builds cannot run arbitrarycommands.It is fast due to using a buildengine that is scalable to largedependency graphs.And it's expressive due to usingthe Swift language for thepackage manifest.And this also allows you to usea programming language you'realready familiar with.For the rest of the section, Iwill take you on a journeythrough the different steps youwill encounter when creatingyour own Swift packages.We will start withconfiguration.

    As we saw earlier, SwiftPM'smanifest is based on Swift.Using Swift makes it easy tounderstand because there is nonew language to learn.We follow Swift's API designguidelines to make it even morefamiliar.And, it allows us to takeadvantage of existing toolingwritten for Swift, but whenwriting your own manifest, youshould still prefer declarativesyntax and avoid side effects.

    Because SwiftPM makes noguarantees about when or howoften your source code is beingevaluated.On the left-hand side here, yousee an example that is notreally declarative.We cannot see the name that isbeing generated and it's used ina couple of times across thepackage.In contrast, on the right-handside, we have a fullydeclarative manifest by usingstring constants.It is easy to understand and seewhat the targets are.

    So, as you can see, not usingdeclarative syntax also makesyour manifest harder tounderstand for you and yourusers.

    Source files organized on disksin folders named after eachtarget in the package.This makes it easy to getstarted and allows packages toadopt a common structure, sothat you can navigate themquickly.

    Package Managers and other buildtools often have attentionbetween what is explicitlyconfigured by the user and theconventions that are imposed bythe Package Manager.

    As I told you earlier, thesource file is automaticallypicked up from convention baselocations on disk, so that youcan very easily add or removesource files without having toedit the package manifest.

    Products and targets, however,are worth explicitly configuringto make it easier to understandthe package and what it defineswithout needing to crossreference with the layout ondisk.

    It also makes it easy forclients to see what a packageprovides just by looking at themanifest.

    SwiftPM also supports buildingsource code for other programmanager languages, such as C,C++, and Objective-C.This allows integration withexisting code.

    Note however that we do notsupport mixing those languageswith Swift in the same target.Next, we're going to look atdependencies and versioning.

    To make sure your packages canbenefit from bug fixes withoutconstant churn, Swift packagesshould adhere to semanticversioning.

    This is a commonly used standardwhich assign specific semanticmeaning to each of a versionnumber's components.

    The major version signifiesbreaking changes which requiredclients to update their code.

    Examples for the changes couldbe deleting an existing type,deleting a message, or changingits signature.But they also include backwardsincompatible bug fixes, or majorchanges to the behavior ofexisting API.The minor version should beupdated if functionality isadded in a backwards compatiblemanner.Examples for this is adding anew method or type.And finally, the patch versionshould be increased when you'remaking backwards compatible bugfixes.

    This allows clients to benefitfrom bug fixes without riskingbreaking the source code.SwiftPM needs to determine theexact versions of all packagesin the package graph before itis ready to build.We do this with a process calleddependency resolution.As part of this, SwiftPM looksat all the requirementsspecified for packages and findsthe latest version that iscompatible with all of them.Let's take a closer look at whatSwiftPM is doing in the processusing the demo I showed youbefore.

    The dealer package from the demohas two direct dependencies.One, is SwiftPM itself, and theother one is deck of playingcards.SwiftPM will resolve theversions of these directdependencies.For the first one, this isstraightforward because wespecified an exact version,beginning exactly that tag.For the second one, we're usingthe from syntax.That means we're getting updatesto the minor or patchcomponents.In this case, we're ending upwith a tag 3.1.4.The whole process is recursive.So next SwiftPM will look at thetransitive dependencies of alldiary points.So, PM has no furtherdependencies, so there's nothingto do there.But a deck of playing cardsdepends on the Fisher-Yates andplaying card packages.Next, SwiftPM has to resolve theversions of these packagesagain.For the Fisher-Yates package,this works the same way asbefore because we are also usingthe from syntax.In this case, we're ending upwith a tag, 2.2.5.For the playing card package,we're using up to the next minorsyntax.This means, we're getting onlyupdates to the patch component.

    You might want to use thatsyntax if you want to be moreconservative as a dependency andonly take bug fixes.In this case, we're ending upwith a tag 3.0.2.Finally, when looking at atarget, SwiftPM has to match upits required products with thepackages that we resolved.For this, we're looking at thedealer target from the demo, andas you can see, the utilityproduct is provided by theSwiftPM package.

    And the rest of the packagesprovide the other products.After dependency resolution, theresolves are recorded in a filecalled package.resolved.the purpose of this file is sothat you can share the resolveversions with other members ofyour team, or your continuousintegration infrastructure, sothat you get dependable buildresults, and you candeliberately choose when youwant to update a dependency.You do so by running SwiftPackage Update, when you'reready to update.Note also, that this is yourtop-level package that containspackage.resolved.If one of the transitivedependencies contains apackage.resolve file, it will beignored for dependencyresolution.

    Next, let's take a look atbuilding your package.SwiftPM uses llbuild as itsunderlying build engine.llbuild is a set of librariesfor building build systems.And it's built around a generalpurpose and reusable buildengine. This provides us with ability todo fast as well as correctincremental builds.And is also used by Xcode's newbuild system.It is also the part of the Swiftopen source project.

    Developing software in isolationwith all dependencies explicitlydeclared ensures that evenpackages with complexrequirements can be reliablybuilt and used in differentenvironments.

    Instead of installing packagesglobally into the system,SwiftPM only allows you to usepackages that you explicitlydepend on.

    We also leverage buildsandboxing so that nothing canwrite to arbitrary locations onthe file system during thebuild.

    SwiftPM does not allow executingarbitrary commands, or shellscripts, as part of the build.

    This allows us to fullyunderstand your build graph andall of its inputs and outputs todo fast, as well as correct,incremental builds.Because we have a view of allyour dependencies.As I showed you in the demobefore, SwiftPM also supportstesting.This is based on the XCTestframework that you're alreadyfamiliar with.We support parallel testing, sothat you can get your testresults faster.

    And we support test filtering sothat you can run a subset oftests and iterate on a singlefeature.As we're evolving SwiftPM, we'rethinking about workflowfeatures, especially so that youcan do all of your developmenton the command line.One such feature is edit mode,which allows overwriting alltransitive occurrences of aspecific package, with a locallychecked out copy so thattemporary edits can be made, andchanges to transitivedependencies can be testedwithout having to forward allpackages in the graph upfront.

    Branch dependencies allowdepending on packages withoutstrict versioning requirements.This is useful when you'redeveloping multiple packagestogether.

    This is a development onlyfeature, so you have to changeto certain version dependenciesbefore you publish a tag.Local packages allow you to usepackages directly from the filesystem, instead of from a gitrepository.This is useful to make itpossible to bring up multiplepackages during the initialcreation.

    So, last topic, I want to talkto you about adopting newversions of SwiftPM and theSwift language.Each new version of Swift canbring a new version of thepackage.swift manifest API.The previous API is stillavailable, so that you can takeadvantage of new source toolswithout having to update yourpackage or losing access toexisting packages.

    New API can be adoptedindependently of changing to anew Swift language version foryour packages' source code.To specify which version of theAPI is being used, we're usingthe Swift Tools Version commandat the top of the package.swiftmanifest file.This specifies the minimumrequired version of the Swifttools that is needed to processthe given manifest.Each package can also declarewhich versions of the Swiftlanguage it uses for compilingits source code.This is a list, so you canchoose to support multipleversions of Swift with the sameversion of your package, byusing compiler directives.A package graph can be amix-and-match of packages withdifferent language versions.

    I told you a lot about howSwiftPM works today, next I'dlike to invite Rick back up tothe stage to tell you where wecan go from here.Thanks, Boris.

    So, Boris has shown you what youcan do today, but there's a lotmore potential out there.SwiftPM is still a young projectwith lots of room to grow.

    Swift uses an open evolutionprocess which means that anyone,including you, can contributeyour ideas.If you're looking for someinspiration, we'd like to sharesome of our ideas, but none ofthis is a plan of record.We're sharing these ideas sothat you can see the potentialof the Swift Package Manager.And we welcome you to provideyour feedback, comments, andyour own ideas as we evolve isthis product.

    The ideas I'm going to covertoday break down into fourdifferent themes.

    Letting the Swift PackageManager integrate with othertools that may want to sit ontop of it. Helping you publish new versionsof your package and deploy theirproducts.Supporting more complex packagesthan SwiftPM is able to buildtoday. And finally, some forwardlooking thoughts on packagediscovery and trust.While the SwiftPM command lineexperience is important, we wantto make sure that SwiftPM canintegrate with other tools, suchas development environments,automation, and more.

    We've already laid thegroundwork for this withSwiftPM's library-basedarchitecture.SwiftPM doesn't have a stableAPI today, but for tools thatare willing to keep up with theSwiftPM changes, it's availablefor adoption and additionstoday.

    If you're looking to buildsupport for SwiftPM into yourdeveloper tools, we welcome yourcontributions and discussion.We want to make SwiftPM part ofa thriving ecosystem ofdeveloper tools.One thing we've seen requestedrecently on the Swift forums isa way for people to edit theirpackage.swift manifest fromautomated tools, instead ofmaking their users always editthe source code directly.We think that it's possible forSwiftPM to support this,probably by using libSyntax.libSyntax is a library beingdeveloped in the Swift opensource project that makes iteasier for you to understand andmanipulate Swift syntax fromother tools.

    Boris told you earlier that youshould prefer declarative syntaxfor your package.swift manifest,and this is another reason why.That will make it much easierfor SwiftPM to understand yourmanifest, so that it can makeautomatic changes, such asadding new dependencies ortargets as shown here.So, there's a lot of room forSwiftPM also to add newfunctionality to help youpublish new versions of yourpackages and deploy theirproducts.

    Today, when you want to publisha new version of your package,you tag it manually with git.And if you want to inspect yourpublished tags, you use gitdirectly for that as well.

    We could add new functionalityto automate this process andperform additional housekeeping,validation, and other auxiliarytasks you might want as part ofa streamlined publishingworkflow.

    One especially useful feature wecould add here would beassistance with maintainingcorrect semantic versioning.We could have SwiftPM analyzethe API differences in the newversion of your package anddetect when you've made a changethat is not compatible atcompile time, so that it cansuggest updating the majorversion of your package.

    Another thing we could do ismake it easier to deploy theproducts of your packages fromSwiftPM. You may want to customize thelinkage with libraries, or theproduct layout for your specificdeployment environment, whetherlocal or on a server.Or, maybe you want to includeversion information about whatpackages were built into theproduct.Or, you otherwise want to usethe context that SwiftPM hasabout your packages somewhere inyour product.SwiftPM could add new commandsto support all of these needs.

    There's a lot that you can buildwith SwiftPM today, but we alsowant to be able to support morecomplex packages with moresophisticated needs.

    The biggest gap we have rightnow, is probably support forresources.If you have any images, datafiles, or other assets, SwiftPMcurrently provides no way tobundle these up you're yourproducts.

    The foundation core library,actually just added API thisspring for doing resources in across-platform manner so SwiftPMcould adopt this API if we wantto build this feature.

    We know that some users alsowant support for specifyingcompiler flags, linker flags,and other properties thatSwiftPM doesn't support today.It would be really great for usto add a robust build settingsmodel, potentially includingthings like conditionalsettings, or fine-grain controlover what parts of the packageget which setting values.

    Boris also talked to you earlierabout SwiftPM build isolation,and why it's important.

    We don't let you run arbitraryshell scripts.But many users may want somelevel of customization for theirbuild, either because they wantto support custom languages, orprocessors, they want to runtheir own documentationgenerator implementor, or theyhave other steps that they needto bring to the build process.We think that SwiftPM couldsupport this safely, possiblyeven through real tools packagesthat bring new tools into yourbuild process.The important thing we need tomake sure here, if we do such afeature, is that any new toolthat's brought into the buildprocess have to correctlydeclare their input and outputdependencies, so SwiftPM cancontinue to maintain correctincremental, and parallelizablebuilds.

    Finally, I want to talk a littlebit about some forward-lookingthoughts on package discovery,trust, and management.Git itself supports, theprotocols that get supports,provides security mechanismslike TLS to make sure thatyou're actually talking to theremote repository that you thinkyou are.But a malicious actor couldstill compromise remoterepository and put maliciouscontent in.This is actually somethinganytime you're using third-partycode, you should be aware ofthese sorts of risks.But the Swift Package Managerprovides a great opportunity forus to build security features tomake sure that you're actuallygetting the package content thatyou expected.

    SwiftPM also prevents yourpackage.swift manifestevaluation in your build fromescaping and writing things outinto your file system oraccessing the network.We're using macOS' sandboxingtechnology for this today.And it's great.But we'd like to bring this kindof security to other platformsas well.

    Many users may want to be ableto fork their packages easily,either because they want to makea private customization to oneof the packages in their graph.Or, even because they just wantto override the origin URL ofwhere they're getting each ofthose packages from, so thatthey can point at a privatemirror that they control and notdepend on the original packagealways being there.Ultimately, I'd like some dayfor us to have a real index forSwift packages.In addition to providing astandardized namespace andmaking it easier to discover newpackages, we could even supportthings like quality metrics fora package.Like what is its automated testcoverage?Or ways for you to evaluate thetrustworthiness of a new packagethat you're considering adopting.So, I've gone over a lot here.But these are just some of thepossibilities.Ultimately for those of you, whoare interested, we're interestedin your feedback, ideas, andcontributions to help make SwiftPackage Manager the best toolthat it can be for the developercommunity.So, to talk about how you can dothat if you'd like to, I'd liketo talk about Swift's opensource process.

    As I said earlier, the PackageManager is part of the Swiftopen source project.And Swift.org is also a greatplace to go if you want to learnabout the community and theprocess.

    SwiftPM uses the Swift languageevolution process which meansanyone can submit a proposal formajor new features or changes tothe Swift Package Manager.

    Before you go off and draft awhole formal proposal though, Irecommend that you swing by thePackage Manager section of theSwift forums and socialize youridea with the community.You may get a lot of feedbackthat helps make your idea evenbetter.If you're interested in dippingyour toe in the water with asmaller contribution the Swiftbug tracker at bugs.swift.orghas plenty of ideas.In particular, you may want tolook for bugs tagged with thisstarter bug tag.And since, as I said, SwiftPM iswritten in Swift, you may findit's actually pretty easy todive in and take a look.Of course, if you find bugs whenyou're using SwiftPM, weencourage you to go file them onbugs.swift.org as well, whereyou can track how we handlethem. SwiftPM gets to take advantageof the same great continuousintegration infrastructure thatthe Swift project has.Which means that poll requestscan be automatically built andhad their tests run before theirmerged.Because the SwiftPM code baseitself has great test coverage,we found that this infrastructure is really usefulfor us.

    When you're ready to try out thelatest changes, you can downloadthe Trunk Snapshot Toolchainsthat are updated on a regularbasis available on Swift.org.

    We've been really happy with thegrowth of the SwiftPM communityso far. We've had over 180 peoplecontribute, either with bugfeatures or new features.And the Swift Package ecosystemis growing at a healthy rate aswell, often with cross-platformpackages, and many publicpackages available on GitHub.What this means is that you canfocus on what makes your productspecial and let packagedependencies handle the rest.There are a couple things that Irecommend you try SwiftPM outfor today, even though it has alot of room to grow in thefuture.Those two things are commandline utilities and libraries andfor developing with Swift on theserver.

    The server-side Swift communityhas been making extensive use ofthe Swift package manager.And server-side Swift has beengrowing well itself with manyframeworks now available fordoing web and backenddevelopment.If you would like to take a lookat this approach, I think you'llfind that Swift is a greatlanguage to do this kind ofcross-platform development on.

    But you could also go ahead anduse SwiftPM for creatingcommand-line utilities andlibraries today, whatever makessense for you.Ultimately getting started is aseasy as opening a terminalwindow and running Swift packageinit.So, the next time you'rethinking about trying somethingnew, I encourage you to give ita try.And if you're interested incontributing, swing by the Swiftforums and start a conversation.If you'd like to come chat withus, you can find us in the labstomorrow at 3 p.m. Ultimately,I'm really excited about wherewe can go from here and whatthis developer community can dotogether.Your contributions will help usdesign a package manager thatwill be great for the wholeSwift community.Thank you.Enjoy the rest of WWDC.[ Applause ]