Where to Find the desktop.msi File for Performing a Silent Install of ArcGIS for Windows

If you want to perform a silent install of ArcGIS desktop for Windows using the Microsoft Installer (.msi) file, you need the actual .msi file. But finding the .msi file can seem difficult. It’s nowhere to be found on the ESRI download site (that I could find).

It turns out to be quite easy to get the installer. First, download the ArcGIS desktop installation executable from the my.esri.com site. This will be named something like ArcGIS_desktop_108_172737.exe.  Using Winzip, 7-zip, or another compression utility, extract the files just like you would with a .zip file.  In the extracted directory, click into Desktop, then into SetupFiles. The desktop.msi file will be sitting right there.

Which ESRI Javascript API Version to Use – 3.x vs 4.x

Which ESRI Javascript API version should you use for your next mapping app? I wish it were as simple as just using the most up to date version. And maybe it will be for your situation. But there are a few considerations to address before pushing ahead.

Start With Your Goal

Always start with your goal. What do you want the app to look like and do? Probably, the biggest question you should ask yourself is whether you will be creating a 3D app or a standard 2D app?

As of this post, the 4.x API is at version 4.11. According to ESRI’s API Capability page, version 3.28 (the latest 3.x version) doesn’t support 3D rendering. Version 4.11 only has partial support for 2D. So, if you’re planning to create an app that supports 3D visualization in any way, you’ll need to use 4.11.

On the other hand, if you are only supporting 2D visualization you’ll want to use the latest API that supports all of the other functionality you might want to use. This is really important to consider if you’re thinking about migrating existing 3.x code to the 4.x world. You would hate to get deep into the conversion only to find out a critical component you rely on is not yet supported.

Check out ESRI’s functionality matrix to determine if 4.11 is implementing all the functionality you need right now. ESRI’s stated goal is to eventually have 4.11 eventually exceed the functionality of the 3.28 API. But until it does, you’ll want to proceed with at least a little bit of caution.

Consider Your Time

Another thing you will want to think about is the time you will spend getting up to speed on the syntax changes that version 4.11 brings. Right away, when creating a basic map with 4.11, you’ll notice that you now have to not only create a map but a map view (or map scene if it’s a 3D app) in order to get anything to render.

With the old 3.x API you would simply declare the map object and pass in a reference to the HTML element you want to use to render it in. As of 4.11, the Map object is now simply a container for the various layers you want to associate with the app you’re creating. There is now a new class called View that manages UI methods for your app like rendering the map within the HTML element and placing components (like widgets and images) on screen.

I like most of the new syntax changes that I’ve seen in 4.11 but I realize these slow down my development time as I get used to them.

Toggle Layer Visibility Using URL Parameters in Web App Builder Developer Edition

ESRI’s developer edition of their Web App Builder (WAB) is a handy stand-alone tool for creating web mapping apps. While the WAB is a tool for building an app without needing to code anything, the developer edition allows users to create their own widgets and extend current functionality or themes. Even with this capability, however, there are some situations where the pre-formed development framework just doesn’t go far enough.

This was the situation I found myself in recently when trying to use the WAB to replace our custom built web map viewer at work. Our current viewer interfaces with a few third-party apps by accepting URL parameters that turn on or off layers and query various layers. The WAB does allow for querying layers using URL parameters but it doesn’t have the ability to toggle layers using the URL method.

I searched around the internet trying to find someone who has solved this problem but never found a useable solution. ESRI provides URL parameter layer visibility functionality on their ArcGIS Online platform but this hasn’t made it to the WAB Developer Edition yet. I’m not sure when or if it will.

Since layer toggling is a must-have functionality for us I decided to work up a solution myself. Thankfully, the developers at ESRI named the WAB’s URL handling module mapUrlParamsHandler.js so it was pretty easy to figure out what needed to be modified.

Parameter Modeling

To fix my problem I just had to add one new function. The actual turning on and off of the layers in this function was taken care of by the WAB API. The biggest concern for me was deciding on how the parameters should be passed in the URL so they would be easy to use on the client side and easy (and fast) to process on the server.

I considered using the Esri ArcGIS Online model of ?layers=show:0,1,2,3 for passing in layer visibility parameters. However, this becomes very cumbersome when considering showing and hiding both layers and sublayers. It would look something like ?layers=show:0,1.0-2-5,3,4;hide:6,7 or some other cryptic looking mash of numbers and characters. I wasn’t even sure the online API accepted a hidden parameter. They don’t show one in their documentation.

I then considered using two separate parameters for showing and hiding (?showLayers=1,2,3&hideLayers=1,2,3) but this just adds more complexity to the code on the back side as well as the parameters the client has to plug in. Ultimately I settled on using a single parameter called layers. But then I needed to decide how to reference those layers.

If I used a zero-based index URL parameter list, then if the layers in the web map ever change position, I’ll have to go in and change the URL references to those layers. On the other hand, if I used the titles of the layers, it wouldn’t matter what the index position of the layer is. The name of the layer and the title would still be the same.

It’s true that the title of the layer could change too. In that case, we’d still have to update the URLs we’re passing into the app. But in our situation, this is less likely to happen than the position changing. Using titles has another advantage of making it clear to the casual observer exactly what layers are being acted upon. This wouldn’t matter that much since the public isn’t going to be encouraged to pass parameters into the URL. But it might be nice for us developers to know what we’re doing.

I ended up using layer titles since they’re human readable and don’t rely on positioning within the web map that drives the web app. However, I created both versions of my modifications so that someone else who wants to use layer indices can do so just as easily.

In my parameter, layers are separated by commas with layers to be shown represented by the layer title (or positive index integers) and layers to be hidden represented by layer titles with a minus(-) symbol in front of them (or negative index integers).

Toggling Sub Layers

I also wanted to be able to toggle sub-layers on and off. Sublayers to be toggled will be shown by separating the parent layer from the sublayers with a colon. The sublayers themselves will be separated by semicolons.

In WAB apps, sublayers are 0 indexed underneath their parent. Suppose you have an active layer called School Boundaries with a map index position of 6 and it has three sublayers for High Schools, Middle Schools, and Elementary Schools. These sub-layers would be indexed as 0, 1, 2.

I decided to stick with index references for the sublayers since it was easy to do so and makes sense. It’s also easier to read in and understand within the URL since the parent layer is text so there’s some contrast. 

With the above model of building your URL for layer toggling, you can take care of almost any layer manipulation scenario you can think of.

This would turn on the Schools layer as well as the first, second and third sublayers: 
?layers=Schools:0;1;2

This would turn off the Schools layer and deselect sublayer 0, then turn the schools back on and select sub layers five and six:
?layers=-Schools:0,Schools:5;6

Edit: Having to turn an entire layer off and then on again, just to get at the sublayer, was cumbersome. Now you can use the minus symbol in front of the layer title or the sublayer index to turn them on and off independently. For example – ?layers=Schools:-0;5;6


Multiple Params


Another problem I have with the URL parameter handling capabilities of the Web App Builder is that you can’t add multiple parameters. In other words, you can’t pass in layers to turn on and do a query on a layer at the same time. To solve that problem I just modified the main function in the module to check all URL parameters rather than stopping after finding the first one.

mo.postProcessUrlParams = function(urlParams, map){
    //urlParams have been decoded.
    for(var key in urlParams){
      //Loop through the urlParams object
        if(urlParams.hasOwnProperty(key)){
          //For each parameter found, run its function
          if('layers' === key){
            toggleLayers(urlParams, map);
          }else if('extent' === key){
            setExtent(urlParams, map);
          }else if('center' === key){
            setCenter(urlParams, map);
          }else if('marker' === key){
            createMarker(urlParams, map);
          }else if('find' === key){
            sendMessageToSearch(urlParams);
          }else if('query' === key){
            queryFeature(urlParams, map);
          }
        }
    }
  };

How to Use

In order to use the modified mapUrlParamsHandler module in your WAB project, you first need to download the appropriate one (index driven or title driven) from Github at https://github.com/RyanDavison/WAB_URL_Parameters. Then replace the native file located in \WebAppBuilderForArcGIS\server\apps\<Your Apps ID Number>\jimu.js .

If you’ve already exported your app and are hosting it on your own server just find the jimu.js folder and paste the file in there. Alternatively, you could just copy the code out of the files on Github and paste it right into the native mapUlrParamsHandler.js file. That’s all you have to do to get layer toggling functionality through your URL.

In the future, ESRI might enable this same functionality in the WAB Developer Edition. If they do, it’s a good bet they won’t have the same URL structure as me. As I’m writing this, The Web App Builder is at version 2.9. So if you start using my modified code now, you might be changing your own URL structures to match the ESRI API in the future.

If you have any questions, comments or problems feel free to leave them in the comments section below or contact me on Github.

EDIT:
When I first wrote the modification You could only turn sub-layers on and off along with their parent layer. So, if you wanted to turn off sub layer 2  but turn on sub layer 3 of LayerX you would have to write

?layers=-LayerX:2,LayerX:3

It was a two-step process that was very clunky. Now, you can turn off sublayers independently. So the query above would now simply read:

?layers=LayerX:-2;3

Of course you can still turn off an entire layer like this:

?layers=-LayerX

Custom Color Ramps For ArcGIS

Back in 2012, I wrote a post about loading custom color ramps into ArcGIS. It’s been a consistently popular post but is very outdated. In that post, I suggested downloading and installing ESRI’s ColorRamps2.o package. ColorRamps2.0 was replaced with ColorRamps3.0 around the time I wrote the post. Today, although there’s a link to download ColorRamps3.0 it doesn’t seem to be functioning.

Instead of simply updating an old blog post, I thought it would be helpful to write a new updated post. I’ll show you how to create color ramps on the fly in ArcGIS and how to import custom color ramps into ArcGIS 10.5. I’ll also provide a list of color ramp creation and download sources.

If you already have a color ramp .style file, you can follow the steps below to add them into ArcMap:

  1. Copy the .style files to C:\Program Files (x86)\ArcGIS\Desktop10.5\Styles. Your Styles will be different depending on your operating system and version of ArcGIS.
  2. Open ArcMap.
  3. Click the customize dropdown and select Style Manager.
  4. On the right side of the Style Manager click on the Styles… button. At this point, your custom style name should just show up in your style manager. However, if it doesn’t because you put your style file in a different directory or some other reason, you need to follow two more steps. First, click Add Style to List. Navigate to the directory where you placed the .style files and select your custom file. Highlight the file and click Open. You will have to do this for each .style file you want to add.
  5. After all of the styles are shown in the Style References list, make sure they are check marked and click the Set as Default List button.
  6. Click OK and you should see your styles on the left side of the Style Manager. Close the style manager. Now, you can go to the symbology tab of your Layer Properties for a given raster and select one of the new color ramps. If you want to see the text descriptions of the ramps, click on the color ramp dropdown and uncheck ‘Graphic View’.

So where can you find alternative .style files with color ramps? It seems to be getting harder to find them these days. As I mentioned above, you used to be able to get them from the ESRI mapping center but that resource seems to be abandoned. Let me know if you know where to find ColorRamps3.0 or 2.0 today.

ColorBrewer2.org used to be another good resource but the site has removed the option to download ramps in the .style format. Fortunately, you can still access them by going to http://www.reachresourcecentre.info/arcgis-colorbrewer-color-ramp-style and downloading the zip file.

You can check out Fred Lott’s color ramps on GitHub too. Of course, if you aren’t finding quite the ramp for your needs and you’re trying to show quantities, you could always build your own.

To set your own quantities ramp for your layer, follow these steps:

  1. Right click on your layer and go to the Symbology tab.
  2. Click on Show Quantities and select Graduated Colors.
  3. Choose your field’s value and normalization and how many classes you want your data broken into. Your classes will already have a color ramp assigned
  4. Double click on the first class symbol color. Choose another color that you want for the start of your custom ramp.
  5. Next, double click on the last class symbol color and choose the last color in your ramp.
  6. Right click on any of the symbols and select Ramp Colors. This will generate the color ramp between the top and bottom colors you chose.

So there you have it. If you’re not happy with what ESRI has already provided you for styles you have a few options for customization. If you know of any other style resources I didn’t mention, please feel free to let me know in the comments. Thanks for reading.

 

Shapefile as a Multi User Editing Environment?

I had a ArcGIS user that I support come to me with a corrupted shapefile the other day. It had the old “number of shapes does not match number of table records” error. It turns out, he’s still using this shapefile as his layer’s main data source and he and several others regularly edit it! In this day and age?

I tried to convince him using a file geodatabase would be more stable for editing but he had been using shapefiles so long I don’t even think my comments registered. He just wanted a tool that could fix the shapefile.

I pointed him to the shapechk tool by Andrew Williamson. I’ve used the tool for years because <sarcasm>for some odd reason</sarcasm> I often run into people with corrupted shapefiles after people edit them over long periods of time. The shapefile works OK as a data exchange format but doesn’t always hold together under regular heavy use.

In the words of Pete Seeger, “when will we ever learn?”

Efficient ArcServer Cache Management with a Staging Server

Getting a cache built can sometimes be a challenge but caching an ArcServer map service can be important if you want your web apps to display fast. Depending on the subject of your cache and what scales you cache at you can end up with tens or hundreds of gigabytes of data.

Art Tiles

In my office we cache both vector and imagery data for an entire county. We have 20 aerial mosaic data sets dating back to 1937 and several vector data sets that also cover the entire county. Caching all of this can put a real strain on a server and takes up limited server resources that can cause the server to perform poorly.

I’ve found that the best solution is to cache on a staging server and then transfer the cached tiles to a prepared production server. Here is how you can easily do the same:

  1. Create a service on your staging server.
  2. Set up caching in the Services Properties dialog.
  3. Create a service on your production server that has the same name as your staged service.
  4. Set up caching in the Services Properties dialog. From the “Tiling Scheme” drop down select “An existing cached map / image service” and navigate to the staging service. This will import all of your cache advanced setting that you defined on your staging service like scale levels.
  5. Run the Manage Map Server Cache Tiles tool to create the staged cache.
  6. Now you just want to copy your level folder (L00, L01, L02…) located in \arcgisserver\directories\arcgiscache\[your map service name]\_alllayers to the same location on your production server.

At this point your production server should have a full working cache. If you have a cache service that will need to be updated regularly you could even script the whole process and schedule it to run at night or on the weekend.

 

Better Basemap Management for the ESRI Javascript API

If you use ESRI’s Javascript API to build web maps you have probably noticed that your ability to manage base map layers is somewhat limited. You really only have three out-of-the-box alternatives:

  1. Define an ArcGIS.com basemap in the map constructor
  2. Create and fill a Basemap Gallery
  3. Switch between two basemaps with Basemap Toggle

But what if you don’t want to use an ArcGIS.com basemap, you want to use something other than thumbnail images to switch between basemaps or you have more than two basemaps to switch between? In that case, you probably need something a little more flexible.

I recently found myself in one of those situations. I wanted to be able to switch between a vector basemap and a basemap of current imagery as well as give viewers the opportunity to choose imagery as far back as the 1930s. I really didn’t want to show thumbnails for every year of imagery. Instead, it seemed to make more sense to allow selection by year from a drop down menu.

So I went ahead and built my own little version of a basemap switcher to handle different situations. To be fair, I could have used the ESRI Basemap Toggle dijit to handle the two main basemaps but I like having more control over my code than Dojo based dijits allow.

I put together a simple map to demonstrate switching multiple basemaps through various means like buttons, thumbnails and drop downs. The html markup and CSS  styling is pretty basic. We create a header as a toolbar (in this case our switcher is the only tool), a div for the map to be injected and markup for the the jQuery UI dialog box that will hold our switching controls.

Edit: FC Basson’s comment got me thinking about the code samples here being too complicated for what they were actually doing. I went back through and removed the toolbar and all jQuery UI and jQuery code. None of these added anything special to the app and since Dojo is already available to us, it should be used instead

<!doctype html>
<html lang="en">
<head>

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=7, IE=9, IE=10">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="apple-mobile-web-app-capable" content="yes">
    <meta name="apple-mobile-web-app-status-bar-style" content="translucent-black">

    <title>Custom Basemap Switcher</title>

    <script src="http://js.arcgis.com/3.7/"></script>

    <link rel="stylesheet" href="http://js.arcgis.com/3.7/js/dojo/dijit/themes/nihilo/nihilo.css">
    <link rel="stylesheet" href="http://js.arcgis.com/3.7/js/esri/css/esri.css">

    <style>
        html, body, #map
        {
            height: 100%;
            width: 100%;
            margin: 0;
            padding: 0;
            overflow: hidden;
        }

        #map
        {
            position: fixed;
        }

        #basemapButton
        {
            position: absolute;
            top: 25px;
            left: 70px;
            z-index: 100;
        }

        button
        {
            border-radius: 4px;
            background-color: #F1F1F1;
            font-size: 1em;
            color: #33A7DE;
        }

        #basemapDialog
        {
            background: white;
            display: none;
            font-size: 0.75em;
            height: 200px;
            left: 65px;
            position: absolute;
            top: 70px;
            width: 300px;
            border: 0.5px solid black;
            border-radius: 5px;
        }

            #basemapDialog ul, li
            {
                list-style: none;
                margin-top: 1em;
            }

        a img
        {
            border: 1px solid black;
        }

        a:hover img
        {
            border: 1px solid white;
        }
    </style>


</head>
<body class="container tundra">
 <button id="basemapButton" title="Open a basemap switching dialog">Switch Basemaps</button>
 <div id="map"></div>
 <div id="basemapDialog" title="Choose a Basemap">
 <ul>
 <li>
 <button id="wldImagery" title="Load the World Imagery layer as a basemap using this button">Imagery</button> Select by button</li>
 <li><a id="streetView">
 <img title="You can use an image thumbnail like this one to select a basemap" src="http://ryanrandom.com/examples/images/vegas.jpg" alt="Street Map View" height="50" width="50" />
 </a> Select by thumbnail</li>
 <li id="differentBase" title="View historical imagery as a basemap">
 <select name="bmSelect" id="bmSelect" title="Selecting a basemap from a dropdown is handy when you have many choices, like historical basemaps">
 <option value="none" selected>None Selected</option>
 <option value="topo">Topo</option>
 <option value="relief">Shaded Relief</option>
 </select> Select by dropdown
 </li>
 </ul>
 </div>
</body>
</html>

The code needed to run the switcher isn’t too complex either. You just need to create an ESRI Javascript map reference, create variable references to the basemaps you want to use then create a function that does the actual changing of the layers.  The rest of the code is just some jQuery Dojo/JavaScript to interact with the controls.

    <script>
        require([
            "esri/map",
            "esri/layers/ArcGISTiledMapServiceLayer",
            "esri/geometry/Extent",
            "dojo/dom",
            "dojo/dom-attr",
            "dojo/dom-prop",
            "dojo/on",
            "dojo/query",
            "dojo/NodeList-traverse"
        ], function (
            Map, ArcGISTiledMapServiceLayer, Extent, dom, domAttr, domProp, on, query
          ) {
            //We can set an extent like this one which takes us to Las Vegas, NV
            var initialExtent = new Extent(-115.68, 35.77, -114.75, 36.45);

            //Create an empty array to store a reference to the currently selected basemap layer
            currentBasemap = []

            map = new Map("map", {
                extent: initialExtent,
                logo: false,
                slider: true
            });

            //create references to all of your basemap layers but only add one to the map initially
            imagery = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer");
            //add the current basemap layer to the currentBasemap array so you can keep track of what is active.
            currentBasemap.push(imagery);
            map.addLayer(imagery);
            //call reorderLayer on the map object and set your basemap to zero every time you change basemaps. This ensures that your basemap is always the bottom layer.
            map.reorderLayer(imagery, 0);

            street = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer");
            topo = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Topo_Map/MapServer");
            relief = new ArcGISTiledMapServiceLayer("http://services.arcgisonline.com/ArcGIS/rest/services/World_Shaded_Relief/MapServer");

            //Opens and closes the basemap dialog using the basemaps button
            on(dom.byId("basemapButton"), "click", function () {
                if (!(dom.byId("basemapDialog").style.display == 'block')) {
                    dom.byId("basemapDialog").style.display = "block";
                } else {
                    dom.byId("basemapDialog").style.display = 'none';
                }
            });

            //This function does just what its name says and changes the layer
            function changeLayer(layerName) {
                map.removeLayer(currentBasemap[0])
                currentBasemap.length = 0;
                currentBasemap.push(layerName)
                map.addLayer(layerName)
                map.reorderLayer(layerName, 0);
            }

            //Here is where we switch the basemap using a button
            on(dom.byId("streetView"), "click", function () {
                domProp.set(query("#differentBase option")[0], "selected", true);
                changeLayer(street)
            });

            //Here is where we switch the basemap using a thumbnail image
            on(dom.byId("wldImagery"), "click", function () {
                domProp.set(query("#differentBase option")[0], "selected", true);
                changeLayer(imagery)
            });

            //Here is the code for handling changing your basemap based on a html select box dropdown.
            dom.byId("bmSelect").onchange = function () {
                var newBasemap = dom.byId("bmSelect").value

                //Default to the imagery basemap if "none Selected" is the choice in the dropdown 
                if (newBasemap === "none") {
                    changeLayer(imagery)
                } else {
                    //The following allows you to map the dropdown values with an object reference to the tiled service layers created above
                    var bmList = ({
                        "topo": topo, "relief": relief
                    })

                    //Find the selected basemap in the array above and use the associated object to add the selected theme layer to the map
                    for (var x in bmList) {
                        if (x === newBasemap) {
                            changeLayer(bmList[x])
                        }
                    }
                }
            }
        });
    </script>

You will notice in the code above that we have create an empty array called currentBasemap.  This array is the key to making the tool work. We store a reference to the currently loaded basemap here. That way we can access it, remove it’s corresponding basemap and load a reference to whatever new basemap that we want to display.

That’s all there is to it. You can get the full code from GitHub. You can also view a working example of the code.

Let me know if you have any suggestions on how to make it better.

Easily Export ArcGIS Attribute Table to CSV

 

For many ArcGIS users, exporting an attribute table to a .csv file or excel format file is a common part of their workflow. Unfortunately, exporting to either of these formats directly from the attribute table has never been a core functionality of ArcGIS. To overcome this you typically have had two choices: write a script using third-party libraries or use an ArcToolbox tool such as the Export Feature Attributes to ASCII Tool or the Table to Excel conversion tool. But with each of these, you have to open special tools or go through multiple steps to convert.

However, there is a little shortcut you can take to quickly export your data to .csv right from the attribute table.

  • From the table, click Export.
  • Under “Output Table” browse to the folder you want to put your .csv file in, save type as .txt and enter a name for your output making sure to leave off the ‘.txt’ from the end of the name.

If the ‘.txt’ extension is present in the name the tool will output a normal text file of your data. If, however, your name does not include the ‘.txt’ extension or if you specify .csv, then your output will be a .csv file by default.

 

Can I export directly to a .xlsx format?

There’s another handy little trick for quickly getting ArcGIS attribute table data into any Microsoft Excel data format.

    1. With your attribute table still open, click on the Table Options drop-down in the upper left corner of the table and click “Select All”.
    2. Alternatively, you can select only the rows you want to export by holding your Ctrl button while clicking the leftmost, gray box by each desired row. You can also use the Select By Attributes button at the top of the table or select features (and their corresponding table records) from the map itself. The point is, select some records to export.
    3. Now right click on any of the leftmost, gray box of any record in the table and choose “Copy Selected”.
    4. With an Excel spreadsheet already open, right-click in the uppermost left cell and click “paste”. All of your records will be pasted to the worksheet.
    5. Save your worksheet as a .xlsx file (or any file extension that Excel provides for saving) and you’re done.

It takes longer to explain the above method than it does to put it into practice. After you do it once or twice it will be very quick.