Shapefile to KML from OGR GDAL

Over the last few years there have been quite a few tools developed, including from Esri, that generate .kml files from your Shapefiles. They can be tough to use at times or easy but none are as satisfying as running from OGR GDAL at the command line. Why? Who cares, its just cool to be able to run things from the command line. Go here to follow Google's how-to:http://code.google.com/apis/kml/articles/vector.html

If you download and install GDAL OGRhttp://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries

Go here at Google Code http://code.google.com/apis/kml/articles/vector.html and follow the easy steps. And then open up your new .kml in Google Earth.

Even later on you can customize more with libkml. http://code.google.com/p/libkml/

My first Stata command, summing two fields into a new field, why its easier than ArcGIS

So I barely use any software that's not GIS-related. Anyway, without making a list of the silly things that take forever in ArcGIS - at modelbuilder and even at python - (aka, deleting multiple fields for large datasets, took 8 hours last night!!!!), I've known I need to learn Stata to better manage my output files, so I requested Stata 12 and figured I'd share a little of my learning experience for my own reference but also in case anyone else is just starting out.

I'd like to start generating some calculated variables outside of ArcGIS. Some of my variables such as percent (PCT) of a population or densities could more efficiently be calculated outside of ArcGIS.

Here's how I've been adding a new field and calculating in ArcGIS:

I created model that has input parameters and allows the user to input everything they need to add a new field and calculate. It saves me just a little time by not having to add both models or type out all of these lengthy commands.

Here's the Python code:

# Import arcpy module
import arcpy

# Script arguments
Input_Table = arcpy.GetParameterAsText(0)

New_Field_Name = arcpy.GetParameterAsText(1)

Field_Type = arcpy.GetParameterAsText(2)
if Field_Type == '#' or not Field_Type:
Field_Type = "LONG" # provide a default value if unspecified

Calculate_Field_Name = arcpy.GetParameterAsText(3)

Calc_Expression = arcpy.GetParameterAsText(4)

Field_Expression_Type = arcpy.GetParameterAsText(5)
if Field_Expression_Type == '#' or not Field_Expression_Type:
Field_Expression_Type = "PYTHON_9.3" # provide a default value if unspecified

# Local variables:
Output_Feature_Class = Input_Table
Output_Feature_Class__2_ = Output_Feature_Class

# Process: Add Field
arcpy.AddField_management(Input_Table, New_Field_Name, Field_Type, "", "", "", "", "NULLABLE", "NON_REQUIRED", "")

# Process: Calculate Field
arcpy.CalculateField_management(Output_Feature_Class, Calculate_Field_Name, Calc_Expression, Field_Expression_Type, "")

Granted, there's Parameters instead of real values, but you get the idea, each of those parameters need to be filled out and all this code needs to run.

Here's the Stata code:

gen SUM_M11M12 = TRT_POPM11 + TRT_POPM12

That was easy!

Granted, this was only a numeric calculation, I still feel super confident about my Python/VBScript string 'skillz'[:5] + 's', but just saying, Stata is super fast and at the end of the day, those last numeric calc's might be best served in Stata.

Calculating Geometry Measurements in Python

Calculating Geometry Measurements in Python

from Esri - http://resources.esri.com/help/9.3/arcgisdesktop/com/gp_toolref/data_management_tools/calculate_field_data_management_.htm

All below from Esri:

Python expressions can use the geometry Area and Length properties with an areal or linear unit to convert the value to a different unit of measure.

!shape.area@acres! 

Areal unit of measure keywords: Unit of measure keywords: ACRES | ARES | HECTARES | SQUARECENTIMETERS | SQUAREDECIMETERS | SQUAREINCHES | SQUAREFEET | SQUAREKILOMETERS | SQUAREMETERS | SQUAREMILES | SQUAREMILLIMETERS | SQUAREYARDS | SQUAREMAPUNITSLinear unit of measure keywords: CENTIMETERS | DECIMALDEGREES | DECIMETERS | FEET | INCHES | KILOMETERS | METERS | MILES | MILLIMETERS | NAUTICALMILES | POINTS | YARDS

Split Layer By Attributes for ArcGIS 10

This is an update to a tool I used a lot before migrating over to 10. I just needed something similar recently - for some zonal stats stuff and saw it was updated for 10. Thanks Dan Patterson.

http://resources.arcgis.com/gallery/file/geoprocessing/details?entryID=37AEB018-1422-2418-A036-CA6D9920F808

VBA: Link From ArcMap to Google Maps

The link below has a tool to link ArcMap to Google Maps and Streetview;

http://gis.utah.gov/code-visual-basic/vba-link-from-arcmap-to-google-maps

I've used this tool for a while at 9.3.1 but just started to use it again at ArcGIS 10.

Removing or Flattening Z and M values from a point Feature Class

I've had to flatten a Z value-laden point file before, or for whatever reason seen CAD data with M-awareness. For the life of me though I couldn't recall how the heck I did it in the past. It's like those tasks in GIS you do only once in a blue moon and cannot remember the name of the tool or what terms to search (see lattice dem).

Anyway, its actually a very simple process of exporting your data in ArcCatalog and just setting the environments.

http://support.esri.com/en/knowledgebase/techarticles/detail/35818

from Esri:

Summary

Instructions provided describe how to create a copy of a feature class and disable M- and Z-value associations for the output feature class. These associations cannot be removed from an existing feature class.

M-values are routing values.

Z-values are elevation values.

Procedure

1. Browse to ArcToolbox > Conversion Tools > To Geodatabase.
2. Open the Feature Class to Geodatabase (multiple) tool.
3. Add all the feature classes into the Input Feature Class parameter.
4. Select an Output Geodatabase.
5. Click the Environments button at the bottom of the tool dialog box.
6. Expand the General Settings.
7. For the parameter, Output has Z Values, change the value to Disabled.
8. For the parameter, Output has M Values, change the value to Disabled.
9. Click OK in the Environments dialog box.
10. Click OK to execute the geoprocessing tool.

Esri Press Books I've been reading & Float vs Double

It's been a while since my last post, I've been using twitter more and more, which means I've been posting less and less content and tips and tricks. I've been spending a lot of time messing around with Python (and after a few months of realizing I had the wrong installation of Python, I've now got my arcpy and other modules running smoothly) and relearning how to do things with modelbuilder.

I've started going through Getting to Know ArcGIS: Modelbuilder (http://esripress.esri.com/display/index.cfm?fuseaction=display&websiteID=195&moduleID=0) to help relearn and understand the things I've been doing for a few years. This is the first comprehensive text on modelbuilder I've seen and there is so much value in Iteration and a whole slew of other tools that would have helped my modelbuilder experiences earlier on.

I've also been reading Modeling Our World (2nd Edition): The ESRI Guide to Geodatabase Concepts (http://store.esri.com/esri/showdetl.cfm?SID=2&Product_ID=1247&Category_ID=56). This book is really great. It provides a great overview of GIS data and is a great all-in-one refresher. So I'm going to try and post more and more about the little things in ArcGIS.

Now, on to Float vs. Double.

So for a long time I've been defaulting to using Double because it allows for 16 significant digits. However, as I read in Modeling Our World, Float stores 7 significant digits and is half the storage space of Double. I usually use Double when dealing with decimals but after realizing that I'm essentially expanding all my databases, with sometimes many many fields, to be double (no pun intended) the size they need to be because often when I deal with decimals, 7 digits is more than enough. So by cutting down the size of my databases I will be reducing my storage footprint but also hopefully speeding up my processing time.

DUHHHHH.

So the takeaway should be, if you don't need Double, use Float.

"Floats use half the byte storage of doubles" - Michael Zeller, Modeling Our World

For more info on field types see:
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//003n0000001m000000.htm

My First Google Fusion Map

Below is my first Google Fusion map showing 2000 Census Tract boundaries in NYC. I haven't messed around with Fusion much but will be looking forward to seeing if I can use it for some of my projects.

The future of ArcGIS on Apple Computers (running BootCamp) and using external drives with the faster I/O Thunderbolt

So with the release of the latest Apple Macintosh MacBook Pro and iMac computers with the new Thunderbolt I/O I have been thinking a lot about what this means for running ArcGIS software on Mac hardware. When I'm not in the office I'm running ArcGIS one of three ways on my mac; Remote Desktop Connection, Virtualization (Parallels or VMWare Fusion), OR by using Bootcamp.

Remote Desktop Connection - requires internet connection and my desktop at work to be on; screen space and graphics are lacking, faster b/c it writes directly to internal drives.

Virtualization - I've pretty much given up on this since there's reduced performance from allocation of RAM.

Bootcamp - this is my preferred method of using ArcGIS, however since my disk space is finite I tend to run this on external drives, thereby reducing my processing speed b/c of the read/write time. I use either FAT32 (which stinks b/c of the file size restrictions) or NTSF which supposedly, but I haven't had trouble with it yet, has problems writing when using the Mac OS.

When using Bootcamp, I also write to my Mac's main hard-drive sometimes as well as Mac OSX (Journaled) formatted disks using MacDrive.

Why do I run ArcGIS on a mac? Well mostly b/c I just find the Mac hardware and experience much better. Also, I find that my system is actually more stable. So if I could write to disk using Thunderbolt ports, assuming Bootcamp has the write Drivers and I can use MacDrive to access Mac OSX (Journaled) drives for data, I would most certainly build a very nice reliable primary system using ArcGIS on Apple hardware utilizing Thunderbolt I/O technology.

I look forward to learning if and how I could do this.

How Much Do GIS Consultants Charge in 2011?

In Directions Magazine Atanas Entchev posted an article entitled How Much Do GIS Consultants Charge in 2011?

Billing rates, summary of all respondents



Billing rates, summary of private entities

Commentary

• A few interesting (to me, at least) observations:
• While private entities charge more (as expected), they do not charge that much more than non-profits and academic institutions.
• The lowest prices in all categories (except data conversion) came from private entities.
• The survey results jive nicely with the GSA schedules and the New Jersey GIS state contract pricing.
• The only notable exception is the data conversion numbers – higher on average in the survey than in other data sources.
• I am surprised that training does not command higher rates.
• Applying the 25% rate of inflation change to the 2001 prices gets us very close to the numbers in the above tables.

Comments from survey respondents

• “Tough economy has started fierce price wars.”
• “Consulting jobs in West Texas are rare. I usually quote by the job.”
• “I think you should have also asked how many years you have been in business. While I put my hourly rate down, 80% of my business is project-based fees as opposed to hourly.”
• “I charge different rates on a sliding scale depending on the client; e.g. charge less for non-profits and more for private clients.”
• “I live in a rural community and realize my rates are very low. I have lower rates when I have on-site services with a set hourly commitment.”
• “I also work for a flat project fee. My clients are all non-profits, so I tend to think I charge less than say private company contractors.”
• “I have to spend a lot of 'business development' time to get work. I charge $150 per hour.”
• “These rates are for our NYC staff, most of our other locations have lower rates.”
• “Many projects not on hourly basis but fixed price.”
• “We also charge a flat fee for plotting maps- $35 (does not include labor for setting up map files or data for presentation).”

READ FULL ARTICLE HERE