解决方法：
卸载 Microsoft sql server 2012 Express
LocalDB 即可
修改后如图
本文已收录于以下专栏：
相关文章推荐
转载地址：
安装sql2005后，没有SQL Server management studio。该怎么办？
1：原来安装的有SQL 2000 和VS2005
很简单 下一个management 微软中国 上面下载SQL Server management studio expree版就行
你安装了SQL2000的时候自带了使用计算机管理的SQL管理插件。你安装了微软的05的话，你应该在安装组建中选择一个SQL Server management studio,可能你下载的是精简版本，一般正常下载的是一张光碟大小。解压1.4G。如果急需管理界面可以临时使用SQL 2000的管理界面连接进行管理。
http://blog.csdn.net/w/article/details/8948877
原因 ：因为你装了vs2012吧，哈哈哈哈~~~
解决方案：
...
在安装SQL Server的时候经常会遇到安装失败，这是很伤脑筋的事情，花费很多时间和精力也不一定能够解决。 针对于规则检查导致的安装错误，如果有一份列表能够说明规则以及解决办法将会给我们提高极大的帮助。微软其实已经帮助我们提供了类似的文档。
安装操作完成之前，SQL Server 安装程序会验证您的计算机配置。下表描述了系统配置检查器的检查参数和妨碍性问题的解决办法。
今天想复制一份SLQ2008下的数据库,需要把数据库暂停一下,但是打开资源管理器却打不打,提示无权限,因为SQL2008是VS2010附带安装的,以为程序有什么损坏,就重新安装了一下VS2010修复了一下,结果无效.
又用一份单独的SQL2008安装程序修复,结果还是一样的问题,折腾一晚上,没解决问题.
上网一百度,找到一个和我一样情况的网友,而且附带了解决办法如下:
原问题地址:&a href=&/question/.ht
刚刚安装了SQL SERVER 2008 EXPRESS，打开配置管理器后却无法打开SQL Server服务，提示“远程过程调用失败”，
通过网上查，才知道是因为SQL Server2008 与V...
&h1 class=&postTitle& style=&margin-top: 10 margin-right: 0 margin-bottom: 10 margin-left: 0 padding-top: 0 padding-right: 0 padding-bottom: 0 padding-left: 0 border-bottom-width: 1 border-bottom-style: border-bottom-color: rgb(221,221,221); font-size: 14 line-he
解决方法：
1. 修改数据库为紧急模式
ALTER DATABASE GPSDB SET EMERGENCY
2、使数据库变为单用户模式
ALTER DATABASE GPSDB SET S...
SQL Server 2008安装过程中出现1608错误的解决办法
一直使用SQL Server 2000 ,觉得安装方便快速,便不想升级到.今天有个项目突然要用到2008,于是就在Windows7下安装了一下,没想到碰到一个1608 错误,中间提示一大堆信息,上网搜索了一下,也好多遇到过这个错误,试了网上的方法都没有成功,郁闷中搜索到一个国外的方法:
SQL Server 2008 Setup fails on Windows 7 Enterprise, Error code 1608
To troubleshooting the issue
解决方法一： 1.运行对话框中输入mmc
------打开控制台，将cmd改写成mmc
------在弹出来的对话框里，点击文件
------添加/删除...
您举报文章：
举报原因：
原文地址：
原因补充：
(最多只允许输入30个字)(Australia's Commonwealth Scientific and Industrial Research Organisation and the Integrated Marine Observing System)
(Indian National Centre for Ocean Information Services)
(Maritime Affairs Unit, Institute for the Protection and Security of the Citizen,
Joint Research Centre of the European Commission)
IRD (Institut de Recherche pour le Développement, France)
CNRS (Centre National de la Recherche Scientifique, France)
UPMC (Université Pierre et Marie CURIE, Paris, France)
UCAD (Université Cheikh Anta Diop de Dakar, Sénégal)
UGB (Université Gaston Berger - Saint-Louis du Sénégal)
UFHB (Université Félix HOUPHOU?T-BOIGNY, Abidjan, C?te d'Ivoire)
IPSL (Institut Pierre Simon Laplace des sciences de l'environnement, Paris, France)
LMI ECLAIRS (Laboratoire Mixte International <>)
Marine Instruments S.A. (Spain)
(CoastWatch Caribbean/Gulf of Mexico Node)
(CoastWatch West Coast Node) which is co-located with and works with
(Environmental Research Division of SWFSC of NMFS)
(Central and Northern California Ocean Observing System, run by Axiom Data Science)
(Northeastern Regional Association of Coastal and Ocean Observing Systems)
(National Glider Data Assembly Center)
(Pacific Islands Ocean Observing System) at the University of Hawaii (UH)
(Southern California Coastal Ocean Observing System)
(Southeast Coastal Ocean Observing Regional Association)
(National Centers for Environmental Information) / NCDDC
NOAA NGDC STP (National Geophysical Data Center, Solar - Terrestrial Physics)
(Observing System Monitoring Center)
(Unified Access Framework)
Spyglass Technologies
Stanford University, Hopkins Marine Station
USGS Coastal and Marine Geology Program
University of Washington, Applied Physics Laboratory
This is just a list of some of the organizations where ERDDAP has been installed
by some individual or some group.
It does not imply that the individual, the group, or the organization recommends
or endorses ERDDAP.
ERDDAP is recommended within NOAA
includes ERDDAP in its list of recommended data servers for use by groups within NOAA.
Is the installation procedure hard? Can I do it?
The initial installation takes some time, but it isn't very hard. You can do it.
If you get stuck, email me at bob dot simons at noaa dot gov .
I will help you.
Or, you can join the
and post your question there.
To Do the Initial Setup of ERDDAP on Your Server
ERDDAP can run on any server that supports Java and Tomcat (and perhaps other application servers).
ERDDAP has been tested on Linux (including on Amazon's AWS), Mac, and Windows computers.
Amazon - If you are installing
ERDDAP on an Amazon Web Services EC2 instance, see this
Docker - Axiom now offers
If you don't already use Docker, we don't recommend this.
If you chose to install ERDDAP via Docker, we don't offer any support
for the installation process.
We haven't worked with Docker yet. If you work with this, please send us your comments.
Linux and Macs - ERDDAP works great on Linux and Mac computers.
Windows - Windows is fine for testing ERDDAP and for personal use,
but we don't recommend using it for public ERDDAPs.
Running ERDDAP on Windows may have problems:
notably, ERDDAP may be unable to delete
and/or rename files quickly. This is probably due to antivirus software (e.g., from MacAfee
and Norton) which is checking the files for viruses. If you run into this problem (which
can be seen by error messages in the log.txt file like "Unable to delete ..."), changing the
antivirus software's settings may partially alleviate the problem. Or consider using a Linux
or Mac server instead.
The standard ERDDAP installation instructions are:
Type "java -version" from your server's command line to make sure you have
(JRE or JDK) version 1.8 or higher installed
(Java 1.7 is no longer supported because it reached its
and so is no longer supported by Oracle).
For security reasons, it is almost always best to use the latest version of Java.
ERDDAP works with 32 bit or 64 bit Java. 64 bit is preferred for 64-bit operating systems.
On Linux, we recommend that you download and install Java even if your computer came with Java
installed. This lets you be in control of which Java you have and where it is (usr/local?).
To install Java on Linux, see these
For security reasons, it is almost always best to use the latest version of Tomcat.
Below, the Tomcat directory will be referred to as tomcat.
Warning! If you already have a Tomcat running some other web application (especially THREDDS),
we recommend that you install ERDDAP in
because ERDDAP may need different settings
and shouldn't have to contend with other applications for memory.
Follow the instructions at
to set up Tomcat on your server.
On Linux, we recommend installing it in /usr/local.
On a Mac, it is probably already installed in /Library/Tomcat.
On Windows, select the "32-bit/64-bit Windows Service Installer".
It will be installed in c:/Program Files.
Security recommendation: you can remove some possible avenues for attackers by following
(especially the article's second recommendation)
to disable certain HTTP methods
(TRACE, PUT, OPTIONS, DELETE) in Tomcat.
On Linux and Macs, it is best to set up Tomcat (the program) as belonging to
user "tomcat" (a separate user with limited permissions and which .
Thus, only the super user can switch to acting as user tomcat.
This makes it impossible for hackers to log in to your server as user tomcat.
And in any case, you should make it so that the tomcat user has very limited
permissions on the server's file system
(read+write+execute privileges for the apache-tomcat directory tree and
&bigParentDirectory&
and read-only privileges for directories with data that ERDDAP needs access to).
You can create the tomcat user account (which has no password) by using the command
sudo useradd tomcat -s /bin/bash -p '*'
You can switch to working as user tomcat by using the command
sudo su - tomcat
(It will ask you for the superuser password for permission to do this.)
You can stop working as user tomcat by using the command
Do most of the rest of the Tomcat and ERDDAP setup instructions as user "tomcat".
Later, run the startup.sh and shutdown.sh scripts as user "tomcat" so that Tomcat has
permission to write to its log files.
After unpacking Tomcat, from the parent of the apache-tomcat directory:
Change ownership of the apache-tomcat directory tree to the tomcat user.
chown -R tomcat apache-tomcat-8.5.15
(but substitute the actual name of your tomcat directory).
Change the "group" to be tomcat, your username,
or the name of a small group that includes tomcat and
all the administrators of Tomcat/ERDDAP, e.g.,
chgrp -R yourUserName apache-tomcat-8.5.15
Change permissions so that tomcat and the group have
read, write, execute privileges, e.g,.
chmod -R ug+rwx apache-tomcat-8.5.15
Remove "other" user's permissions to read, write, or execute:
chmod -R o-rwx apache-tomcat-8.5.15
This is important, because it prevents other users from
reading possibly sensitive information in ERDDAP setup files.
On Linux and Macs:
Create a file tomcat/bin/setenv.sh
(or in Red Hat Enterprise Linux [RHEL], edit ~tomcat/conf/tomcat8.conf)
to set Tomcat's environmental variables.
This file will be used by tomcat/bin/startup.sh and shutdown.sh.
The file should contain something like:
export JAVA_HOME=/usr/local/jdk1.8.0_141/jre
export JAVA_OPTS='-server -Djava.awt.headless=true -Xmx1500M -Xms1500M'
export TOMCAT_HOME=/usr/local/apache-tomcat-8.5.15
export CATALINA_HOME=/usr/local/apache-tomcat-8.5.15
(but substitute the directory names from your computer).
(If you previously set JRE_HOME, you can remove that.)
On Macs, you probably don't need to set JAVA_HOME.
On Windows:
Create a file tomcat\bin\setenv.bat
to set Tomcat's environmental variables.
This file will be used by tomcat\bin\startup.bat and shutdown.bat.
The file should contain something like:
SET "JAVA_HOME=\Program Files\Java\jdk1.8.0_141/jre"
SET "JAVA_OPTS=-server -Xmx1500M -Xms1500M"
SET "TOMCAT_HOME=\Program Files\apache-tomcat-8.5.15"
SET "CATALINA_HOME=\Program Files\apache-tomcat-8.5.15"
(but substitute the directory names from your computer).
If this is just for local testing, remove "-server".
(If you previously set JRE_HOME, you can remove that.)
The -Xmx and -Xms memory settings are important
because ERDDAP works better with more
memory. Always set -Xms to the same value as -Xmx.
For 32 bit Operating Systems and 32 bit Java:
The more physical memory in the server the better: 4+ GB is really good, 2 GB is okay,
less is not recommended.
With 32 bit Java, even with abundant physical memory,
Tomcat and Java won't run if you try to set -Xmx much above 1500M (1200M on some computers).
If your server has less than 2GB of memory, reduce the -Xmx value (in 'M'egaBytes)
to 1/2 of the computer's physical memory.
For 64 bit Operating Systems and 64 bit Java:
64 bit Java will only work on a 64 bit operating system.
To enable 64 bit Java, add -d64 to the list of JAVA_OPTS in startup.sh and shutdown.sh, for example,
export JAVA_OPTS='-server -Djava.awt.headless=true -Xmx8000M -Xms8000M -d64'
With 64 bit Java, Tomcat and Java can use very high -Xmx and -Xms settings.
The more physical memory in the server the better. As a simplistic suggestion:
we recommend you set -Xmx and -Xms to (in 'M'egaBytes) to 1/2 (or less) of the computer's
physical memory.
You can see if Tomcat, Java, and ERDDAP are indeed running in 64 bit mode by searching for
" bit," in ERDDAP's Daily Report email or in the
bigParentDirectory/logs/log.txt file (bigParentDirectory is specified in
Linux and Macs, change the permissions of all
*.sh files in tomcat/bin/ to be executable by the owner, e.g., with
chmod +x *.sh
for images: We strongly prefer the free Vera Sans fonts
to the standard Linux/Java fonts.
Installing these fonts isn't required.
If you don't install these fonts, you need to change the fontFamily setting in
setup.xml to &fontFamily&SansSerif&/fontFamily& .
To install the fonts, please download
(344,753 bytes, MD5=E16AF0C4F6E0E9FD0A0D)
and unzip the font files to a temporary directory.
On Linux (as the root user) and Windows XP (as the administrator),
copy the font files into JAVA_HOME/lib/fonts
so Java can find the fonts.
Remember: if/when you later upgrade to a newer version of Java, you need to reinstall
these fonts.
On Macs, for each font file, double click on it and then click Install Font.
On Windows Vista and 7, in Windows Explorer, select all of the font files.
Right click.
Click on Install.
your Tomcat installation.
As user "tomcat", run tomcat/bin/startup.sh
View your URL + ":8080/" in your browser (e.g.,
You should see the Tomcat "Congratulations" page.
If there is trouble, see the Tomcat log file tomcat/logs/catalina.out.
Mac (run tomcat as the system administrator user):
Run tomcat/bin/startup.sh
View your URL + ":8080/" in your browser (e.g.,
You should see the Tomcat "Congratulations" page.
If there is trouble, see the Tomcat log file tomcat/logs/catalina.out.
Windows localhost:
Right click on the Tomcat icon in the system tray, and choose "Start service".
in your browser.
You should see the Tomcat "Congratulations" page.
If there is trouble, see the Tomcat log file tomcat/logs/catalina.out.
Troubles with the Tomcat installation?
On Linux and Mac, if you can't reach Tomcat or ERDDAP (or perhaps
you just can't reach them from a computer outside your firewall),
you can test if Tomcat is listening to port 8080, by typing (as root)
on a command line of the server:
netstat -tuplen | grep 8080
That should return one line with something like:
tcp 0 0 :::8080 :::* LISTEN ## ##### ####/java
(where '#' is some digit), indicating that
a "java" process (presumably Tomcat) is listening on port "8080" for "tcp" traffic.
If no lines were returned, if the line returned is significantly different,
or if two or more lines were returned,
then there may be a problem with the port settings.
See the Tomcat log file tomcat/logs/catalina.out.
Tomcat problems and some ERDDAP startup problems are almost always indicated there.
This is common when you are first setting up ERDDAP.
web site or search the web for help,
but please let us know the problems you had and the solutions you found.
Email me at bob dot simons at noaa dot gov .
I will try to help you.
Or, you can join the
and post your question there.
On Linux, Mac, and Windows, download
(version 1.80, size=23,878 bytes, MD5=7F85DEF49CEB, dated )
and unzip it into tomcat, creating tomcat/content/erddap .
Other Directory: For Red Hat Enterprise Linux (RHEL) or for other situations where
you want/need to put the ERDDAP content directory in some other location,
unzip erddapContent.zip into ~tomcat (or some other directory to which only tomcat has access)
and set the system property
erddapContentDirectory=~tomcat/content/erddap
in ~tomcat/conf/tomcat8.conf
so ERDDAP can find the content directory.
[Some previous versions are also available:
(size=24,996 bytes, MD5=8E375D13D388E0D117E3DC, dated )
(size=25,887 bytes, MD5=70B46B5E96DD17F4DCEAC27CC95BDFBB, dated )
(size=25,896 bytes, MD5=8E375D13D388E0D117E3DC, dated )
(size=26,759 bytes, MD5=97084B0ABE38D4DE1AD9, dated )
(size=26,760 bytes, MD5=8EFBD16CEBC249A2E21CFEB, dated )
(size=26,708 bytes, MD5=D70DF53AE971E719BC54, dated )
(size=23,878 bytes, MD5=DB5BF5AD, dated )
(size=23,878 bytes, MD5=9AB513BE1DFB6C9EA8DAA24A, dated )
the comments in tomcat/content/erddap/setup.xml
and make the requested changes.
setup.xml is the file with all of the settings which specify how your ERDDAP behaves.
For the initial setup, you MUST at least change these settings:
&bigParentDirectory&, &emailEverythingTo&, &baseUrl&,
&baseHttpsUrl&, &email.*&, &admin.*&
(and &fontFamily& if you are using Bitstream Vera Sans fonts).
When you create the bigParentDirectory, from the parent directory of
bigParentDirectory:
Make user=tomcat the owner of the bigParentDirectory, e.g.,
chown -R tomcat bigParentDirectory
Change the "group" to be tomcat, your username,
or the name of a small group that includes tomcat and
all the administrators of Tomcat/ERDDAP, e.g.,
chgrp -R yourUserName bigParentDirectory
Change permissions so that tomcat and the group have
read, write, execute privileges, e.g,.
chmod -R ug+rwx bigParentDirectory
Remove "other" user's permissions to read, write, or execute:
chmod -R o-rwx bigParentDirectory
reading possibly sensitive information in ERDDAP log files
and files with information about private datasets.
the comments in
, then modify the XML in
tomcat/content/erddap/datasets.xml to specify all of the datasets
you want your ERDDAP to serve.
(Unlikely) Now or (slightly more likely) in the future,
if you want to modify erddap's CSS file,
make a copy of tomcat/content/erddap/images/erddapStart.css called erddap.css
and then make changes to it.
Changes to erddap.css only take affect when ERDDAP is restarted
and often also require the user to clear the browser's cached files.
After you edit the .xml files, it is a good idea to verify that the result is well-formed XML
by pasting the XML text into an XML checker like
In the unusual situation where you aren't allowed to modify the Tomcat directory,
you can put the ERDDAP content directory somewhere else (e.g., /usr/local/erddap).
To let ERDDAP know where it is,
set the system property erddapContentDirectory=/usr/local/erddap/content/erddap/
(or wherever it is).
If you aren't allowed to set this property in startup.sh,
perhaps you can set it in Tomcat's context.xml.
the erddap.war file.
On Linux, Mac, and Windows, download
into tomcat/webapps .
(version 1.80, size=502,908,740 bytes, MD5=F09D55AAACF1FEF61AE8, dated )
The .war file is big because it contains high resolution coastline, boundary,
and elevation data needed to create maps.
[Some previous versions are also available.
(size=511,898,199 bytes, MD5=0E188C022F, dated )
(size=512,007,729 bytes, MD5=71CB248B05FCF71F3BD8E6, dated )
(size=511,995,047 bytes, MD5=0E919CBDAD014B724DE89F40C8CA9379, dated )
(size=556,163,762 bytes, MD5=3320ADB87E9F3AB31EBF, dated )
(size=527,743,058 bytes, MD5=4C8413E6CEE, dated )
(size=552,359,929 bytes, MD5=5976EDF01CC9D3ACF10EA0, dated )
(size=534,583,353 bytes, MD5=72F90E2C26E9E3806573CADD, dated )
(size=534,713,263 bytes, MD5=E8407EDC977C073CB816D873EF85320F, dated )
change the Apache timeout settings
so that time-consuming user requests don't timeout
(with what often appears as a "Proxy" or "Bad Gateway" error).
As the root user:
Modify the Apache httpd.conf file (usually in /etc/httpd/conf/ ):
Change the existing &Timeout& setting
(or add one at the end of the file)
to 3600 (seconds), instead of the default 60 or 120 seconds.
Change the existing &ProxyTimeout& setting
(or add one at the end of the file)
to 3600 (seconds), instead of the default 60 or 120 seconds.
Restart Apache: /usr/sbin/apachectl -k graceful
(but sometimes it is in a different directory).
ProxyPass so users don't have to put the port number, e.g., :8080, in the URL.
On Linux computers, if Tomcat is running in Apache, please modify the
Apache httpd.conf file
(usually in /etc/httpd/conf/ )
to allow HTTP traffic to/from ERDDAP without requiring the port number, e.g., :8080, in the URL.
As the root user:
Modify the existing &VirtualHost& tag (if there is one),
or add one at the end of the file:
&VirtualHost *:80&
ServerName YourDomain.org
ProxyRequests Off
ProxyPreserveHost On
ProxyPass /erddap http://localhost:8080/erddap
ProxyPassReverse /erddap http://localhost:8080/erddap
&/VirtualHost&
Then restart Apache: /usr/sbin/apachectl -k graceful
(but sometimes it is in a different directory).
(I don't recommend using the Tomcat Web Application Manager. If you don't
fully shutdown and startup Tomcat, sooner or later you will have PermGen
memory issues.)
(In Linux or Mac OS, if you have created a special user to run Tomcat,
e.g., tomcat, remember to do the following steps as that user.)
If Tomcat is already running, shut down Tomcat
with (in Linux or Mac OS) tomcat/bin/shutdown.sh
or (in Windows)
tomcat\bin\shutdown.bat
On Linux, use ps -ef | grep tomcat before and after shutdown.sh
to make sure the tomcat process has stopped.
The process should be listed before the shutdown and eventually not listed
after the shutdown. It may take a minute or two for ERDDAP to fully shut down.
Be patient.
Or if it looks like it won't stop on its own, use:
kill -9 processID
Start Tomcat with (in Linux or Mac OS) tomcat/bin/startup.sh
or (in Windows) tomcat\bin\startup.bat
Use a browser to try to view http://www.YourServer.org/erddap/
ERDDAP starts up without any datasets loaded.
Datasets are loaded in a background thread and so become available one-by-one.
If ERDDAP won't run or some datasets won't load, look for error messages in
(tomcat/logs/catalina.out) and
(bigParentDirectory/logs/log.txt).
If you are using a version of Java that is too old for ERDDAP,
ERDDAP won't run and you will see an error message in Tomcat's log file like
Exception in thread "main" java.lang.UnsupportedClassVersionError:
some/class/name: Unsupported major.minor version someNumber
The solution is to update to the most recent version of Java
and make sure that Tomcat is using it.
Tomcat has to do a lot of work the first time an application like
ERDDAP is started.
On some servers, the first attempt to view ERDDAP stalls (30 seconds?) until
this work is finished.
On other servers, the first attempt will fail immediately. But if you wait
30 seconds and try again,
it will succeed if ERDDAP was installed correctly.
There is no fix for this, but it usually only occurs the first time after
you install a new version of ERDDAP.
If Tomcat throws warnings during startup about
"org.apache.catalina.webresources.Cache.getResource Unable to add the resource
at someFile to the cache because there was insufficient free space
available after evicting expired cache entries - consider increasing the
maximum size of the cache",
the solution is:
In your [tomcat]/conf/context.xml, add this block right before &/Context& :
&Resources cachingAllowed="true" cacheMaxSize="100000" /&
(or some other number) (The units for cacheMaxSize are KB.)
For more information see
In the future, to shut down (and restart) ERDDAP, see
Troubles installing Tomcat or ERDDAP?
Email me at bob dot simons at noaa dot gov .
I will help you.
Or, you can join the
and post your question there.
of New Versions of ERDDAP
If you want to receive an email whenever a new version of ERDDAP
is available, send an email to bob dot simons at noaa dot gov
requesting to be added to the ERDDAP Announcements mailing list.
This list averages roughly one email every three months.
your ERDDAP to highlight your organization (not ERD).
To change the banner that appears at the top of all ERDDAP .html pages,
change the HTML content specified in the setup.xml file in the &startBodyHtml& tag.
For example, you could:
Use a different image (i.e., your organization's logo).
Change the background color.
Change "ERDDAP" to "YourOrganization's ERDDAP"
Change "Easier access to scientific data" to "Easier access to YourOrganization's data".
Change the "Brought to you by" links to be links to your organization and funding sources.
To change the information on the left side of the home page, e.g., to
information about your project, change the HTML content specified in the setup.xml
file in the &theShortDescriptionHtml& tag.
To change the icon that appears on browser tabs, put your organization's favicon.ico in
tomcat/content/erddap/images/ .
To Do an Update of an Existing ERDDAP on Your Server
Make the changes listed in
in the section entitled "Things ERDDAP Administrators Need to Know and Do"
for all of the ERDDAP versions since the version you were using.
into tomcat/webapps .
(version 1.80, size=502,908,740 bytes, MD5=F09D55AAACF1FEF61AE8, dated )
Common: If you are upgrading from ERDDAP version 1.46 (or above)
and you just use the standard messages,
the new standard messages.xml will be installed automatically
(amongst the .class files via erddap.war).
Rare: If you are upgrading from ERDDAP version 1.44 (or below),
you MUST delete the old messages.xml file:
tomcat/content/erddap/messages.xml .
The new standard messages.xml will be installed automatically
(amongst the .class files via erddap.war).
Rare: If you always make changes to the standard messages,
you need to make those changes to the new messages.xml file (which is
WEB-INF/classes/gov/noaa/pfel/erddap/util/messages.xml after erddap.war
is decompressed by Tomcat).
Rare: If you maintain a custom messages.xml file
in tomcat/content/erddap/,
you need to figure out (via diff) what changes have been made to the default messages.xml
(which are in the new erddap.war as
WEB-INF/classes/gov/noaa/pfel/erddap/util/messages.xml)
and modify your custom messages.xml file accordingly.
Install the new ERDDAP in Tomcat:
* Don't use Tomcat Manager. Sooner or later there will be PermGen memory issues.
It is better to actually shutdown and startup Tomcat.
* Replace references to tomcat below with the actual
Tomcat directory on your computer.
For Linux and Macs:
Shutdown Tomcat:
From a command line, use: tomcat/bin/shutdown.sh
And use ps -ef | grep tomcat to see if/when
the process has been stopped.
(It may take a minute or two.)
Remove the decompressed ERDDAP installation:
In tomcat/webapps, use
rm -rf erddap
Delete the old erddap.war file:
In tomcat/webapps, use rm erddap.war
Copy the new erddap.war file from the temporary directory to
tomcat/webapps
Restart Tomcat and ERDDAP:
use tomcat/bin/startup.sh
View ERDDAP in your browser to check that the restart succeeded.
(Often, you have to try a few times and wait a minute before you see ERDDAP.)
For Windows:
Shutdown Tomcat:
From a command line, use: tomcat\bin\shutdown.bat
Remove the decompressed ERDDAP installation:
In tomcat/webapps, use
del /S/Q erddap
Delete the old erddap.war file:
In tomcat\webapps, use del erddap.war
Copy the new erddap.war file from the temporary directory to
tomcat\webapps
Restart Tomcat and ERDDAP: use tomcat\bin\startup.bat
View ERDDAP in your browser to check that the restart succeeded.
(Often, you have to try a few times and wait a minute before you see ERDDAP.)
Troubles updating ERDDAP?
Email me at bob dot simons at noaa dot gov .
I will help you.
Or, you can join the
and post your question there.
Ctrl-F To Find Things On This Web Page
All of the information about administering ERDDAP
(other than
is on this one, very long, .html web page,
not several .html pages as some people prefer.
The advantage of one .html web page is that you can use Ctrl-F (Command-F on a Mac)
in your web browser to search for text (for example, flag) within this web page.
Alternatively, at the top of this document, there is a
(a Table of Contents).
Sometime, a request to ERDDAP will return a Proxy Error,
an HTTP 502 Bad Gateway Error, or some similar error.
These errors are being thrown by Apache or Tomcat, not ERDDAP itself.
If every request generates these errors, especially when you are first
setting up your ERDDAP, then it probably is a proxy or bad gateway error,
and the solution is probably to fix
This may also be the problem when an established ERDDAP suddenly starts
throwing these errors for every request.
Otherwise, "proxy" errors are usually actually time out errors thrown by Apache or Tomcat.
Even when they happen relatively quickly, it is some sort of response from
Apache or Tomcat that occurs when ERDDAP is very busy, memory-limited,
or limited by some other resource.
In these cases, see the advice below to deal with
Requests for a long time range (&30 time points)
from a gridded dataset are prone to time out failures,
which often appear as Proxy Errors,
because it takes significant time for ERDDAP to open all of the data files one-by-one.
If ERDDAP is otherwise busy during the request, the problem is more likely to occur.
If the dataset's files are compressed, the problem is more likely to occur,
although it's hard for a user to determine if a dataset's files are compressed.
The solution is to make several requests, each with a smaller time range.
How small of a time range?
I suggest starting really small (~30 time points?),
then (approximately) double the time range until the request fails,
then go back one doubling.
Then make all the requests (each for a different chunk of time)
needed to get all of the data.
An ERDDAP administrator can lessen this problem by increasing the
or if just certain requests are responding slowly,
you may be able to figure out if the slowness is reasonable
and temporary (e.g., because of lots of requests from scripts or WMS users),
or if something is inexplicably wrong and you need to
If ERDDAP is responding slowly, see the advice below to determine the
cause, which hopefully wil enable you to fix the problem.
You may have a specific starting point (e.g., a specific request URL)
or a vague starting point (e.g., ERDDAP is slow).
You may know the user involved (e.g., because they emailed you), or not.
You may have other clues, or not.
Since all of these situations and all of the possible causes of the problems
blur together, the advice below tries to deal will all possible
starting points and all possible problems related to slow responses.
Look for clues in
(bigParentDirectory/logs/log.txt).
[On rare occasions, there are clues in
(tomcat/logs/catalina.out).]
Look for error messages.
Look for a large number of requests coming from one (or a few) users and perhaps
hogging a lot of your server's resources (memory, CPU time, disk access,
internet bandwidth).
If the trouble is tied to one user,
you can often get a clue about who the user is via web services like
that can give you information related to the user's IP address
(which you can find in ERDDAP's log.txt file).
If the user seems to be a bot bahaving badly
(notably, a search engine trying to fill out the ERDDAP forms
with every possible permutation of entry values), make sure you have
properly set up your server's
If the user seems to be a script(s) that is making multiple simultaneous
contact the user, explain that your ERDDAP has limited resources
(e.g., memory, CPU time, disk access, internet bandwidth),
and ask them to be considerate of other users
and just make one request at a time.
You might also mention that you will blacklist them if they don't back off.
If the user seems to be a script
making a large number of time-consuming requests,
ask the user to be considerate of other users by putting
a small pause (2 seconds?) in the script between requests.
WMS client software can be very demanding. One client will often
ask for 6 custom images at a time.
If the user seems to be a WMS client that is making legitimate requests,
Ignore it. (recommended, because they'll move on pretty soon)
Turn off your server's WMS service via ERDDAP's setup.html file. (not recommended)
If the requests seem stupid, insane, excessive, or malicious,
or if you can't resolve the problem any other way,
consider temporarily or permanently adding the user's IP address to the
Try to duplicate the problem yourself, from your computer.
Figure out if the problem is with one dataset or all datasets,
for one user or all users, for just certain types of requests, etc..
If you can duplicate the problem, try to narrow down the problem.
If you can't duplicate the problem, then the problem may be tied to the
user's computer, the user's internet connection, or your institution's
internet connection.
If just one dataset is responding slowly
(perhaps only for one type of request from one user),
the problem may be:
ERDDAP's access to the dataset's source data
(notably from relational databases, Cassandra, and remote datasets)
may be temporarily or permanently slow.
Try to check the source's speed independent of ERDDAP.
If it is slow, perhaps you can improve it.
Is the problem related to the specific request or general type of request?
The larger the requested subset of a dataset, the more likely the
request will fail. If the user is making huge requests,
ask the user to make smaller requests
that are more likely to get a fast and successful response.
Almost all data sets are better at handling some types of requests
than others types of requests.
For example, when a dataset stores different time chunks in different files,
requests for data from a huge number of time points may be very slow.
If the current requests are of a difficult type,
consider offering a variant of the dataset that is optimized for these requests.
Or just explain to the user that that type of request is difficult and
time consuming, and ask for their patience.
The dataset may be not optimally configured. You may be able to make changes to the dataset's
datasets.xml chunk to help ERDDAP handle the dataset better. For example,
EDDGridFromNcFiles datasets that access data from compressed nc4/hdf5 files are
slow when getting data for the entire geographic range (e.g., for a world map)
because the entire file must be decompressed.
You could convert the files to uncompressed files, but then the disk space
requirement will be much, much larger. It is probably better to just
accept that such datasets will be slow in certain circumstances.
The configuration of the
tag has a huge influence on how ERDDAP handles EDDTable datasets.
You may be able to increase the
Many EDDTable datasets can be sped up by
which ERDDAP can read very quickly.
If you want help speeding up a specific dataset,
email a description of the problem and the dataset's chunk of datasets.xml to
bob dot simons at noaa dot gov.
Or, you can join the
and post your question there.
If everything in ERDDAP is always slow,
the problem may be:
The computer that is runnning ERDDAP may not have enough memory or processing power.
It is good to run ERDDAP on a modern, multi-core server.
For heavy use, the server should have a 64-bit operating system and 8 GB or more of memory.
The computer that is runnning ERDDAP may be also running other applications that
are consuming lots of system resources. If so, can you get a dedicated server
for ERDDAP? For example (this is not an endorsement), you can get
a quad-core Mac Mini Server with 8 GB of memory for ~$1100.
If everything in ERDDAP is temporarily slow, view your ERDDAP's
in your browser.
Does the ERDDAP status page fail to load?
Did the ERDDAP status page load slowly (e.g., &5 seconds)?
That is a sign that everything in ERDDAP is running slowly,
but it isn't necessarily trouble. ERDDAP may just be really busy.
For "Response Failed Time (since last major LoadDatasets)", is n= a large number?
That indicates there have been lots of failed requests recently.
That may be trouble or the start of trouble.
The median time for the failures is often large (e.g., 210000 ms),
which means that there were (are?) lots of active threads.
which were tying up lots of resources (like memory, open files, open sockets, ...),
which is not good.
For "Response Succeeded Time (since last major LoadDatasets)", is n= a large number?
That indicates there have been lots of successful requests recently.
This isn't trouble. It just means your ERDDAP is getting heavy use.
Is the "Number of non-Tomcat-waiting threads" double a typical value?
This is often serious trouble that will cause ERDDAP to slow down and eventually
If this persists for hours, you may want to proactively
At the bottom of the "Memory Use Summary" list, is the last
"Memory: currently using" value very high?
That may just indicate high usage, or it may be a sign of trouble.
Look at the list of threads and their status.
Are an unusual number of them doing something unusual?
Is your institution's internet connection currently slow?
Search the internet for "internet speed test"
and use one of the free online tests, such as
If your institution's internet connection is slow,
then connections between ERDDAP and remoted data sources will be slow,
and connections between ERDDAP and the user will be slow.
Sometimes, you can solve this by stopping unnecessary internet use
(e.g., people watching streaming videos or on video conference calls).
Is the user's internet connection currently slow?
Have the user search the internet for "internet speed test"
and use one of the free online tests, such as
If the user's internet connection is slow, it slows down their access to ERDDAP.
Sometimes, they can solve this by stopping unnecessary internet use at
their institution
(e.g., people watching streaming videos or on video conference calls).
Email all the details to bob.simons at noaa.gov .
Down and Restart Tomcat and ERDDAP
You don't need to shut down and restart Tomcat and ERDDAP
if ERDDAP is temporarily slow, slow for some known reason
(like lots of requests from scripts or WMS users),
or to apply changes to datasets.xml file.
You do need to shut down and restart Tomcat and ERDDAP
if you need to apply changes to the setup.xml file,
or if ERDDAP freezes, hangs, or locks up.
In extreme circumstances, Java may freeze for a minute or two
while it does a full garbage collection, but then recover. So it is good to
wait a minute or two to see if Java/ERDDAP is really frozen or
if it is just doing a long garbage collection.
(If garbage collection is a common problem,
I don't recommend using the Tomcat Web Application Manager to
start or shutdown Tomcat.
If you don't fully shutdown and startup Tomcat, sooner or later you will
have PermGen memory issues.
To shutdown and restart Tomcat and ERDDAP:
If you use Linux or a Mac:
(If you have created a special user to run Tomcat, e.g., tomcat,
remember to do the following steps as that user.)
Use cd tomcat/bin
Use ps -ef | grep tomcat to find the java/tomcat processID
(hopefully, just one process will be listed),
which we'll call javaProcessID below.
If ERDDAP is frozen/hung/locked up, use kill -3 javaProcessID
to tell Java (which is running Tomcat)
to do a thread dump to the Tomcat log file: tomcat/logs/catalina.out .
After you reboot, you can diagnose the problem by finding the thread
dump information
(and any other useful information above it) in tomcat/logs/catalina.out
and also by reading relevant parts of the
If you want, you can email that information to
bob dot simons at noaa dot gov
so I can see what went wrong.
Or, you can join the
and post your question there.
Use ./shutdown.sh
Use ps -ef | grep tomcat repeatedly until the
java/tomcat process isn't listed.
Sometimes, the java/tomcat process will take up to two minutes to fully
shut down.
The reason is: ERDDAP sends a message to its background threads
to tell them to stop, but sometimes it takes these threads a long time
to get to a good stopping place.
If you don't want to wait for java/tomcat to stop by itself,
you can use
kill -9 javaProcessID
to force the java/tomcat process to stop immediately.
If possible, use this only as a last resort.
The -9 switch is powerful, but it may cause various problems.
To restart ERDDAP, use ./startup.sh
View ERDDAP in your browser to check that the restart succeeded.
(Sometimes, you need to wait 30 seconds and try to load ERDDAP again
in your browser for it to succeed.)
If you use Windows:
Use cd tomcat/bin
Use shutdown.bat
You may want/need to use the Windows Task Manager (accessible via Ctrl Alt Del)
to ensure that the Java/Tomcat/ERDDAP process/application has fully stopped.
Sometimes, the process/application will take up to two minutes to shut down.
The reason is: ERDDAP sends a message to its background threads
to tell them to stop, but sometimes it takes these threads a long time
to get to a good stopping place.
To restart ERDDAP, use startup.bat
View ERDDAP in your browser to check that the restart succeeded.
(Sometimes, you need to wait 30 seconds and try to load ERDDAP again
in your browser for it to succeed.)
If ERDDAP becomes slow, crashes or freezes, something is wrong.
to try to figure out the cause. If you can't,
please email the details to bob dot simons at noaa dot gov .
The most common problem is a troublesome user who is
running several scripts at once and/or someone making a large number of invalid requests.
If this happens, you should probably blacklist that user.
When a blacklisted user makes a request, the error message in the response encourages
them to email you to work out the problems. Then, you can encourage them
to run just one script at a time and to fix the problems in their script
(e.g., requesting data from a remote dataset that can't respond before timing out).
In extreme circumstances, Java may freeze for a minute or two
while it does a full garbage collection, but then recover. So it is good to
wait a minute or two to see if Java/ERDDAP is really frozen or
if it is just doing a long garbage collection.
(If garbage collection is a common problem,
If ERDDAP becomes slow or freezes and the problem isn't a troublesome user
or a long garbage collection,
you can usually solve the problem by
My experience is that ERDDAP can run for months without needing a restart.
You can monitor your ERDDAP's status by looking at the ,
notably the statistics in the top section.
If ERDDAP becomes slow or freezes and the problem isn't just extremely heavy usage,
you can usually solve the problem by
My experience is that ERDDAP can run for months without needing a restart.
If you need to restart ERDDAP frequently, something is wrong.
to try to figure out the cause. If you can't,
please email the details to bob dot simons at noaa dot gov .
As a temporary solution, you might try using
to monitor your ERDDAP and restart it if needed.
Or, you could make a cron job to restart ERDDAP (proactively) periodically.
It may be a little challenging to write a script to automate monitoring and restarting ERDDAP.
Some tips that might help:
You can simplify testing if the Tomcat process is still running
by using the -c switch with grep:
ps -u tomcatUser | grep -c java
That will reduce the output to "1" if the tomcat process is still alive,
or "0" if the process has stopped.
If you are good with gawk, you can extract the processID from the results of
ps -u tomcatUser | grep java, and use the
processID in other lines of the script.
If you do set up Monit or a cron job, please email the details to
bob dot simons at noaa dot gov .
Or, you can join the
and share the information there.
If you repeatedly use Tomcat Manager to Reload (or Stop and Start) ERDDAP,
ERDDAP may fail to start up and throw java.lang.OutOfMemoryError: PermGen.
The solution is to periodically (or every time?)
instead of just reloading ERDDAP.
[Update: This problem should be greatly minimized or fixed in ERDDAP version 1.24.]
If ERDDAP doesn't start up or if something isn't working as expected,
it is very useful to look at the error and diagnostic messages in the ERDDAP log file.
The log file is bigParentDirectory/logs/log.txt
(bigParentDirectory is specified in ).
If there is no log.txt file or if the log.txt file hasn't been updated since
you restarted ERDDAP, look in the
to see if there is an error message there.
Types of diagnostic messages in the log file:
The word "error" is used when something went so wrong that the procedure failed to complete.
Although it is annoying to get an error, the error forces you to deal with the problem.
Our thinking is that it is better to throw an error, than to have ERDDAP hobble along,
working in a way you didn't expect.
The word "warning" is used when something went wrong, but the procedure was able to complete.
These are pretty rare.
Anything else is just an informative message.
You can control how much information is logged with &logLevel& in
Information is written to the log file on the disk drive in fairly big chunks.
The advantage is that this is very efficient -- ERDDAP will never block
waiting for information to be written to the log file.
The disadvantage is that the log will almost always end with a
partial message, which won't be completed until the next chunk is written.
You can make it up-to-date (for an instant) by viewing your ERDDAP's
status web page at http://your.domain.org/erddap/status.html .
When the log.txt files gets to 20 MB,
the file is renamed log.txt.previous and a new log.txt file is created.
So log files don't accumulate.
In setup.xml, you can specify a different maximum size for the log file, in MegaBytes.
The minimum allowed is 1 (MB). The maximum allowed is 2000 (MB).
The default is 20 (MB). For example:
&logMaxSizeMB&20&/logMaxSizeMB&
Whenever you restart ERDDAP,
ERDDAP makes an archive copy of the
log.txt and log.txt.previous files with a time stamp in the file's name.
If there was trouble before the restart,
it may be useful to analyze these archived files for clues as to what the trouble was.
You can delete the archive files if they are no longer needed.
If ERDDAP doesn't start up because an error occured very early in ERDDAP's startup,
the error message will show up in Tomcat's log files
(tomcat/logs/catalina.today.log or
tomcat/logs/catalina.out),
ERDDAP always writes the text of all out-going email messages
in the current day's emailLogYEAR-MM-DD.txt file in bigParentDirectory/logs
(bigParentDirectory is specified in ).
If the server can't send out email messages, or if you have configured ERDDAP not to send
out email messages, or if you are just curious, this file is a convenient way to see
all of the email messages that have been sent out.
You can delete previous days' email log files if they are no longer needed.
The Daily Report has lots of useful information -- all of the information from
your ERDDAP's
It is the most complete summary of your ERDDAP's status.
Among other statistics, it includes a list of datasets that didn't load and the exceptions
they generated.
It is generated when you start up ERDDAP (just after ERDDAP finishes trying to load all of
the datasets) and generated soon after 7 am local time every morning.
Whenever it is generated, it is written to .
Whenever it is generated, it is emailed to &emailDailyReportsTo& and
&emailEverythingTo& (which are specified in
provided you have set up the email system (in setup.xml).
You can view the status of your ERDDAP from any browser by going to
&baseUrl&/erddap/status.html
This page is generated dynamically, so it always has up-to-the-moment statistics for your ERDDAP.
It includes statistics regarding the number of requests, memory usage, thread stack traces,
the taskThread, etc.
Because the Status Page can be viewed by anyone, it doesn't include quite as much information
Datasets -
ERDDAP usually rereads datasets.xml every loadDatasetsMinMinutes
(specified in ).
So you can make changes to datasets.xml any time, even while ERDDAP is running.
A new dataset will be detected soon, usually within
loadDatasetsMinMinutes.
A changed dataset will be reloaded when it is reloadEveryNMinutes
old (as specified in datasets.xml).
ERDDAP to Try to
a Dataset As Soon As Possible
ERDDAP won't notice any changes to a dataset's setup in datasets.xml until ERDDAP reloads the dataset.
To tell ERDDAP to reload a dataset as soon as possible
(before the dataset's &reloadEveryNMinutes& would cause it to be reloaded),
put a file in bigParentDirectory/flag
(bigParentDirectory is specified in ) that has the
same name as the dataset's datasetID.
This tells ERDDAP to try to load new and changed datasets.
The old version of the dataset will remain available to users until
the new version is available and swapped atomically into place.
For EDDGridFromFiles and EDDTableFromFiles, the reload is a will look for
new or changed files, read those, and incorporate them into the dataset.
So the time to reload is dependent on the number of new or changed files.
If the dataset has active="false", ERDDAP will remove the dataset.
There is a variant of the /flag directory: the /hardFlag directory. (Added in ERDDAP v1.74.)
If you put a file in bigParentDirectory/hardFlag /a>) that has the
same name as the dataset's datasetID,
as soon as ERDDAP sees the hardFlag file,
ERDDAP will:
Delete the hardFlagFile.
Remove the dataset from ERDDAP.
Delete all of the information that ERDDAP has stored
about this dataset.
For EDDGridFromFiles and EDDTableFromFiles subclasses,
this deletes the internal database of data files and their contents.
For datasets like EDDGridSideBySide that have childDatasets,
this also deletes the internal database of data files and their contents
for all child datasets.
Reload the dataset.
For EDDGridFromFiles and EDDTableFromFiles subclasses,
this causes ERDDAP to reread all of the data files.
Thus, the reload time is dependent on the total number of data files in the dataset.
Because the dataset was removed from ERDDAP when the hardFlag was noticed,
the dataset will be unavailable until the dataset finishes reloading.
Be patient. Look in the log.txt file if you want to see what's going on.
The hardFlag variant deletes the dataset's stored information
even if the dataset isn't currently loaded in ERDDAP.
HardFlags are very useful when you do something that causes a
change in how ERDDAP reads and interprets the source data, for example,
when you install a new version of ERDDAP or when you have made a change to a
dataset's definition in datasets.xml
The contents of the flag and hardFlag file are irrelevant.
In between major dataset reloads, ERDDAP looks continuously for flag and hardFlag files.
Note that when a dataset is reloaded, all files in the
bigParentDirectory//datasetID
directory are deleted. This includes .nc and image files that are normally cached
for ~15 minutes.
Note that if the dataset's xml includes
a flag will cause the dataset to be made inactive
(if it is active), and in any case, not reloaded.
Any time ERDDAP runs LoadDatasets to do a major reload (the timed reload
controlled by &loadDatasetsMinMinutes&) or a minor reload
(as a result of an external or internal flag),
ERDDAP reads all &user&, &requestBlacklist&, &slowDownTroubleMillis&,
and &subscriptionEmailBlacklist& tags and switches to the new settings.
So you can use a flag as a way to get ERDDAP to notice changes
to those tags ASAP.
has a web service so that flags can be set via URLs.
For example,
https://coastwatch.pfeg.noaa.gov/erddap/setDatasetFlag.txt?datasetID=rPmelTao&flagKey=
(that's a fake flagKey) will set a flag for the rPmelTao dataset.
There is a different flagKey for each datasetID.
Administrators can see a list of flag URLs for all datasets by looking at the bottom of
Administrators should treat these URLs as confidential, since they give someone the right to
reset a dataset at will.
If you think the flagKeys have fallen into the hands of someone who is abusing them,
you can change &flagKeyKey& in
restart ERDDAP
to force ERDDAP to generate and use a different set of flagKeys.
If you change &flagKeyKey&, delete all of the old subscriptions (see
the list in your Daily Report) and remember to send the new URLs to the
people who you do want to have them.
The flag system can serve as the basis for a more efficient mechanism for telling ERDDAP when to
reload a dataset. For example, you could set a dataset's &reloadEveryNMinutes&
to a large number (e.g., 10080 = 1 week).
Then, when you know the dataset has changed (perhaps because you added a file to the dataset's data
directory), set a flag so that the dataset is reloaded as soon as possible.
Flags are usually seen quickly.
But if the LoadDatasets thread is already
busy, it may be a while before it is available to act on the flag.
But the flag system is much more responsive and much more efficient than setting
&reloadEveryNMinutes& to a small number.
Force Dataset
If a dataset is active in ERDDAP, and you want to deactivate it as soon as
possible, add
to the dataset tag and set a .
Flags are usually seen quickly.
But if the LoadDatasets thread is already
busy, it may be a while before it is available to act on the flag.
Once the dataset is not active (i.e., not visible in ERDDAP's list of datasets),
you can remove the dataset's description from the datasets.xml file if you want to.
A thread called RunLoadDatasets is the master thread that controls
when datasets are reloaded. RunLoadDatasets loops forever:
RunLoadDatasets notes the current time.
RunLoadDatasets starts a LoadDatasets thread to do a "majorLoad".
You can see information about the current/previous majorLoad at the top of
your ERDDAP's
LoadDatasets makes a copy of datasets.xml.
LoadDatasets reads through the copy of datasets.xml and, for each dataset,
sees if the dataset needs to be (re)loaded or removed.
file exists for this dataset,
the file is deleted and
the dataset is removed if active="false"
or (re)loaded if active="true" (regardless of the dataset's age).
If the dataset's dataset.xml chunk has active="false"
and the dataset is currently loaded (active), is is unloaded (removed).
If the dataset has active="true" and the dataset isn't already loaded,
it is loaded.
If the dataset has active="true" and the dataset is already loaded,
the data set is reloaded if the dataset's age (time since last load) is greater than
its &reloadEveryNMinutes& (default = 10080 minutes),
otherwise, the dataset is left alone.
LoadDatasets finishes.
The RunLoadDatasets thread waits for the LoadDatasets thread to finish.
If LoadDatasets takes longer than loadDatasetsMinMinutes
(as specified in setup.xml), RunLoadDatasets interrupts the LoadDatasets thread.
Ideally, LoadDatasets notices the interrupt and finishes.
But if it doesn't notice the interrupt within a minute,
RunLoadDatasets calls loadDatasets.stop(), which is undesirable.
While the time since the start of the last majorLoad is less than
loadDatasetsMinMinutes (as specified in setup.xml, e.g., 15 minutes),
RunLoadDatasets repeatedly looks for
in the bigParentDirectory/flag directory.
If one or more flag files are
found, they are deleted, and RunLoadDatasets starts a LoadDatasets thread to do a
"minorLoad" (majorLoad=false).
You can't see minorLoad information on
your ERDDAP's .
LoadDatasets makes a copy of datasets.xml.
LoadDatasets reads through the copy of datasets.xml and,
for each dataset for which there was a flag file:
If the dataset's dataset.xml chunk has active="false"
and the dataset is currently loaded (active), is is unloaded (removed).
If the dataset has active="true", the dataset is (re)loaded,
regardless of its age.
Non-flagged datasets are ignored.
LoadDatasets finishes.
RunLoadDatasets goes back to step 1.
When you restart ERDDAP, every dataset with active="true" is loaded.
When a dataset is (re)loaded, its cache (including any data response files
and/or image files) is emptied.
If you have a lot of datasets and/or one or more datasets are slow to (re)load,
a LoadDatasets thread may take a long to finish its work,
perhaps even longer than loadDatasetsMinMinutes.
There is never more than one LoadDatasets thread running at once.
If a flag is set when LoadDatasets is already running,
the flag probably won't be noticed or acted on until that LoadDatasets thread finishes running.
You might say: "That's stupid. Why don't you just start a bunch of new threads
to load datasets?"
But if you have lots of datasets which get data from
one remote server, even one LoadDatasets thread will put substantial stress
on the remote server.
The same is true if you have lots of datasets which get data from
files on one RAID.
There are rapidly diminishing returns from having
more than one LoadDatasets thread.
Setting a flag just signals that the dataset should be (re)loaded
as soon as possible, not necessarily immediately.
If no LoadDatasets thread is currently running, the dataset will start to be reloaded
within a few seconds.
But if a LoadDatasets thread is currently running, the dataset probably won't be reloaded
until after that LoadDatasets thread is finished.
In general, if you put a flag file in the bigParentDirectory/erddap/flag
directory (by visiting the dataset's flagUrl or putting an actual file there), the
dataset will usually be reloaded very soon after that flag file is deleted.
If you have some external way of knowing when a dataset needs to be reloaded
and if it is convenient for you,
the best way to make sure that a dataset is always up-to-date is to
set its reloadEveryNMinutes to a large number (10080?) and
set a flag (via a script?) whenever it needs to be reloaded.
That is the system that EDDGridFromErddap and EDDTableFromErddap use
receive messages that the dataset needs to be reloaded.
Lots of relevant information is written to the
bigParentDirectory/logs/log.txt file.
If things aren't working as you expect, looking at log.txt
lets you diagnose the problem by finding out exactly what ERDDAP did.
Search for "majorLoad=true" for the start of major LoadDataset threads.
Search for "majorLoad=false" for the start of minor LoadDatasets threads.
Search for a given dataset's datasetID for information about it being
(re)loaded or queried.
In general, ERDDAP doesn't cache (store) responses to user requests.
The rationale was that most requests would be slightly different so the cache wouldn't be very effective.
The biggest exceptions are requests for image files
(which are cached since browsers and programs like Google Earth often re-request images)
and requests for .nc files (because they can't be created on-the-fly).
ERDDAP stores each dataset's cached files in a different directory:
bigParentDirectory/cache/datasetID
since a single cache directory might have a huge number of files which might become slow to access.
Files are removed from the cache for one of three reasons:
All files in this cache are deleted when ERDDAP is restarted.
Periodically, any file more than &cacheMinutes& old (as specified in
) will be deleted.
Removing files in the cache based on age (not Least-Recently-Used) ensures that files won't stay
in the cache very long.
Although it might seem like a given request should always return the same response, that isn't true.
For example, a tabledap request which includes &time>someTime will change if
new data arrives for the dataset.
And a griddap request which includes [last] for the time dimension will change if new
data arrives for the dataset.
Images showing error conditions are cached, but only for a few minutes (it's a difficult situation).
Every time a dataset is reloaded, all files in that dataset's cache are deleted.
Because requests may be for the "last" index in a gridded dataset, files in the cache may become
invalid when a dataset is reloaded.
For all types of datasets, ERDDAP gathers lots of information when a dataset
is loaded and keeps that
in memory. This allows ERDDAP to respond very quickly to searches, requests
for lists of datasets,
and requests for information about a dataset.
For a few types of datasets (notably EDDGridCopy, EDDTableCopy,
EDDGridFromXxxFiles, and
EDDTableFromXxxFiles), ERDDAP stores on disk some information about the
dataset that is reused
when the dataset is reloaded. This greatly speeds the reloading process.
Some of the dataset information files are human-readable .json files and are stored in
bigParentDirectory/dataset/last2LettersOfDatasetID/datasetID .
ERDDAP only deletes these files in unusual situations, e.g., if you add
or delete a variable from the dataset's datasets.xml chunk.
Most changes to a dataset's datasets.xml chunk (e.g., changing a
global attribute or a variable attribute) don't necessitate that you delete these files.
A regular dataset reload will handle these types of changes.
You can tell ERDDAP to reload a dataset ASAP by setting a
for the dataset.
Similarly, the addition or deletion of files will be handled when
ERDDAP reloads a dataset.
But ERDDAP will notice this type of change soon and automatically
if the dataset is using the
It should only rarely be necessary for you to delete these files.
The most common situation where you need to force ERDDAP to delete the stored
information (because it is out-of-date/incorrect and won't be automatically
fixed by ERDDAP)
is when you make changes
to the dataset's datasets.xml chunk that affect how ERDDAP interprets
data in the source data files, for example, changing the time variable's format string.
To delete a dataset's stored information files from an ERDDAP that is running
(even if the dataset isn't currently loaded), set a
for that dataset.
Remember that if a dataset is an aggregation of a large number of files,
reloading the dataset may take considerable time.
To delete a dataset's stored information files when ERDDAP isn't running, run
for that dataset
(which is easier than figuring in which directory the info is located and
deleting the files by hand).
Remember that if a dataset is an aggregation of a large number of files,
reloading the dataset may take considerable time.
You will probably get an email from ERDDAP periodically with the subject
"Unusual Activity: >25% of requests failed". It is worth figuring out what is going wrong.
The only way to figure out what is going wrong is to read the
[bigParentDirectory]/logs/log.txt file and search for "error".
For each error message, look just above it to find the line where the original request is logged.
The line starts with "{{{{#" and includes the user's IP number and the actual request.
Use the user's IP number (for example, with
to try to figure who or what the user is.
Sometimes that will tell you pretty accurately who the user is (e.g., it's a search engine's web crawler).
Most of the time it just gives you a clue
(e.g., it's an amazonaws computer, it's from some university,
it's someone in Washington State ( I like Washington)).
By looking at the actual request, the IP number, and the error message (all from log.txt) for
a series of errors, you can usually figure out basically what is going wrong.
In my experience, there are three common causes of lots of failed requests:
1) A search engine is naively trying the URLs listed in ERDDAP web pages and
ISO 19115 documents. For example, there are many places which list the base OPeNDAP URL, for example,
https://coastwatch.pfeg.noaa.gov/erddap/griddap/jplMURSST, to which the user is
supposed to add a file type (e.g., .das, .dds, .html).
But the search engine doesn't know this. And the request to the base URL fails.
A related situation is when the search engine generates bizarre requests or
tries to fill out forms in order to get to "hidden" web pages.
But the search engines often do a bad job of this, leading to failures.
The solution is: create a
2) Some user is running a script that is repeatedly asking for something that isn't there.
Maybe it is a dataset that used to exist, but is gone now (temporarily or permanently).
Scripts often don't expect this and so don't deal with it intelligently.
So the script just keeps making requests and the requests keep failing.
If you can guess who the user is (from the IP number above), contact them
and tell them the dataset is no longer available and ask them to change their script.
3) Something is really wrong with some dataset. Usually, ERDDAP will
make the troubled dataset inactive. Sometimes it doesn't, so all the requests
to it just lead to errors. If so, fix the problem with the dataset or (if you can't)
set the dataset to
Of course, this may lead to problem #2.
Sometimes the errors aren't so bad, notably, if ERDDAP can detect the error
and respond very quickly (&=1ms). So you may decide to take no action.
If all else fails, there is a universal solution:
add the user's IP number to the .
This isn't as bad or as drastic an option as it might seem.
The user will then get an error message saying s/he has been blacklisted and
telling them your (the ERDDAP administrator's) email address.
Sometimes the user will contact you and you can resolve the problem.
Sometimes the user doesn't contact you and you will see the exact same behavior
coming from a different IP number the next day.
Blacklist the new IP number and hope that they will eventually get the message.
(Or this is your Groundhog Day, from which you will never escape. Sorry.)
The search engine companies use web crawlers (e.g., GoogleBot) to examine all of
the pages on the web to add the content to the search engines.
For ERDDAP, that is basically good.
ERDDAP has lots of links between pages, so the crawlers
will find all of the web pages and add them to the search engines.
Then, users of the search engines will be able to find datasets on your ERDDAP.
Unfortunately, some web crawlers (e.g., GoogleBot) are now filling out and submitting
forms in order to find additional content.
For web commerce sites, this is great.
But this is terrible for ERDDAP because it just leads to an infinite number of
undesirable and pointless attempts to crawl the actual data.
This can lead to more requests for data than from all other users combined.
And it fills the search engine with goofy, pointless subsets of the actual data.
To tell the web crawlers to stop filling out forms and just generally not
looking at web pages they don't need to look at, you need to create a text file called
in the root directory of your web site's document hierarchy so that it can be viewed by
anyone as, e.g., http://www.your.domain/