Task #3199 (closed)
Opened 14 years ago
Closed 13 years ago
Bug: "fastcgi" (non-tcp) doesn't seem to work
Reported by: | jamoore | Owned by: | cxallan |
---|---|---|---|
Priority: | minor | Milestone: | OMERO-Beta4.3 |
Component: | Deployment | Version: | n.a. |
Keywords: | n.a. | Cc: | cneves |
Resources: | n.a. | Referenced By: | n.a. |
References: | n.a. | Remaining Time: | n.a. |
Sprint: | 2010-12-23 (2) |
Description
bin/omero web server fastcgi-tcp bin/omero web start bin/omero web config nginx > tcp.conf # Manually edit tcp.conf. See #3198 nginx -c `pwd`/tcp.conf # Access via localhost:8000 works fine bin/omero web stop bin/omero web server fastcgi bin/omero config nginx > socket.conf nginx -c `pwd`/socket.conf # localhost:8000 hangs with "Waiting on localhost
$ netstat -na | grep 8000 tcp4 0 0 *.8000 *.* LISTEN $ pstree ... |-+- 15429 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefork | |--= 15430 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | |--= 15431 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | |--= 15432 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | |--= 15433 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | \--= 15434 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo |-+= 15459 moore nginx: master process nginx -c /Users/moore/code/git/dist/socket.conf | |--- 15460 moore nginx: worker process | |--- 15461 moore nginx: worker process | |--- 15462 moore nginx: worker process | |--- 15463 moore nginx: worker process | \--- 15464 moore nginx: worker process # And no errors in var/log/nginx_error.log
Change History (11)
comment:1 Changed 14 years ago by jburel
- Sprint changed from 2010-10-28 (18) to 2010-11-11 (19)
comment:2 Changed 14 years ago by cxallan
With a socket configuration I have no problems getting it to work.
pid /Users/callan/code/OMERO-TRUNK/dist/var/pid.nginx; error_log /Users/callan/code/OMERO-TRUNK/dist/var/log/nginx_error.log; worker_processes 5; working_directory /Users/callan/code/OMERO-TRUNK/dist/var; events { worker_connections 1024; } http { access_log /Users/callan/code/OMERO-TRUNK/dist/var/log/nginx_access.log; include /Users/callan/code/OMERO-TRUNK/dist/etc/mime.types; default_type application/octet-stream; client_body_temp_path /Users/callan/code/OMERO-TRUNK/dist/tmp; keepalive_timeout 65; server { listen 8000; server_name _; fastcgi_temp_path /Users/callan/code/OMERO-TRUNK/dist/var; proxy_temp_path /Users/callan/code/OMERO-TRUNK/dist/var; # weblitz django apps serve media from here location /appmedia { alias /Users/callan/code/OMERO-TRUNK/dist/lib/python/omeroweb/media; } # django's admin media location /media { alias /Users/callan/code/OMERO-TRUNK/dist/lib/python/omeroweb/media/django_admin_media; } location / { if (-f /Users/callan/code/OMERO-TRUNK/dist/var/maintenance.html) { error_page 503 /maintenance.html; return 503; } fastcgi_pass unix:/Users/callan/code/OMERO-TRUNK/dist/var/django_fcgi.sock; fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param QUERY_STRING $query_string; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SERVER_NAME $server_name; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param SERVER_PORT $server_port; fastcgi_pass_header Authorization; fastcgi_intercept_errors on; fastcgi_read_timeout 300; } location /maintenance.html { root /Users/callan/code/OMERO-TRUNK/dist/var; } } }
$ nginx -V nginx version: nginx/0.7.65 configure arguments: --prefix=/opt/local --with-cc-opt='-I/opt/local/include -O2' --with-ld-opt=-L/opt/local/lib --conf-path=/opt/local/etc/nginx/nginx.conf --error-log-path=/opt/local/var/log/nginx/error.log --http-log-path=/opt/local/var/log/nginx/access.log --pid-path=/opt/local/var/run/nginx/nginx.pid --lock-path=/opt/local/var/run/nginx/nginx.lock --http-client-body-temp-path=/opt/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/opt/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/opt/local/var/run/nginx/fastcgi_temp $ dist/bin/omero web server OMERO.web is configured to be served by 'fastcgi' $ ps auxww | grep manage.py | head -n 1 callan 27069 0.0 0.0 2477056 1556 ?? S 4:59pm 0:00.00 python manage.py runfcgi workdir=./ method=prefork socket=/Users/callan/code/OMERO-TRUNK/dist/var/django_fcgi.sock pidfile=/Users/callan/code/OMERO-TRUNK/dist/var/django.pid daemonize=true maxchildren=5 minspare=1 maxspare=5 maxrequests=400
comment:3 Changed 14 years ago by jmoore
Same version:
~/code/omero.git/dist $ nginx -V nginx version: nginx/0.7.65 configure arguments: --prefix=/opt/local --with-cc-opt='-I/opt/local/include -O2' --with-ld-opt=-L/opt/local/lib --conf-path=/opt/local/etc/nginx/nginx.conf --error-log-path=/opt/local/var/log/nginx/error.log --http-log-path=/opt/local/var/log/nginx/access.log --pid-path=/opt/local/var/run/nginx/nginx.pid --lock-path=/opt/local/var/run/nginx/nginx.lock --http-client-body-temp-path=/opt/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/opt/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/opt/local/var/run/nginx/fastcgi_temp
Same conf:
~/code/omero.git/dist $ perl -i -pe 's/callan.code.OMERO-TRUNK/moore\/GlencoeSoftware.git\/git\/omero.git/' chris.conf ~/code/omero.git/dist $ diff socket.conf chris.conf 61d60 <
Seems to be running:
~/code/omero.git/dist $ bin/omero web server fastcgi OMERO.web has been configured to be served by 'fastcgi' ~/code/omero.git/dist $ bin/omero web start Starting OMERO.web... [OK] ~/code/omero.git/dist $ ps auxww | grep -v grep | grep manage.py | head -n 1 moore 37424 0.0 0.0 2475192 1452 ?? S 6:09PM 0:00.00 /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefork socket=/Users/moore/GlencoeSoftware.git/git/omero.git/dist/var/django_fcgi.sock pidfile=/Users/moore/GlencoeSoftware.git/git/omero.git/dist/var/django.pid daemonize=true maxchildren=5 minspare=1 maxspare=5 maxrequests=400
- Going to http://localhost:8000 gives me the 404 just fine
- Clicking on "webclient" hangs.
Any other ideas what I can try? (I'll kick off a clean build now)
comment:4 Changed 14 years ago by jmoore
After a clean build I'm still getting the same hang on http://localhost:8000/webclient (also tried "syncmedia" :
~/code/omero.git $ cd dist ~/code/omero.git/dist $ bin/omero web server Please use 'web settings' first ~/code/omero.git/dist $ bin/omero web settings You just installed OMERO, which means you didn't have settings configured in OMERO.web. Please enter the domain you want to run OMERO.web on (http://www.domain.com:8000/):^CCancelled ~/code/omero.git/dist $ bin/omero web settings --force --domain=http://localhost:8000 --email=j@gs.com --smtphost=localhost --server=fastcgi --quiet You just installed OMERO, which means you didn't have settings configured in OMERO.web. Saved to /Users/moore/GlencoeSoftware.git/git/omero.git/dist/var/lib/custom_settings.py PYTHONPATH updated. ~/code/omero.git/dist $ bin/omero web server OMERO.web is configured to be served by 'fastcgi' ~/code/omero.git/dist $ bin/omero web start Starting OMERO.web... [OK] ~/code/omero.git/dist $ cp ../dist-old/chris.conf . ~/code/omero.git/dist $ nginx -c `pwd`/chris.conf [alert]: could not open error log file: open() "/opt/local/var/log/nginx/error.log" failed (13: Permission denied) ~/code/omero.git/dist $
~code/omero.git/dist $ pstree ... |-+- 74167 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefork | |--= 74168 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | |--= 74169 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | |--= 74170 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | |--= 74171 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo | \--= 74172 moore /opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python manage.py runfcgi workdir=./ method=prefo |-+= 74188 moore nginx: master process nginx -c /Users/moore/code/omero.git/dist/chris.conf | |--- 74189 moore nginx: worker process | |--- 74190 moore nginx: worker process | |--- 74191 moore nginx: worker process | |--- 74192 moore nginx: worker process | \--- 74193 moore nginx: worker process
~/code/omero.git/dist $ file /Users/moore/GlencoeSoftware.git/git/omero.git/dist/var/django_fcgi.sock /Users/moore/GlencoeSoftware.git/git/omero.git/dist/var/django_fcgi.sock: socket
~/code/omero.git/dist/var/log $ cat OMEROweb.log Fri, 29 Oct 2010 09:20:19 root INFO Application Starting... Fri, 29 Oct 2010 08:20:57 views-feedback ERROR handler404: Page not found Fri, 29 Oct 2010 08:20:57 views-feedback ERROR Traceback (most recent call last): File "/Users/moore/GlencoeSoftware.git/git/omero.git/dist/lib/python/django/core/handlers/base.py", line 83, in get_response request.path_info) File "/Users/moore/GlencoeSoftware.git/git/omero.git/dist/lib/python/django/core/urlresolvers.py", line 233, in resolve raise Resolver404, {'tried': tried, 'path': new_path} Resolver404: {'path': u'', 'tried': ['^favicon\\.ico$', '^appmedia/webgateway/(?P<path>.*)$', '^appmedia/webmobile/(?P<path>.*)$', '^appmedia/omeroweb/(?P<path>.*)$', '(?i)^webadmin/', '(?i)^webclient/', '(?i)^feedback/', '(?i)^webgateway/', '(?i)^webtest/', '(?i)^webemdb/', '(?i)^webmobile/']} Fri, 29 Oct 2010 08:20:59 webemdb WARNING Failed to import EMAN2. Some features of webemdb will not be supported. Fri, 29 Oct 2010 08:20:59 views-admin INFO INIT '74168' Fri, 29 Oct 2010 08:20:59 views-web INFO INIT '74168' Fri, 29 Oct 2010 08:21:02 views-admin INFO INIT '74169' Fri, 29 Oct 2010 08:21:03 views-web INFO INIT '74169'
~/code/omero.git/dist/var/log $ cat nginx_error.log ~/code/omero.git/dist/var/log $ cat nginx_access.log 127.0.0.1 - - [29/Oct/2010:09:20:59 +0200] "GET / HTTP/1.1" 404 1025 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10"
comment:5 Changed 14 years ago by jmoore
Just reproduced with the latest. Waiting until nginx times out shows this in nginx_error.log:
2010/11/01 13:49:03 [error] 86601#0: *4 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 127.0.0.1, server: _, request: "GET /webclient/ HTTP/1.1", upstream: "fastcgi://unix:/Users/moore/GlencoeSoftware.git/git/omero.git/dist/var/django_fcgi.sock:", host: "localhost:8000", referrer: "http://localhost:8000/webclient/login/?url=/webclient/"
comment:6 Changed 14 years ago by jmoore
When running with debug here's OMEROweb-DEBUG.log on "fastcgi":
Mon, 01 Nov 2010 14:46:29 root INFO Application Starting... Mon, 01 Nov 2010 13:46:40 omero.util.TempFileManager DEBUG Added file /Users/moore/.lock_testZizd1Q.tmp Mon, 01 Nov 2010 13:46:40 omero.util.TempFileManager DEBUG Chose gloabl tmpdir: /Users/moore Mon, 01 Nov 2010 13:46:40 omero.util.TempFileManager DEBUG Using temp dir: /Users/moore/omero/tmp/omero_moore/98991 Mon, 01 Nov 2010 13:46:40 webgateway DEBUG INIT Mon, 01 Nov 2010 13:46:40 views-admin INFO INIT '98991' Mon, 01 Nov 2010 13:46:40 views-web INFO INIT '98991' Mon, 01 Nov 2010 13:46:40 webgateway DEBUG trying stored connection Mon, 01 Nov 2010 13:46:40 webgateway DEBUG [] Mon, 01 Nov 2010 13:46:40 webgateway DEBUG creating new connection with "S:" (False) Mon, 01 Nov 2010 13:46:40 blitz_gateway DEBUG Connect attempt, sUuid=b83f7401-726d-4246-a742-2735f2f0fe11, group=None, self.sUuid=None Mon, 01 Nov 2010 13:46:40 blitz_gateway DEBUG connected? False var/log/OMEROweb-DEBUG.log (END)
and here's "fastcgi-tcp":
Mon, 01 Nov 2010 14:51:49 root INFO Application Starting... Mon, 01 Nov 2010 13:52:02 omero.util.TempFileManager DEBUG Added file /Users/moore/.lock_test4ErtCb.tmp Mon, 01 Nov 2010 13:52:02 omero.util.TempFileManager DEBUG Chose gloabl tmpdir: /Users/moore Mon, 01 Nov 2010 13:52:02 omero.util.TempFileManager DEBUG Using temp dir: /Users/moore/omero/tmp/omero_moore/887 Mon, 01 Nov 2010 13:52:02 webgateway DEBUG INIT Mon, 01 Nov 2010 13:52:02 views-admin INFO INIT '887' Mon, 01 Nov 2010 13:52:02 views-web INFO INIT '887' Mon, 01 Nov 2010 13:52:02 omero.util.TempFileManager DEBUG Added file /Users/moore/.lock_testTQPmjD.tmp Mon, 01 Nov 2010 13:52:02 omero.util.TempFileManager DEBUG Chose gloabl tmpdir: /Users/moore Mon, 01 Nov 2010 13:52:02 omero.util.TempFileManager DEBUG Using temp dir: /Users/moore/omero/tmp/omero_moore/886 Mon, 01 Nov 2010 13:52:02 webgateway DEBUG INIT Mon, 01 Nov 2010 13:52:02 views-admin INFO INIT '886' Mon, 01 Nov 2010 13:52:02 views-web INFO INIT '886' Mon, 01 Nov 2010 13:52:02 webgateway DEBUG trying stored connection Mon, 01 Nov 2010 13:52:02 webgateway DEBUG [] Mon, 01 Nov 2010 13:52:02 webgateway DEBUG creating new connection with "S:" (False) Mon, 01 Nov 2010 13:52:02 blitz_gateway DEBUG Connect attempt, sUuid=None, group=None, self.sUuid=None Mon, 01 Nov 2010 13:52:02 blitz_gateway DEBUG ## Creating proxies Mon, 01 Nov 2010 13:52:02 blitz_gateway DEBUG ## Reusing proxies Mon, 01 Nov 2010 13:52:02 blitz_gateway DEBUG ## - refreshing getAdminService Mon, 01 Nov 2010 13:52:02 blitz_gateway INFO created connection (uuid=b3a75606-3c4a-4375-aa41-0cf3ec576e5c) Mon, 01 Nov 2010 13:52:02 blitz_gateway DEBUG .. connected! Mon, 01 Nov 2010 13:52:07 webgateway DEBUG created new connection <webclient.webclient_gateway.OmeroWebGateway object at 0x10372e950> Mon, 01 Nov 2010 13:52:07 webgateway DEBUG [('S:b3a75606-3c4a-4375-aa41-0cf3ec576e5c#None', <webclient.webclient_gateway.OmeroWebGateway object at 0x10372e950>) ] Mon, 01 Nov 2010 13:52:07 webgateway DEBUG blitz session key: b3a75606-3c4a-4375-aa41-0cf3ec576e5c Mon, 01 Nov 2010 13:52:07 webgateway DEBUG stored as session.S:b3a75606-3c4a-4375-aa41-0cf3ec576e5c#None Mon, 01 Nov 2010 13:52:07 webemdb WARNING Failed to import EMAN2. Some features of webemdb will not be supported. Mon, 01 Nov 2010 13:52:07 webgateway DEBUG INIT Mon, 01 Nov 2010 13:52:07 webgateway DEBUG trying stored connection Mon, 01 Nov 2010 13:52:07 webgateway DEBUG [] Mon, 01 Nov 2010 13:52:07 webgateway DEBUG creating new connection with "S:" (False) Mon, 01 Nov 2010 13:52:07 blitz_gateway DEBUG Connect attempt, sUuid=b3a75606-3c4a-4375-aa41-0cf3ec576e5c, group=None, self.sUuid=None Mon, 01 Nov 2010 13:52:07 blitz_gateway DEBUG connected? False Mon, 01 Nov 2010 13:52:07 webgateway DEBUG trying stored connection Mon, 01 Nov 2010 13:52:07 webgateway DEBUG [('S:b3a75606-3c4a-4375-aa41-0cf3ec576e5c#None', <webclient.webclient_gateway.OmeroWebGateway object at 0x10372e950>) ] Mon, 01 Nov 2010 13:52:07 blitz_gateway DEBUG joinSession(b3a75606-3c4a-4375-aa41-0cf3ec576e5c) Mon, 01 Nov 2010 13:52:07 blitz_gateway DEBUG ## Creating proxies Mon, 01 Nov 2010 13:52:07 omero.util.TempFileManager DEBUG Added file /Users/moore/.lock_testnpkkWH.t ...
For some reason, the "fastcgi" case is doing far less. The major difference I can find is that one is on port 8000 and one is on port 8888. Is there some django cache issue going on?
comment:7 Changed 14 years ago by cxallan
Also tested to be working on Howe:
omero@howe:~$ nginx -V nginx version: nginx/0.7.65 TLS SNI support enabled configure arguments: --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/body --http-proxy-temp-path=/var/lib/nginx/proxy --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-debug --with-http_stub_status_module --with-http_flv_module --with-http_ssl_module --with-http_dav_module --with-http_gzip_static_module --with-http_realip_module --with-mail --with-mail_ssl_module --with-ipv6 --add-module=/build/buildd/nginx-0.7.65/modules/nginx-upstream-fair omero@howe:~$ cat nginx.conf pid /home/omero/OMERO-CURRENT/var/pid.nginx; error_log /home/omero/OMERO-CURRENT/var/log/nginx_error.log; worker_processes 5; working_directory /home/omero/OMERO-CURRENT/var; events { worker_connections 1024; } http { access_log /home/omero/OMERO-CURRENT/var/log/nginx_access.log; include /home/omero/OMERO-CURRENT/etc/mime.types; default_type application/octet-stream; client_body_temp_path /home/omero/OMERO-CURRENT/tmp; keepalive_timeout 65; server { listen 8080; server_name _; fastcgi_temp_path /home/omero/OMERO-CURRENT/var; proxy_temp_path /home/omero/OMERO-CURRENT/var; # weblitz django apps serve media from here location /appmedia { alias /home/omero/OMERO-CURRENT/lib/python/omeroweb/media; } # django's admin media location /media { alias /home/omero/OMERO-CURRENT/lib/python/omeroweb/media/django_admin_media; } location / { if (-f /home/omero/OMERO-CURRENT/var/maintenance.html) { error_page 503 /maintenance.html; return 503; } fastcgi_pass unix:/home/omero/OMERO-CURRENT/var/django_fcgi.sock; fastcgi_param PATH_INFO $fastcgi_script_name; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param QUERY_STRING $query_string; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SERVER_NAME $server_name; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param SERVER_PORT $server_port; fastcgi_pass_header Authorization; fastcgi_intercept_errors on; fastcgi_read_timeout 300; } location /maintenance.html { root /home/omero/OMERO-CURRENT/var; } } }
comment:8 Changed 14 years ago by jburel
- Sprint changed from 2010-11-11 (19) to 2020-11-25 (20)
Moved from sprint 2010-11-11 (19)
comment:9 Changed 13 years ago by jburel
- Milestone changed from OMERO-Beta4.2.1 to OMERO-Beta4.3
- Sprint changed from 2010-11-25 (20) to 2010-12-09 (21)
Moved from sprint 2010-11-25 (20)
comment:10 Changed 13 years ago by jburel
- Sprint changed from 2010-12-09 (21) to 2010-12-23 (22)
Moved from sprint 2010-12-09 (21)
comment:11 Changed 13 years ago by jburel
- Resolution set to fixed
- Status changed from new to closed
Closing cf. discussion during 09/12/10 meeting
Moved from sprint 2010-10-28 (18)