Rakefile #2

  • //
  • guest/
  • tjuricek/
  • p4ws/
  • Rakefile
  • Commits
# Change User Description Committed
#2 15791 tjuricek Removing very old p4ws branch
#1 13266 tjuricek Reorganized depot to avoid stream/main in path.

This may be re-imported.
//guest/tjuricek/p4ws/stream/main/Rakefile
#42 13261 tjuricek Don't use a temp 'dist' folder and include the file extensions in the package manifests.
#41 13255 tjuricek Allow the environment to set the changelist, which is generally true in CI systems.
#40 13254 tjuricek Revise package and gem versioning.

Packages will use [gem]-[changelist] as their versions. Gems will use a standard Ruby MAJOR.MINOR.REVISION format.

P4WEBAPI-64
#39 13241 tjuricek Remove commented out rake tasks that have been superceded by Salt.
#38 13240 tjuricek Add p4_phoenix_services package and Salt configuration for deployment.

This uncovered a couple of issues from the C++ API during it's conversion to C++03.

So, in a nutshell, most operations, except for notifications, appear to be working (well, using Vagrant machines).
#37 13237 tjuricek Added a basic p4-project-services .deb package

There is some kind of configuration issue with the production config that causes tests to fail. The service is running, however, so this is likely related to having a few things not managed via salt.
#36 13230 tjuricek Created a 'cluster' build procedure that creates an installer on build, and executes the install on a test instance.

The main change is to package all gem dependencies via 'vendor/cache' (using the 'bundle package' command).

Right now, there appears to be an issue with test data initialization, which may need a revised approach.
#35 13229 tjuricek Added a 'buildmaster' environment.

The 'buildmaster' sets up Ruby locally for doing some work via Rake.

Additional steps are now being figured out in the README.md, which may end up being a bash script that executes on the master.
#34 13223 tjuricek Update several files that were missing from the last commit regarding a basic 'test' environment
#33 13214 tjuricek The 'build-ubuntu12' seems to generate the appropriate debian packages for the distribution.

I had permissions issues (probably related to shared folder usage) with the dpkg tools. So I've removed them. If we happen to need them, we'll have to figure out a build process.

This environment probably shouldn't be used to test the packages, since omnibus likes to dirty everything up, leave files in the /opt/perforce directory owned by vagrant, etc.
#32 13210 tjuricek Reorganizing Vagrant definitions.

Each platform should allow developers to launch environments for development, builds, or 'production', so the entire CD process can be evaluated by anyone wiht access to the source tree.

This may eventually bring up a transient vSphere machine up and down, which may be necessary for sandboxing Omnibus related builds.
#31 13199 tjuricek Add zookeeper service to cluster for development evaluation.

It's a little unclear exactly what values need to be distributed via zk, since a lot of the configuration may end up being handled via ports. I wanted to at least provide a way to evaluate this.

P4WEBAPI-60
#30 13198 tjuricek Added hsm:startworld and hsm:stopworld tasks to do global starts and stops.

This is not without apparent issues. It appears bundler or ruby is pausing before the nginx launch task happens. I'm not sure if there's some kind of resource issue first.

P4WEBAPI-49
#29 13195 tjuricek Wrapping issues with setting up :test namespace and ignoring for now.
#28 13194 tjuricek Adding back the :build task as :publish, ignoring gems for the moment.

gems may eventually be archived again, but right now that's not needed for basic integration.
#27 13191 tjuricek Tests for the p4 web api and p4 project services now pass against a development setup both in and out of the docker cluster.

Note that configuration *has not* been finalized, so conventions to dealing with development vs production need to be organized a bit.
#26 13190 tjuricek Creating several Rake tasks for our "hsm" configuration that are based on a Ruby API config file.

The config file points to machines and services in the system which then takes care of setting up the docker-machine environment and docker-compose configuration.
#25 13187 tjuricek More docker-compose configuration for projects in research for HSM.

Added a geminabox host for the cluster, which allows for quick rebuilds. Each cluster will cache the gems it needs, and publish as well. We may want to put the overwrite commands for all publish steps by default.

Note: referencing "external links" can only be done 'at runtime', so "bundle install" really needs to be run in the context of 'docker-compose up' not 'docker-compose build'. This was edited in all configured libraries so far.

Finally, added the p4_project_services projects to configuration, which is running under it's own puma instance.

The postgres instance is a little tricky to figure out the exact workflow. Right now I'm using the host rake db:migrate task and calling createdb manually, since we need to pass the password along.
#24 13184 tjuricek Corrected regressions that broke the API and Project services specs.
#23 13183 tjuricek Change the name of our ':pg' rake tasks to just ':db' for some consistency.

Fixed a few initial schemas to normalize on using integers for primary keys. I may need to come up with some constraints for project IDs, we'll see.
#22 13182 tjuricek Implementation of the phoenix services side of notification handling.

This is just implementation and work-in-progress.

Phoenix projects will now have paths cached in the phoenix services process, which we'll use to "guess" what file changes affect which Phoenix project. We basically see a changed path, then see that it might be relevant to one of the Phoenix project stream views, and then issue a "p4 changes -m1 //stream..." to see what the last change number is. If the change number goes up, we trigger an update.

Note that how this all gets configured is with an account from notification services to phoenix services, which ideally is some kind of system account that sees all relevant files. Otherwise, you'll likely get changes filtered by protections, and thus, updates may not get sent out.
#21 13180 tjuricek Phoenix notification services, client API, including new phoenix_updater

This is an interim commit containing a first pass implementation of the phoenix_updater. Notably missing parts:

- The Qt API doesn't yet actually interact with the phoenix_updater
- The phoenix_services web service doesn't filter out notifications

I *may* end up creating another web application *just* to filter out notifications, since this may end up taking up a lot of background workers.
#20 13179 tjuricek Initial implementation of change-commit notification services.

Set up a new database just for storing webhook configuration for the notification services.

It uses the stable version of Resque, which needs to be evaluated in more detail. This should be easier for an admin to monitor than something like Sidekiq, but performance evaluation needs to happen before production release.

No automated tests yet, that will rely upon a Phoenix notification mechanism existing first.
#19 13178 tjuricek Set up resque worker that will trigger web hooks in the background.
#18 13177 tjuricek Add a basic notification_services endpoint that can setup a trigger entry and accept basic POST requests.
#17 13173 tjuricek Replace crappy indexing mechanism with Postgres queries.
#16 13172 tjuricek Created a preliminary caching schema and basic database models for the p4 project services.

This may break some tests momentarily until I implement caching.
#15 13168 tjuricek Revising P4 Web API docbook documentation to become the Perforce Web Services guide.

Right now this is just focused on the Qt SDK. The remaining protocol documentation, etc, will happen eventually.
#14 13153 tjuricek Reorganized the Qt libraries into a separate part of the tree, to make it easier for CMake configuration.

Added 'qt_build' and 'qt_test' tasks to at least execute tests and fail the build if they don't pass.

Started the implementation of the project library interaction.
Change: 1018651

Date: 3/4/15 11:43 AM

Client: tjuricek_dhcp-141-n100_5959

User: tjuricek

Status: pending

Type: public

Description:
Reorganized the Qt libraries into a separate part of the tree, to make it easier for CMake configuration.

Added 'qt_build' and 'qt_test' tasks to at least execute tests and fail the build if they don't pass.

Started the implementation of the project library interaction.


JobStatus:

Jobs:


Files:
//web-services/p4ws-main/Rakefile
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/CMakeLists.txt
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/PhoenixIntegrationTests.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/README
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix.h
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/PhoenixProject.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/PhoenixProject.h
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/PhoenixServicesClient.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/PhoenixServicesClient.h
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/RequestError.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/RequestError.h
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/Session.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/phoenix/Session.h
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/test/PhoenixServicesClientTests.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/test/PhoenixServicesClientTests.h
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/test/SessionTests.cpp
//web-services/p4ws-main/p4_phoenix_services/clients/qt/p4_phoenix_services_client/test/SessionTests.h
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/CMakeLists.txt
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services.h
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services/Branch.cpp
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services/Branch.h
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services/Project.cpp
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services/Project.h
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services/View.cpp
//web-services/p4ws-main/p4_project_services/clients/qt/p4_project_services_client/p4_project_services/View.h
//web-services/p4ws-main/qt/CMakeLists.txt
//web-services/p4ws-main/qt/p4_phoenix_services_client/CMakeLists.txt
//web-services/p4ws-main/qt/p4_phoenix_services_client/PhoenixIntegrationTests.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/README
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix.h
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/PhoenixProject.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/PhoenixProject.h
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/PhoenixServicesClient.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/PhoenixServicesClient.h
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/RequestError.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/RequestError.h
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/Session.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/phoenix/Session.h
//web-services/p4ws-main/qt/p4_phoenix_services_client/test/PhoenixServicesClientTests.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/test/PhoenixServicesClientTests.h
//web-services/p4ws-main/qt/p4_phoenix_services_client/test/SessionTests.cpp
//web-services/p4ws-main/qt/p4_phoenix_services_client/test/SessionTests.h
//web-services/p4ws-main/qt/p4_project_services_client/CMakeLists.txt
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeCache.txt
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CMakeCCompiler.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CMakeCXXCompiler.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CMakeDetermineCompilerABI_C.bin
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CMakeDetermineCompilerABI_CXX.bin
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CMakeSystem.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CompilerIdC/CMakeCCompilerId.c
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CompilerIdC/a.out
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CompilerIdCXX/CMakeCXXCompilerId.cpp
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/2.8.12/CompilerIdCXX/a.out
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/CMakeDirectoryInformation.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/CMakeOutput.log
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/Makefile.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/Makefile2
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/TargetDirectories.txt
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/cmake.check_cache
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/DependInfo.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/build.make
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/cmake_clean.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/cmake_clean_target.cmake
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/depend.make
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/flags.make
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/link.txt
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/p4_project_services_client.dir/progress.make
//web-services/p4ws-main/qt/p4_project_services_client/build/CMakeFiles/progress.marks
//web-services/p4ws-main/qt/p4_project_services_client/build/Makefile
//web-services/p4ws-main/qt/p4_project_services_client/build/cmake_install.cmake
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services.h
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services/Branch.cpp
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services/Branch.h
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services/Project.cpp
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services/Project.h
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services/View.cpp
//web-services/p4ws-main/qt/p4_project_services_client/p4_project_services/View.h
#13 13149 tjuricek Added a *very trivial* logIn implementation.

This just makes the *current* POST request to /p4_phoenix_services/v1/sessions, which *does not* include the ability to create a host locked P4 ticket for the user. So it's not quite usable yet. But, I've made several tweaks to the API which should be stablizing.
#12 13143 tjuricek Restructure 'builds' to be blessed versions of things to be used by clients.

The gems/ folder contains all the ruby distributables. Eventually, these will likely be published to Rubygems.

The other folders are source projects that can be included into dependent streams. These are 'versioned' against the last build that we ideally run some integration tests against.
#11 13141 tjuricek Try out chmod before updating the build.
#10 13139 tjuricek See if I can read directly from the source file.
#9 13138 tjuricek Add steps to create 'build/' output directory for automated builds.

This does require a 'p4' command line client in order to grab the latest changelist number, which may/may not be easily setup.
#8 13137 tjuricek Put retry logic for service start that depends upon curl.
#7 13136 tjuricek Added top level rake build task for automated builds.
#6 13130 tjuricek Setup the main directory to launch specs in subprojects and a unicorn-based daemon via rake.

See rake -T for details, but this should allow for CI builds to collect output.
#5 13129 tjuricek The p4_project_services specs now run that provide basic create-read ability.
#4 13128 tjuricek Added p4_project_services to development config.ru, and reconfigured p4 web api client tests to use the new paths.
#3 13127 tjuricek Added a 'rake work/config.ru' task, that creates a local rack configuration.

This now includes the project services, which is not yet mounted. The p4 web api tests still pass though.
#2 13126 tjuricek Set up the p4_web_api related projects to be loaded by the parent Gemfile to make it a little easier to launch a test p4d with different seed data.

Also fixed a couple of minor issues related to the client api.

You'll notice that the 'rake api_reset api_init' task sets up a p4d for the p4_web_api tests, where the 'rake proj_reset proj_init' tasks set up p4d for project services testing.

An example p4_web_api.ru that can be used for launching just the p4_web_api:

require 'p4_web_api'

# Do *not* let your development environment variables pollute how the p4 web api runs!
ENV.keys.select { |k| k =~ /^P4/ }.each { |k| ENV.delete(k) }

api = P4WebAPI::App.new
puts 'changing paths'
api.settings.token_path = '/Users/tjuricek/dev/p4ws/work/tokens'
api.settings.workspace_folder = '/Users/tjuricek/dev/p4ws/work/workspaces'
api.settings.p4 = {'port' => 'localhost:1666', 'charset' => 'auto'}

run api
#1 13125 tjuricek Add some basic utility methods to the top level.