require 'base64' require 'helix_versioning_engine/change_service' require 'helix_versioning_engine/util' require 'sinatra/base' require 'json' module HelixVersioningEngine # Add 'file' methods # # 'files' are actually a combination of path metadata. There are three # major kinds of file resources: depots, dirs, and files. The true file # resources can be a combination of details, along with the file content. class App < Sinatra::Base # Special depots only variant to match no path get '/helix_versioning_engine/:api/files' do require_p4 p4 = env['p4'] path = params['path'] results = nil if path path = "//#{path}" unless path =~ %r{^//} results = p4.run_files(path) else results = p4.run_depots end results.to_json end # General browsing variation. # # Since we want this to be able to fetch file content in the case you # specify a file, versus a directory listing, we actually execute the # 'p4 files' command on the file. If we get a single result back, we then # add the base64'd content. If the file does not exist, we treat it like # a directory request. Thus, you'll never really get a 404, just an empty # array. get '/helix_versioning_engine/:api/files/*' do require_p4 p4 = env['p4'] dirs = params[:splat].select { |x| !x.empty? } Util.assert_no_special_paths(dirs) results = nil if dirs.empty? results = p4.run_depots else file_selector = '//' + dirs.join('/') files_results = nil p4.at_exception_level(P4::RAISE_NONE) do files_results = p4.run_files(file_selector) end files_results = [] unless files_results if files_results.length == 1 && files_results.first.key?('depotFile') && files_results.first['depotFile'] == file_selector results = files_results[0] print_results = p4.run_print(file_selector) # The first result is a Hash "header" # Then you receive byte buffers until you receive an empty string content = ''.b print_results.drop(1).each do |result| if result content << result end end results['Content'] = Base64.encode64(content) else # Treat request like a directory list selector = '//' + dirs.join('/') + '/*' files_results = p4.run_files('-e', selector) dirs_results = p4.run_dirs(selector) results = files_results + dirs_results end end results.to_json end # File upload mechanism # # This allows for multi-file patching based on particular directory level. # Because sinatra doesn't really support JSON array bodies, this must be # specified via a 'Files' parameter. If that parameter exists, we consider # this to be a directory upload. # # If this is a directory upload, we expect the following parameters on each # array object: # # - 'Content' - The base64 content # - 'DepotFile' - the *relative* path from the main splat # # Otherwise, we mostly just care about the 'Content' # fields for single file uploads. # # In both cases, a 'Description' field can be used to indicate a release # message. patch '/helix_versioning_engine/:api/files/*' do require_p4_with_temp_client p4 = env['p4'] client_root = env['p4_root'] client_name = p4.client path_parts = params[:splat].select { |x| !x.empty? } description = params['Description'] || 'Uploaded files' is_dir = params.key?('Files') Util.assert_no_special_paths(path_parts) files = nil if is_dir dir_root = "//#{path_parts.join('/')}" files = params['Files'].map do |f| { 'DepotFile' => "#{dir_root}/#{f['DepotFile']}", 'Content' => f['Content'] } end else files = [ { 'DepotFile' => "//#{path_parts.join('/')}", 'Content' => params[:Content] } ] end files.each { |f| f['Action'] = 'upload' } depot_paths = files.map { |f| f['DepotFile'] } change_service = ChangeService.new(p4: p4, client_root: client_root, client_name: client_name) files = files.map{ |f| ChangeService::File.from_json(f) } change_service.submit(files: files, description: description) '' end # Delete a single file. delete '/helix_versioning_engine/:api/files/*' do require_p4_with_temp_client p4 = env['p4'] description = params['Description'] || 'Deleting file' path_parts = params[:splat].select { |x| !x.empty? } Util.assert_no_special_paths(path_parts) file_path = "//#{path_parts.join('/')}" change_id = Util.init_changelist(p4, description) begin p4.run_sync(file_path) p4.run_delete('-c', change_id, file_path) p4.run_submit('-c', change_id) rescue StandardError => ex p4.at_exception_level(P4::RAISE_NONE) do p4.run_change('-d', '-f', change_id) if Util.error?(p4) puts "possible issues deleting change #{change_id}: " \ "#{p4.messages}" end end raise ex end '' end end end
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#1 | 15741 | ptomiak | Branch HWS for my use. | ||
//guest/perforce_software/helix-web-services/main/source/helix_web_services/lib/helix_versioning_engine/app/files.rb | |||||
#1 | 15622 | tjuricek |
Move source code to 'source/' subdirectory of branch. build/ will remain where it is. |
||
//guest/perforce_software/helix-web-services/main/helix_web_services/lib/helix_versioning_engine/app/files.rb | |||||
#5 | 15260 | tjuricek |
Fix truncation of content when listing a single file's details. Added message warning people about timeouts. This does work, and doesn't bring the system down, but you'll probably hit timeouts if you're trying to download files using this system. |
||
#4 | 15240 | tjuricek |
Set api level via request path on all Helix Versioning Engine methods. This will allow migration of applications to different P4D versions. Our internal methods (like project API) should attempt to handle backward compatibility similarly. P4WEBAPI-118 |
||
#3 | 15189 | tjuricek | Update files implementation, testing, and documentation. | ||
#2 | 15110 | tjuricek | Revise changes methods for new p4 connection handling, add server specs, remove model references in client, and update asciidoc documentation. | ||
#1 | 15032 | tjuricek |
Starting config and doc revisions. System is now broken while revisions underway. Configuration of the p4d connection is now done via a single HWSSettings middleware object injected into the Rack env. The HWSP4Cleanup middleware now cleans up any p4 injected into the Rack env. The Auth::App class now mostly just contains one method to generate a p4 ticket. /auth/v1/login. Added yard documentation for the main project. Yard docs have been reconfigured to dump into build/ directories. This should probably be done with each release. Hm... The top level rake file contains a task, 'all:doc', to update our documentation. This should probably be run for each checkin. Hm... Specs are now using Rack::Test on top of a 'live' p4d. I'd suggest you still use the p4util mechanism, which now dumps to a /tmp folder, so we can safely add P4IGNORE rules back into your local .p4config file. Old 'perforce' application now called 'helix_versioning_engine'. Removing cache data. Helix Sync may be slow. It may also get axed. We'll see. |
||
//guest/perforce_software/helix-web-services/main/helix_web_services/lib/perforce/app/files.rb | |||||
#3 | 13839 | tjuricek |
Conversion of the p4_project_service microservice to new monolithic system. This may not have an HTTP front end in the monolithic system. Project services are really just about how the core object model is structured. It's likely that each application will add their own wrinkles and extensions to the system, so it's unlikely we'll need a generic "project model". Exactly how extensions are registered and used is still a bit TBD at the moment. Previously they were to be registered webhooks, that model may change. Does not include tests yet. |
||
#2 | 13808 | tjuricek | Finish converting most of the core p4d API into the new monolithic system. | ||
#1 | 13799 | tjuricek |
Start with branch specs hosting in a new monolithic 'helix web services' project. Converting from a microservice to a monolithic architecture due to resource constraints at getting a deployable system running. Additionally, since it's not expected that people will upgrade often, the major benefit of microservices - being able to add services individually without affecting others - is not really a major benefit. The Ruby SDK will be consolidated into a single 'helix web services client' project. It may end up being distributed via Rubygems. This only runs branch specs at the moment. I want to get a CD pipeline setup for the monolithic server before revising more methods. |
||
//guest/perforce_software/helix-web-services/main/p4_web_api/p4_web_api/lib/p4_web_api/app/files.rb | |||||
#2 | 13416 | tjuricek |
Set up the p4_web_api related projects to be loaded by the parent Gemfile to make it a little easier to launch a test p4d with different seed data. Also fixed a couple of minor issues related to the client api. You'll notice that the 'rake api_reset api_init' task sets up a p4d for the p4_web_api tests, where the 'rake proj_reset proj_init' tasks set up p4d for project services testing. An example p4_web_api.ru that can be used for launching just the p4_web_api: require 'p4_web_api' # Do *not* let your development environment variables pollute how the p4 web api runs! ENV.keys.select { |k| k =~ /^P4/ }.each { |k| ENV.delete(k) } api = P4WebAPI::App.new puts 'changing paths' api.settings.token_path = '/Users/tjuricek/dev/p4ws/work/tokens' api.settings.workspace_folder = '/Users/tjuricek/dev/p4ws/work/workspaces' api.settings.p4 = {'port' => 'localhost:1666', 'charset' => 'auto'} run api |
||
#1 | 13412 | tjuricek |
Initial version of the web-services mainline. This is a collection of several projects, that will likely often get released together, though many of them may not always be relevant. See the README for more information. |