class Prism::Translation::Parser
This class is the entry-point for converting a prism syntax tree into the whitequark/parser gem’s syntax tree. It inherits from the base parser for the parser gem, and overrides the parse* methods to parse with prism and then translate.
Note that this version of the parser always parses using the latest version of Ruby syntax supported byPrism. If you want specific version support, use one of the version-specific subclasses, such asPrism::Translation::Parser34. If you want to parse using the same version of Ruby syntax as the currently running version of Ruby, usePrism::Translation::ParserCurrent.
Public Class Methods
Source
# File lib/prism/translation/parser.rb, line 74definitialize(builder =Prism::Translation::Parser::Builder.new,parser:Prism)if!builder.is_a?(Prism::Translation::Parser::Builder)warn(<<~MSG,uplevel:1,category::deprecated) [deprecation]: The builder passed to `Prism::Translation::Parser.new` is not a \ `Prism::Translation::Parser::Builder` subclass. This will raise in the next major version. MSGend@parser =parsersuper(builder)end
Thebuilder argument is used to create the parser using our custom builder class by default.
By using the:parser keyword argument, you can translate in a way that is compatible with theParser gem using any parser.
For example, in RuboCop for Ruby LSP, the following approach can be used to improve performance by reusing a pre-parsedPrism::ParseLexResult:
class PrismPreparsed def initialize(prism_result) @prism_result = prism_result end
defparse_lex(source,**options)@prism_resultend
end
prism_preparsed = PrismPreparsed.new(prism_result)
Prism::Translation::Ruby34.new(builder, parser: prism_preparsed)
In an object passed to the:parser keyword argument, theparse andparse_lex methods should be implemented as needed.
Public Instance Methods
Source
# File lib/prism/translation/parser.rb, line 91defdefault_encodingEncoding::UTF_8end
The default encoding for Ruby files is UTF-8.
Source
# File lib/prism/translation/parser.rb, line 99defparse(source_buffer)@source_buffer =source_buffersource =source_buffer.sourceoffset_cache =build_offset_cache(source)result =unwrap(@parser.parse(source,**prism_options),offset_cache)build_ast(result.value,offset_cache)ensure@source_buffer =nilend
Parses a source buffer and returns the AST.
Source
# File lib/prism/translation/parser.rb, line 112defparse_with_comments(source_buffer)@source_buffer =source_buffersource =source_buffer.sourceoffset_cache =build_offset_cache(source)result =unwrap(@parser.parse(source,**prism_options),offset_cache) [build_ast(result.value,offset_cache),build_comments(result.comments,offset_cache) ]ensure@source_buffer =nilend
Parses a source buffer and returns the AST and the source code comments.
Source
# File lib/prism/translation/parser.rb, line 129deftokenize(source_buffer,recover =false)@source_buffer =source_buffersource =source_buffer.sourceoffset_cache =build_offset_cache(source)result =beginunwrap(@parser.parse_lex(source,**prism_options),offset_cache)rescue::Parser::SyntaxErrorraiseif!recoverendprogram,tokens =result.valueast =build_ast(program,offset_cache)ifresult.success? [ast,build_comments(result.comments,offset_cache),build_tokens(tokens,offset_cache) ]ensure@source_buffer =nilend
Parses a source buffer and returns the AST, the source code comments, and the tokens emitted by the lexer.
Source
# File lib/prism/translation/parser.rb, line 155deftry_declare_numparam(node)node.children[0].match?(/\A_[1-9]\z/)end
Since prism resolves num params for us, we don’t need to support this kind of logic here.
Private Instance Methods
Source
# File lib/prism/translation/parser.rb, line 313defbuild_ast(program,offset_cache)program.accept(Compiler.new(self,offset_cache))end
Build the parser gem AST from the prism AST.
Source
# File lib/prism/translation/parser.rb, line 318defbuild_comments(comments,offset_cache)comments.mapdo|comment|::Parser::Source::Comment.new(build_range(comment.location,offset_cache))endend
Build the parser gem comments from the prism comments.
Source
# File lib/prism/translation/parser.rb, line 296defbuild_offset_cache(source)ifsource.bytesize==source.length-> (offset) {offset }elseoffset_cache = []offset =0source.each_chardo|char|char.bytesize.times {offset_cache<<offset }offset+=1endoffset_cache<<offsetendend
Prism deals with offsets in bytes, while the parser gem deals with offsets in characters. We need to handle this conversion in order to build the parser gem AST.
If the bytesize of the source is the same as the length, then we can just use the offset directly. Otherwise, we build an array where the index is the byte offset and the value is the character offset.
Source
# File lib/prism/translation/parser.rb, line 330defbuild_range(location,offset_cache)::Parser::Source::Range.new(source_buffer,offset_cache[location.start_offset],offset_cache[location.end_offset] )end
Build a range from a prism location.
Source
# File lib/prism/translation/parser.rb, line 325defbuild_tokens(tokens,offset_cache)Lexer.new(source_buffer,tokens,offset_cache).to_aend
Build the parser gem tokens from the prism tokens.
Source
Source
# File lib/prism/translation/parser.rb, line 174deferror_diagnostic(error,offset_cache)location =error.locationdiagnostic_location =build_range(location,offset_cache)caseerror.typewhen:argument_block_multiDiagnostic.new(:error,:block_and_blockarg, {},diagnostic_location, [])when:argument_formal_constantDiagnostic.new(:error,:argument_const, {},diagnostic_location, [])when:argument_formal_classDiagnostic.new(:error,:argument_cvar, {},diagnostic_location, [])when:argument_formal_globalDiagnostic.new(:error,:argument_gvar, {},diagnostic_location, [])when:argument_formal_ivarDiagnostic.new(:error,:argument_ivar, {},diagnostic_location, [])when:argument_no_forwarding_ampDiagnostic.new(:error,:no_anonymous_blockarg, {},diagnostic_location, [])when:argument_no_forwarding_starDiagnostic.new(:error,:no_anonymous_restarg, {},diagnostic_location, [])when:argument_no_forwarding_star_starDiagnostic.new(:error,:no_anonymous_kwrestarg, {},diagnostic_location, [])when:begin_lonely_elselocation =location.copy(length:4)diagnostic_location =build_range(location,offset_cache)Diagnostic.new(:error,:useless_else, {},diagnostic_location, [])when:class_name,:module_nameDiagnostic.new(:error,:module_name_const, {},diagnostic_location, [])when:class_in_methodDiagnostic.new(:error,:class_in_def, {},diagnostic_location, [])when:def_endless_setterDiagnostic.new(:error,:endless_setter, {},diagnostic_location, [])when:embdoc_termDiagnostic.new(:error,:embedded_document, {},diagnostic_location, [])when:incomplete_variable_class,:incomplete_variable_class_3_3location =location.copy(length:location.length+1)diagnostic_location =build_range(location,offset_cache)Diagnostic.new(:error,:cvar_name, {name:location.slice },diagnostic_location, [])when:incomplete_variable_instance,:incomplete_variable_instance_3_3location =location.copy(length:location.length+1)diagnostic_location =build_range(location,offset_cache)Diagnostic.new(:error,:ivar_name, {name:location.slice },diagnostic_location, [])when:invalid_variable_global,:invalid_variable_global_3_3Diagnostic.new(:error,:gvar_name, {name:location.slice },diagnostic_location, [])when:module_in_methodDiagnostic.new(:error,:module_in_def, {},diagnostic_location, [])when:numbered_parameter_ordinaryDiagnostic.new(:error,:ordinary_param_defined, {},diagnostic_location, [])when:numbered_parameter_outer_scopeDiagnostic.new(:error,:numparam_used_in_outer_scope, {},diagnostic_location, [])when:parameter_circularDiagnostic.new(:error,:circular_argument_reference, {var_name:location.slice },diagnostic_location, [])when:parameter_name_repeatDiagnostic.new(:error,:duplicate_argument, {},diagnostic_location, [])when:parameter_numbered_reservedDiagnostic.new(:error,:reserved_for_numparam, {name:location.slice },diagnostic_location, [])when:regexp_unknown_optionsDiagnostic.new(:error,:regexp_options, {options:location.slice[1..] },diagnostic_location, [])when:singleton_for_literalsDiagnostic.new(:error,:singleton_literal, {},diagnostic_location, [])when:string_literal_eofDiagnostic.new(:error,:string_eof, {},diagnostic_location, [])when:unexpected_token_ignoreDiagnostic.new(:error,:unexpected_token, {token:location.slice },diagnostic_location, [])when:write_target_in_methodDiagnostic.new(:error,:dynamic_const, {},diagnostic_location, [])elsePrismDiagnostic.new(error.message,:error,error.type,diagnostic_location)endend
Build a diagnostic from the given prism parse error.
Source
# File lib/prism/translation/parser.rb, line 339defprism_optionsoptions = {filepath:@source_buffer.name,version:convert_for_prism(version),partial_script:true, }# The parser gem always encodes to UTF-8, unless it is binary.# https://github.com/whitequark/parser/blob/v3.3.6.0/lib/parser/source/buffer.rb#L80-L107options[:encoding] =falseif@source_buffer.source.encoding!=Encoding::BINARYoptionsend
Options for how prism should parse/lex the source.
Source
# File lib/prism/translation/parser.rb, line 274defunwrap(result,offset_cache)result.errors.eachdo|error|nextunlessvalid_error?(error)diagnostics.process(error_diagnostic(error,offset_cache))endresult.warnings.eachdo|warning|nextunlessvalid_warning?(warning)diagnostic =warning_diagnostic(warning,offset_cache)diagnostics.process(diagnostic)ifdiagnosticendresultend
If there was a error generated during the parse, then raise an appropriate syntax error. Otherwise return the result.
Source
# File lib/prism/translation/parser.rb, line 163defvalid_error?(error)trueend
This is a hook to allow consumers to disable some errors if they don’t want them to block creating the syntax tree.
Source
# File lib/prism/translation/parser.rb, line 169defvalid_warning?(warning)trueend
This is a hook to allow consumers to disable some warnings if they don’t want them to block creating the syntax tree.
Source
# File lib/prism/translation/parser.rb, line 247defwarning_diagnostic(warning,offset_cache)diagnostic_location =build_range(warning.location,offset_cache)casewarning.typewhen:ambiguous_first_argument_plusDiagnostic.new(:warning,:ambiguous_prefix, {prefix:"+" },diagnostic_location, [])when:ambiguous_first_argument_minusDiagnostic.new(:warning,:ambiguous_prefix, {prefix:"-" },diagnostic_location, [])when:ambiguous_prefix_ampersandDiagnostic.new(:warning,:ambiguous_prefix, {prefix:"&" },diagnostic_location, [])when:ambiguous_prefix_starDiagnostic.new(:warning,:ambiguous_prefix, {prefix:"*" },diagnostic_location, [])when:ambiguous_prefix_star_starDiagnostic.new(:warning,:ambiguous_prefix, {prefix:"**" },diagnostic_location, [])when:ambiguous_slashDiagnostic.new(:warning,:ambiguous_regexp, {},diagnostic_location, [])when:dot_dot_dot_eolDiagnostic.new(:warning,:triple_dot_at_eol, {},diagnostic_location, [])when:duplicated_hash_key# skip, parser does this on its ownelsePrismDiagnostic.new(warning.message,:warning,warning.type,diagnostic_location)endend
Build a diagnostic from the given prism parse warning.